From rkennke at redhat.com Sat Dec 1 11:32:46 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 1 Dec 2018 12:32:46 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <15fad536-763a-6000-ad2b-5bc5643a047d@redhat.com> Hi Coleen, thanks a lot for the review! Roman > > Hi, I looked at the runtime changes, which are very few, since you've > upstreamed most of the runtime dependent changes already. > They look good to me. > Thanks, > Coleen > > On 11/30/18 4:00 PM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> > From vladimir.kozlov at oracle.com Sun Dec 2 00:15:21 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sat, 1 Dec 2018 16:15:21 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> Much better. Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp ? Why not include only shenandoahSupport.hpp when new nodes are defined? I think it is fine to not use #ifdef in loopopts.cpp when you check is_ShenandoahBarrier(). And you don't do that in other files. Code in opto/macro.cpp is ugly but it is only the place so we can live with it I think. Thanks, Vladimir On 11/30/18 1:00 PM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From rkennke at redhat.com Sun Dec 2 09:57:35 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 2 Dec 2018 10:57:35 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> Message-ID: <5aecf696-a125-597c-c02b-5d4da8d453a0@redhat.com> Hi Vladimir, > Much better. Thanks! > Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp > ? Why not include only shenandoahSupport.hpp when new nodes are defined? For no particular reason, I guess. I'll fix it in next round. > I think it is fine to not use #ifdef in loopopts.cpp when you check > is_ShenandoahBarrier(). And you don't do that in other files. > > Code in opto/macro.cpp is ugly but it is only the place so we can live > with it I think. Ok, thanks! We'll see if we can improve macro.cpp. Thanks, Roman > Thanks, > Vladimir > > On 11/30/18 1:00 PM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> From rkennke at redhat.com Sun Dec 2 16:55:38 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 2 Dec 2018 17:55:38 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> Message-ID: <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> Hello Vladimir, > Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp > ? Why not include only shenandoahSupport.hpp when new nodes are defined? I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to be the entry point for external C2 code. No C2 code is supposed to include shenandoahSupport.hpp, this is just an implementation detail of shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? Cheers, Roman > > I think it is fine to not use #ifdef in loopopts.cpp when you check > is_ShenandoahBarrier(). And you don't do that in other files. > > Code in opto/macro.cpp is ugly but it is only the place so we can live > with it I think. > > Thanks, > Vladimir > > On 11/30/18 1:00 PM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> From vladimir.kozlov at oracle.com Sun Dec 2 18:10:16 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sun, 2 Dec 2018 10:10:16 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> Message-ID: On 12/2/18 8:55 AM, Roman Kennke wrote: > Hello Vladimir, > >> Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp >> ? Why not include only shenandoahSupport.hpp when new nodes are defined? > > I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to > be the entry point for external C2 code. No C2 code is supposed to > include shenandoahSupport.hpp, this is just an implementation detail of > shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes > zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? Yes, it is fine. Vladimir > > Cheers, > Roman > > >> >> I think it is fine to not use #ifdef in loopopts.cpp when you check >> is_ShenandoahBarrier(). And you don't do that in other files. >> >> Code in opto/macro.cpp is ugly but it is only the place so we can live >> with it I think. >> >> Thanks, >> Vladimir >> >> On 11/30/18 1:00 PM, Roman Kennke wrote: >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>>> now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the >>>>>> mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>> is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> >>>>>> which is what I intend to commit (and which should be equivalent to >>>>>> the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ??*) shenandoah-gc >>>>>> >>>>>> >>>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ??*) shared-gc >>>>>> >>>>>> >>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>> assert not >>>>>> fail (next to CMS and G1) >>>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ??*) shared-serviceability >>>>>> >>>>>> >>>>>> ???? - The usual code to support another GC >>>>>> >>>>>> ??*) shared-runtime >>>>>> >>>>>> >>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>> iterators to >>>>>> hook up with, >>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ???? - The usual macros in macro.hpp >>>>>> >>>>>> ??*) shared-build >>>>>> >>>>>> >>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>> source >>>>>> files, which is >>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>> significant >>>>>> regression if we >>>>>> ?????? don't) >>>>>> >>>>>> ??*) shared-tests >>>>>> >>>>>> >>>>>> ???? - Test infrastructure to support Shenandoah >>>>>> ???? - Shenandoah test groups >>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>> selected GC >>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>> sense to >>>>>> run with it >>>>>> >>>>>> ??*) shenandoah-tests >>>>>> >>>>>> >>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>> subdirectory >>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in >>>>>> case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ?? - {Windows} x {x86_64}, >>>>>> ?? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ?? - the new Shenandoah tests >>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>> deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>> interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> > From rkennke at redhat.com Sun Dec 2 18:15:15 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 2 Dec 2018 19:15:15 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> Message-ID: <75392bec-8157-2fb2-51d0-f50fd51bbc6b@redhat.com> >> Hello Vladimir, >> >>> Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp >>> ? Why not include only shenandoahSupport.hpp when new nodes are defined? >> >> I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to >> be the entry point for external C2 code. No C2 code is supposed to >> include shenandoahSupport.hpp, this is just an implementation detail of >> shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes >> zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? > > Yes, it is fine. Ok great! Thanks! Can I consider the shared-compiler part reviewed by you then? Roman > > Vladimir > >> >> Cheers, >> Roman >> >> >>> >>> I think it is fine to not use #ifdef in loopopts.cpp when you check >>> is_ShenandoahBarrier(). And you don't do that in other files. >>> >>> Code in opto/macro.cpp is ugly but it is only the place so we can live >>> with it I think. >>> >>> Thanks, >>> Vladimir >>> >>> On 11/30/18 1:00 PM, Roman Kennke wrote: >>>> Hi all, >>>> >>>> here comes round 4 of Shenandoah upstreaming review: >>>> >>>> This includes fixes for the issues that Per brought up: >>>> - Verify and gracefully reject dangerous flags combinations that >>>> disables required barriers >>>> - Revisited @requires filters in tests >>>> - Trim unused code from Shenandoah's SA impl >>>> - Move ShenandoahGCTracer to gc/shenandoah >>>> - Fix ordering of GC names in various files >>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>> >>>> Thanks everybody for taking time to review this! >>>> Roman >>>> >>>>> Hello all, >>>>> >>>>> Thanks so far for all the reviews and support! >>>>> >>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>> Also, I fixed the numbering of my webrevs to match the >>>>> review-round. ;-) >>>>> >>>>> Things we've changed today: >>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>> requires an addition in build machinery though, see >>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>> - Improved zero-disabling and build-code-simplification as >>>>> suggested by >>>>> Magnus and Per >>>>> - Cleaned up some leftovers in C2 >>>>> - Improved C2 loop opts code by introducing another APIs in >>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>>>> now. >>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>> noted earlier. This stuff is Shenandoah-specific, so let's just >>>>> call it >>>>> that. >>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, >>>>> etc) >>>>> - Rebased on jdk-12+22 >>>>> >>>>> - Question: let us know if you need separate RFE for the new BSC2 >>>>> APIs? >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>>> Alright, we fixed: >>>>>> - The minor issues that Kim reported in shared-gc >>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>> >>>>>> Some notes: >>>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>> (expectedly) failing. It can run CMS, default GC and any other GC >>>>>> just >>>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>>> filters >>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>> >>>>>> >>>>>> >>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>> >>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>> those with ZGC? >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>> >>>>>> >>>>>> >>>>>> (Note to Leonid and tests reviewers: I'll add those related >>>>>> filters in >>>>>> next round). >>>>>> >>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code >>>>>> look >>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>> the code and see what is ok, and especially what is not ok, so >>>>>> that we >>>>>> can focus our efforts on the relevant parts? >>>>>> >>>>>> Updated set of webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>> mainline. >>>>>>> I divided the review into parts that roughly correspond to the >>>>>>> mailing lists >>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>> changes and >>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>>> is to >>>>>>> eventually >>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>> >>>>>>> JEP: >>>>>>> ??? https://openjdk.java.net/jeps/189 >>>>>>> Bug entry: >>>>>>> >>>>>>> ???https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>> >>>>>>> Webrevs: >>>>>>> ??? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>> >>>>>>> For those who want to see the full change, have a look at the >>>>>>> shenandoah-complete >>>>>>> >>>>>>> >>>>>>> >>>>>>> directory, >>>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>>> file >>>>>>> shenandoah-master.patch >>>>>>> , >>>>>>> >>>>>>> >>>>>>> which is what I intend to commit (and which should be equivalent to >>>>>>> the >>>>>>> 'shenandoah-complete' webrev). >>>>>>> >>>>>>> Sections to review (at this point) are the following: >>>>>>> ???*) shenandoah-gc >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Actual Shenandoah implementation, almost completely >>>>>>> residing in >>>>>>> gc/shenandoah >>>>>>> >>>>>>> ???*) shared-gc >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - This is mostly boilerplate that is common to any GC >>>>>>> ????? - referenceProcessor.cpp has a little change to make one >>>>>>> assert not >>>>>>> fail (next to CMS and G1) >>>>>>> ????? - taskqueue.hpp has some small adjustments to enable >>>>>>> subclassing >>>>>>> >>>>>>> ???*) shared-serviceability >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - The usual code to support another GC >>>>>>> >>>>>>> ???*) shared-runtime >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - A number of friends declarations to allow Shenandoah >>>>>>> iterators to >>>>>>> hook up with, >>>>>>> ??????? e.g. ClassLoaderData, CodeCache, etc >>>>>>> ????? - Warning and disabling JFR LeakProfiler >>>>>>> ????? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>> Shenandoah C2 optimizations >>>>>>> ????? - Locks initialization in mutexLocker.cpp as usual >>>>>>> ????? - VM operations defines for Shenandoah's VM ops >>>>>>> ????? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>> Shenandoah's logging >>>>>>> ????? - The usual macros in macro.hpp >>>>>>> >>>>>>> ???*) shared-build >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Add shenandoah feature, enabled by default, as agreed with >>>>>>> Vladimir K. beforehand >>>>>>> ????? - Some flags for shenandoah-enabled compilation to get >>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>> ??????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>> Shenandoah's barriers >>>>>>> ????? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>>> source >>>>>>> files, which is >>>>>>> ??????? useful to get the whole marking loop inlined (observed >>>>>>> significant >>>>>>> regression if we >>>>>>> ??????? don't) >>>>>>> >>>>>>> ???*) shared-tests >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Test infrastructure to support Shenandoah >>>>>>> ????? - Shenandoah test groups >>>>>>> ????? - Exclude Shenandoah in various tests that can be run with >>>>>>> selected GC >>>>>>> ????? - Enable/add configure for Shenandoah for tests that make >>>>>>> sense to >>>>>>> run with it >>>>>>> >>>>>>> ???*) shenandoah-tests >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>>> subdirectory >>>>>>> ????? - A couple of tests configurations have been added, e.g. >>>>>>> TestGCBasherWithShenandoah.java >>>>>>> >>>>>>> I intentionally left out shared-compiler for now, because we have >>>>>>> some >>>>>>> work left to do >>>>>>> there, but if you click around you'll find the patch anyway, in >>>>>>> case you >>>>>>> want to take >>>>>>> a peek at it. >>>>>>> >>>>>>> We have regular builds on: >>>>>>> ??? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>> ??? - {Windows} x {x86_64}, >>>>>>> ??? - {MacOS X} x {x86_64} >>>>>>> >>>>>>> This also routinely passes: >>>>>>> ??? - the new Shenandoah tests >>>>>>> ??? - jcstress with/without aggressive Shenandoah verification >>>>>>> ??? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>> >>>>>>> >>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>>> deserves >>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>> advice and support, my collegues in Oracle's GC, runtime and >>>>>>> compiler >>>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>>> interface and >>>>>>> related changes, and of course the many early adopters for reporting >>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>> without any of you! >>>>>>> >>>>>>> Best regards, >>>>>>> Roman >>>>>>> >>>>>> >>>>> >>>> >> From vladimir.kozlov at oracle.com Sun Dec 2 20:37:31 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sun, 2 Dec 2018 12:37:31 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <75392bec-8157-2fb2-51d0-f50fd51bbc6b@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> <75392bec-8157-2fb2-51d0-f50fd51bbc6b@redhat.com> Message-ID: Yes, compiler changes reviewed. Vladimir On 12/2/18 10:15 AM, Roman Kennke wrote: >>> Hello Vladimir, >>> >>>> Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp >>>> ? Why not include only shenandoahSupport.hpp when new nodes are defined? >>> >>> I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to >>> be the entry point for external C2 code. No C2 code is supposed to >>> include shenandoahSupport.hpp, this is just an implementation detail of >>> shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes >>> zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? >> >> Yes, it is fine. > > Ok great! Thanks! > > Can I consider the shared-compiler part reviewed by you then? > > Roman > > >> >> Vladimir >> >>> >>> Cheers, >>> Roman >>> >>> >>>> >>>> I think it is fine to not use #ifdef in loopopts.cpp when you check >>>> is_ShenandoahBarrier(). And you don't do that in other files. >>>> >>>> Code in opto/macro.cpp is ugly but it is only the place so we can live >>>> with it I think. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/30/18 1:00 PM, Roman Kennke wrote: >>>>> Hi all, >>>>> >>>>> here comes round 4 of Shenandoah upstreaming review: >>>>> >>>>> This includes fixes for the issues that Per brought up: >>>>> - Verify and gracefully reject dangerous flags combinations that >>>>> disables required barriers >>>>> - Revisited @requires filters in tests >>>>> - Trim unused code from Shenandoah's SA impl >>>>> - Move ShenandoahGCTracer to gc/shenandoah >>>>> - Fix ordering of GC names in various files >>>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>>> >>>>> Thanks everybody for taking time to review this! >>>>> Roman >>>>> >>>>>> Hello all, >>>>>> >>>>>> Thanks so far for all the reviews and support! >>>>>> >>>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>>> Also, I fixed the numbering of my webrevs to match the >>>>>> review-round. ;-) >>>>>> >>>>>> Things we've changed today: >>>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>>> requires an addition in build machinery though, see >>>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>>> - Improved zero-disabling and build-code-simplification as >>>>>> suggested by >>>>>> Magnus and Per >>>>>> - Cleaned up some leftovers in C2 >>>>>> - Improved C2 loop opts code by introducing another APIs in >>>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>>>>> now. >>>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>>> noted earlier. This stuff is Shenandoah-specific, so let's just >>>>>> call it >>>>>> that. >>>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, >>>>>> etc) >>>>>> - Rebased on jdk-12+22 >>>>>> >>>>>> - Question: let us know if you need separate RFE for the new BSC2 >>>>>> APIs? >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>>> Alright, we fixed: >>>>>>> - The minor issues that Kim reported in shared-gc >>>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>>> >>>>>>> Some notes: >>>>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>>> (expectedly) failing. It can run CMS, default GC and any other GC >>>>>>> just >>>>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>>>> filters >>>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>>> >>>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>>> those with ZGC? >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> (Note to Leonid and tests reviewers: I'll add those related >>>>>>> filters in >>>>>>> next round). >>>>>>> >>>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code >>>>>>> look >>>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>>> the code and see what is ok, and especially what is not ok, so >>>>>>> that we >>>>>>> can focus our efforts on the relevant parts? >>>>>>> >>>>>>> Updated set of webrevs: >>>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Roman >>>>>>> >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>>> mainline. >>>>>>>> I divided the review into parts that roughly correspond to the >>>>>>>> mailing lists >>>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>>> changes and >>>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>>>> is to >>>>>>>> eventually >>>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>>> >>>>>>>> JEP: >>>>>>>> ??? https://openjdk.java.net/jeps/189 >>>>>>>> Bug entry: >>>>>>>> >>>>>>>> ???https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>>> >>>>>>>> Webrevs: >>>>>>>> ??? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>>> >>>>>>>> For those who want to see the full change, have a look at the >>>>>>>> shenandoah-complete >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> directory, >>>>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>>>> file >>>>>>>> shenandoah-master.patch >>>>>>>> , >>>>>>>> >>>>>>>> >>>>>>>> which is what I intend to commit (and which should be equivalent to >>>>>>>> the >>>>>>>> 'shenandoah-complete' webrev). >>>>>>>> >>>>>>>> Sections to review (at this point) are the following: >>>>>>>> ???*) shenandoah-gc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Actual Shenandoah implementation, almost completely >>>>>>>> residing in >>>>>>>> gc/shenandoah >>>>>>>> >>>>>>>> ???*) shared-gc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - This is mostly boilerplate that is common to any GC >>>>>>>> ????? - referenceProcessor.cpp has a little change to make one >>>>>>>> assert not >>>>>>>> fail (next to CMS and G1) >>>>>>>> ????? - taskqueue.hpp has some small adjustments to enable >>>>>>>> subclassing >>>>>>>> >>>>>>>> ???*) shared-serviceability >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - The usual code to support another GC >>>>>>>> >>>>>>>> ???*) shared-runtime >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - A number of friends declarations to allow Shenandoah >>>>>>>> iterators to >>>>>>>> hook up with, >>>>>>>> ??????? e.g. ClassLoaderData, CodeCache, etc >>>>>>>> ????? - Warning and disabling JFR LeakProfiler >>>>>>>> ????? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>>> Shenandoah C2 optimizations >>>>>>>> ????? - Locks initialization in mutexLocker.cpp as usual >>>>>>>> ????? - VM operations defines for Shenandoah's VM ops >>>>>>>> ????? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>>> Shenandoah's logging >>>>>>>> ????? - The usual macros in macro.hpp >>>>>>>> >>>>>>>> ???*) shared-build >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Add shenandoah feature, enabled by default, as agreed with >>>>>>>> Vladimir K. beforehand >>>>>>>> ????? - Some flags for shenandoah-enabled compilation to get >>>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>>> ??????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>>> Shenandoah's barriers >>>>>>>> ????? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>>>> source >>>>>>>> files, which is >>>>>>>> ??????? useful to get the whole marking loop inlined (observed >>>>>>>> significant >>>>>>>> regression if we >>>>>>>> ??????? don't) >>>>>>>> >>>>>>>> ???*) shared-tests >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Test infrastructure to support Shenandoah >>>>>>>> ????? - Shenandoah test groups >>>>>>>> ????? - Exclude Shenandoah in various tests that can be run with >>>>>>>> selected GC >>>>>>>> ????? - Enable/add configure for Shenandoah for tests that make >>>>>>>> sense to >>>>>>>> run with it >>>>>>>> >>>>>>>> ???*) shenandoah-tests >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>>>> subdirectory >>>>>>>> ????? - A couple of tests configurations have been added, e.g. >>>>>>>> TestGCBasherWithShenandoah.java >>>>>>>> >>>>>>>> I intentionally left out shared-compiler for now, because we have >>>>>>>> some >>>>>>>> work left to do >>>>>>>> there, but if you click around you'll find the patch anyway, in >>>>>>>> case you >>>>>>>> want to take >>>>>>>> a peek at it. >>>>>>>> >>>>>>>> We have regular builds on: >>>>>>>> ??? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>>> ??? - {Windows} x {x86_64}, >>>>>>>> ??? - {MacOS X} x {x86_64} >>>>>>>> >>>>>>>> This also routinely passes: >>>>>>>> ??? - the new Shenandoah tests >>>>>>>> ??? - jcstress with/without aggressive Shenandoah verification >>>>>>>> ??? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>>> >>>>>>>> >>>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>>>> deserves >>>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>>> advice and support, my collegues in Oracle's GC, runtime and >>>>>>>> compiler >>>>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>>>> interface and >>>>>>>> related changes, and of course the many early adopters for reporting >>>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>>> without any of you! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Roman >>>>>>>> >>>>>>> >>>>>> >>>>> >>> > From david.holmes at oracle.com Mon Dec 3 00:38:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Dec 2018 10:38:12 +1000 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <2828302a-3e42-d35b-864d-1bbadba7a233@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> <2828302a-3e42-d35b-864d-1bbadba7a233@oracle.com> Message-ID: On 30/11/2018 8:40 pm, Per Liden wrote: > Hi, > > On 11/30/18 10:55 AM, Roman Kennke wrote: >>>> Hi Per, >>>> >>>> Thanks for taking time to look at this! >>>> >>>> We will take care of all the issues you mentioned. >>>> >>>> Regarding the flags, I think they are useful as they are now. >>>> Shenandoah >>>> can be run in 'passive' mode, which doesn't involve any concurrent GC >>>> (iow, it runs kindof like Parallel GC). In this mode we can selectively >>>> turn on/off different kinds of barriers. This is useful: >>>> - for debugging. if we see a crash and we suspect it's caused by a >>>> certain type of barrier, we can turn on/off those barriers to narrow >>>> down >>>> - for performance experiments: passive w/o any barriers give a good >>>> baseline against which we can measure impact of types of barriers. >>>> >>>> Debugging may or may not be useful in develop mode (some bugs only show >>>> up in product build), performance work quite definitely is not >>>> useful in >>>> develop builds, and therefore I think it makes sense to keep them as >>>> diagnostic, because that is what they are: diagnostic flags. >>>> >>>> Makes sense? >>> >>> I can see how these flags can be useful. But I think you might be in >>> violating-spec-territory here, if you're allowing users to turn on flags >>> that crash the VM. If they are safe to use in 'passive' mode, then I >>> think there should be code in shenandoahArguments.cpp that >>> rejects/corrects flag combinations that are unstable. >> >> Let us see if we can do that. >> >> If you think this violates the spec, then please point to the part that >> says diagnostic (!!) options must pass the TCK and/or not crash the JVM? >> I mean, it's called diagnostic for a reason. It seems counter-useful to >> bury diagnostic flags that we would be using for diagnostics. >> >>> I think the correct way to think about this is: We should pass the TCK, >>> regardless of which features are enabled/disabled (unless the VM barfs >>> at start-up because the chosen combination of flags are incompatible or >>> isn't supported in the current environment, etc). >>> >>> CC:ing Mikael here, who might be able to shed some light on what's ok >>> and not ok here. >> >> Yeah, again, where is it said that diagnostic flags must even pass the >> TCK? That is totally new to me. > > I'm not really the right person to speak authoritatively about this. > However, from what I understand it doesn't matter what kind of flag it > is. If a user can enable it in a production/shipped build I think it > must pass the TCK. That cannot be true in a strict sense ref: ./share/runtime/globals.hpp: diagnostic(ccstr, AbortVMOnException, NULL, which would obviously fail the TCK. However I think there is a distinction between a flag that causes the VM to abruptly terminate versus a flag that allows the VM to run in a manner that violates the spec. David ----- > > But let's have someone who knows more about the rules around that weight > in on this. > > cheers, > Per From john.r.rose at oracle.com Mon Dec 3 04:52:43 2018 From: john.r.rose at oracle.com (John Rose) Date: Sun, 2 Dec 2018 20:52:43 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <8769251B-C42F-4DDF-9E72-8628D09C3464@oracle.com> On Nov 30, 2018, at 3:22 PM, coleen.phillimore at oracle.com wrote: > > Hi, I looked at the runtime changes, which are very few, since you've upstreamed most of the runtime dependent changes already. > They look good to me. "since you've upstreamed most of the runtime dependent changes already" Yes. That's what winning looks like! ? John From jcbeyler at google.com Mon Dec 3 05:24:31 2018 From: jcbeyler at google.com (JC Beyler) Date: Sun, 2 Dec 2018 21:24:31 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> Message-ID: Hi all, Would someone on the GC or runtime team be motivated to give this a review? :) It would be much appreciated! Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 Thanks for your help, Jc On Tue, Nov 27, 2018 at 4:36 PM JC Beyler wrote: > Hi Chris, > > Yes I was waiting for another review since you had explicitly asked :) > > And sounds good that when someone from GC or runtime gives a review, I'll > wait for your full review on the webrev.02! > > Thanks again for your help, > Jc > > > On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer > wrote: > >> Hi JC, >> >> I think it would be good to get a review from the gc or runtime teams, >> since this also affects their tests. >> >> Also, once we are settled on this FatalError approach, I still need to >> give your webrev-02 a full review. I only skimmed over parts of it (I did >> look at all the changes in webrevo-01). >> >> thanks, >> >> Chris >> >> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com wrote: >> >> Hi Jc, >> >> I've already reviewed this too. >> >> Thanks, >> Serguei >> >> >> On 11/27/18 06:56, JC Beyler wrote: >> >> Thanks Chris, >> >> Anybody else motivated to look at this and review it? :) >> Jc >> >> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >> wrote: >> >>> Hi JC, >>> >>> I'm ok with the FatalError approach, but would like to hear opinions >>> from others also. >>> >>> thanks, >>> >>> Chris >>> >>> On 11/21/18 8:19 AM, JC Beyler wrote: >>> >>> Hi Chris, >>> >>> Thanks for taking the time to look at it and yes you have raised exactly >>> why the webrev is between two worlds: in cases where a fatal error on >>> failure is wanted, should we simplify the code to remove the return tests >>> since we do them internally? Now that I've looked around for non-fatal >>> cases, I think the answer is yes, it simplifies the code while maintaining >>> the checks. >>> >>> I looked a bit and it seems that I can't find easily a case where the >>> test accepts a JNI failure to then move on. Therefore, perhaps, for now, >>> the fail with a Fatal is enough and we can work on the tests to clean them >>> up? >>> >>> That means that this is the new webrev with only Fatal and cleans up the >>> tests so that it is no longer in between two worlds: >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> >>> (This passes testing on my dev machine for all the modified tests) >>> >>> with the example you provided, it now looks like: >>> >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>> >>> Where it does, to me at least, seem cleaner and less "noisy". >>> >>> Let me know what you think, >>> Jc >>> >>> >>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >>> wrote: >>> >>>> Hi JC, >>>> >>>> Sorry about the delay. I had to go back an look at the initial 8210842 >>>> webrev and RFR thread to see what this was initially all about. >>>> >>>> In general the changes look good. >>>> >>>> I don't have a good answer to your FatalError/NonFatalError question. >>>> It makes the code a lot cleaner to use FatalError, but then it is a >>>> behavior change, and you also need to deal with tests that intentionally >>>> induce errors (do you have an example of that). >>>> >>>> In any case, right now your webrev seems to be between two worlds. You >>>> are producing FatalError, but still checking results. Here's a good example: >>>> >>>> >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> >>>> I'm not sure if this is just a temporary state until it was decided >>>> which approach to take. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> >>>> On 11/20/18 2:14 PM, JC Beyler wrote: >>>> >>>> Hi all, >>>> >>>> Chris thought it made sense to have more eyes on this change than just >>>> serviceability as it will modify to tests that are not only serviceability >>>> tests so I've moved this to conversation here :) >>>> >>>> For convenience, I've copy-pasted the initial RFR: >>>> >>>> Could I have a review for the extension and usage of the >>>> ExceptionJniWrapper. This adds lines and filenames to the end of the >>>> wrapper JNI methods, adds tracing, and throws an error if need be. I've >>>> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >>>> ported a few of the tests that were already changed for the assignment >>>> webrev for JDK-8212884. >>>> >>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> >>>> For illustration, if I force an error to the AP04/ap04t03 test and set >>>> the verbosity on, I get something like: >>>> >>>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>>> >> Calling with these parameter(s): >>>> java/lang/Threadd >>>> Wait for thread to finish >>>> << Called JNI method FindClass from ap04t003.cpp:343 >>>> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >>>> java/lang/Threadd >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> Method) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >>>> at >>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>> at >>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>> at >>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>> ... 3 more >>>> FATAL ERROR in native method: JNI method FindClass : internal error >>>> from ap04t003.cpp:343 >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> Method) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> >>>> Questions/comments I have about this are: >>>> - Do we want to force fatal errors when a JNI call fails in general? >>>> Most of these tests do the right thing and test the return of the JNI >>>> calls, for example: >>>> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >>>> if (thrClass == NULL) { >>>> >>>> but now the wrapper actually would do a fatal if the FindClass call >>>> would return a nullptr, so we could remove that test altogether. What do >>>> you think? >>>> - I prefer to leave them as the tests then become closer to what >>>> real users would have in their code and is the "recommended" way of doing it >>>> >>>> >>>> - The alternative is to use the NonFatalError I added which then >>>> just prints out that something went wrong, letting the test continue. >>>> Question will be what should be the default? The fatal or the non-fatal >>>> error handling? >>>> >>>> On a different subject: >>>> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >>>> wrapper handles the tracing and the verify in almost the same way; only >>>> difference I can really tell is that the complain method from NSK has a max >>>> complain before stopping to "complain"; I have not added that part of the >>>> code in this webrev >>>> >>>> Once we decide on these, I can continue on the files from JDK-8212884 >>>> and then do both the assignment in an if extraction followed-by this type >>>> of webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can >>>> be deprecated as well as we go forward. >>>> >>>> Thanks! >>>> Jc >>>> >>>> On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer < >>>> chris.plummer at oracle.com> wrote: >>>> >>>>> On 11/19/18 10:07 AM, JC Beyler wrote: >>>>> >>>>> Hi all, >>>>> >>>>> @David/Chris: should I then push this RFR to the hotspot mailing or >>>>> the runtime one? For what it's worth, a lot of the tests under the >>>>> vmTestbase are jvmti so the review also affects serviceability; it just >>>>> turns out I started with the GC originally and then hit some other tests I >>>>> had touched via the assignment extraction. >>>>> >>>>> I think hotspot would be best. >>>>> >>>>> Chris >>>>> >>>>> >>>>> @Serguei: Done for the method renaming, for the indent, are you >>>>> talking about going from the 8-indent to 4-indent? If so, would it not just >>>>> be better to do a new JBS bug and do the whole files in one go? I ask >>>>> because otherwise, it will look a bit weird to have parts of the file as >>>>> 8-indent and others 4-indent? >>>>> >>>>> Thanks for looking at it! >>>>> Jc >>>>> >>>>> On Mon, Nov 19, 2018 at 1:25 AM serguei.spitsyn at oracle.com < >>>>> serguei.spitsyn at oracle.com> wrote: >>>>> >>>>>> Hi Jc, >>>>>> >>>>>> We have to start this review anyway. :) >>>>>> It looks good to me in general. >>>>>> Thank you for your consistency in this refactoring! >>>>>> >>>>>> Some minor comments. >>>>>> >>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>>> >>>>>> +static const char* remove_folders(const char* fullname) { >>>>>> >>>>>> I'd suggest to rename the function name to something traditional like get_basename. >>>>>> Otherwise, it sounds like this function has to really remove folders. :) >>>>>> >>>>>> >>>>>> Also, all *Locker.cpp have wrong indent in the bodies of if and while statements. >>>>>> Could this be fixed with the refactoring? >>>>>> >>>>>> I did not look on how this impacts the tests other than serviceability. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 11/16/18 19:43, JC Beyler wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> Anybody motivated to review this? :) >>>>>> Jc >>>>>> >>>>>> On Wed, Nov 7, 2018 at 9:53 PM JC Beyler wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Could I have a review for the extension and usage of the >>>>>>> ExceptionJniWrapper. This adds lines and filenames to the end of the >>>>>>> wrapper JNI methods, adds tracing, and throws an error if need be. I've >>>>>>> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >>>>>>> ported a few of the tests that were already changed for the assignment >>>>>>> webrev for JDK-8212884. >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>>> >>>>>>> For illustration, if I force an error to the AP04/ap04t03 test and >>>>>>> set the verbosity on, I get something like: >>>>>>> >>>>>>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>>>>>> >> Calling with these parameter(s): >>>>>>> java/lang/Threadd >>>>>>> Wait for thread to finish >>>>>>> << Called JNI method FindClass from ap04t003.cpp:343 >>>>>>> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >>>>>>> java/lang/Threadd >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>> Method) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >>>>>>> at >>>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>>> at >>>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>>> at >>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>>> ... 3 more >>>>>>> FATAL ERROR in native method: JNI method FindClass : internal error >>>>>>> from ap04t003.cpp:343 >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>> Method) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>> >>>>>>> Questions/comments I have about this are: >>>>>>> - Do we want to force fatal errors when a JNI call fails in >>>>>>> general? Most of these tests do the right thing and test the return of the >>>>>>> JNI calls, for example: >>>>>>> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >>>>>>> if (thrClass == NULL) { >>>>>>> >>>>>>> but now the wrapper actually would do a fatal if the FindClass call >>>>>>> would return a nullptr, so we could remove that test altogether. What do >>>>>>> you think? >>>>>>> - I prefer to leave them as the tests then become closer to what >>>>>>> real users would have in their code and is the "recommended" way of doing it >>>>>>> >>>>>>> - The alternative is to use the NonFatalError I added which then >>>>>>> just prints out that something went wrong, letting the test continue. >>>>>>> Question will be what should be the default? The fatal or the non-fatal >>>>>>> error handling? >>>>>>> >>>>>>> On a different subject: >>>>>>> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >>>>>>> wrapper handles the tracing and the verify in almost the same way; only >>>>>>> difference I can really tell is that the complain method from NSK has a max >>>>>>> complain before stopping to "complain"; I have not added that part of the >>>>>>> code in this webrev >>>>>>> >>>>>>> Once we decide on these, I can continue on the files from >>>>>>> JDK-8212884 and then do both the assignment in an if extraction followed-by >>>>>>> this type of webrev in an easier fashion. Depending on decisions here, >>>>>>> NSK*VERIFY can be deprecated as well as we go forward. >>>>>>> >>>>>>> Thank you for the reviews/comments :) >>>>>>> Jc >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> Jc >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> Jc >>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>>> >>>> >>>> >>> >>> -- >>> >>> Thanks, >>> Jc >>> >>> >>> >> >> -- >> >> Thanks, >> Jc >> >> >> >> > > -- > > Thanks, > Jc > -- Thanks, Jc From Nick.Gasson at arm.com Mon Dec 3 06:06:47 2018 From: Nick.Gasson at arm.com (Nick Gasson) Date: Mon, 3 Dec 2018 06:06:47 +0000 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 Message-ID: Hi, Could someone please help me review this fix for a test failure on ARM32: Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ This fixes two assertion failures related to biased locking when C2's bias inlining feature is disabled (-XX:-OptoBiasInlining): MacroAssembler::atomic_cas_bool takes three register arguments plus a temporary register for use in the CAS loop. If the caller passes `noreg' as this temporary register (which happens in the !OptoBiasInlining call path from MacroAssembler::fast_lock) then MacroAssembler::atomic_cas_bool will default to using LR as a temporary. But it's possible with C2 that LR is one of the other three arguments which then triggers an assert_different_registers assertion failure. Fix this by supplying an additional scratch register to MacroAssembler::fast_lock if !OptoBiasInlining. In the !OptoBiasInlining case MacroAssembler::fast_lock calls MacroAssembler::biased_locking_enter to handle the case where the mark word contains the biased lock pattern which fast_lock wouldn't otherwise see if OptoBiasInlining was true. However in the case that we fail to acquire the biased lock this falls through to label `failed' and runs the normal fast_lock code that implicitly assumes the mark word does not have the bias pattern. This means we can end up with a BasicLock where _displaced_header contains the biased lock pattern which is an illegal state and subsequently triggers an assertion failure in ObjectSynchronizer::fast_exit. The right thing to do here is branch to `done' whenever the bias lock pattern is present and let the runtime handle the failure case. Also edited the comment describing MacroAssembler::biased_locking_enter to more accurately describe what it does (the comment about `slow_case' being optional is wrong). Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this patch. Thanks, Nick From amith.pawar at gmail.com Mon Dec 3 06:19:34 2018 From: amith.pawar at gmail.com (amith pawar) Date: Mon, 3 Dec 2018 11:49:34 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> Message-ID: Hi Derek On Sat, Dec 1, 2018 at 4:10 AM White, Derek wrote: > Hi Amith, > > > Thanks for tackling this! > > > I think that your patch may fix > https://bugs.openjdk.java.net/browse/JDK-8205051, although it's probably > fixing more than that. I was thinking that the problem was related to > calling " _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr);", > and you replaced that call with more specific node sets, which seems better > to me. > > > You can try the test cases listed in the bug to see if the patch fixes the > problem. > Tested and didn't fix it. > > The existing code disables UseNUMA if the process is bound to only one > node, but it looks like your patch removes this. Is that a correct read of > the patch, intentional, and/or a good idea? > I removed that after testing it with SPECjbb for single node case. With patch : Enabling/Disabling UseNUMA gives same score. Without patch: UseNUMA scores 10-12% lesser than without UseNUMA > > Thanks! > > - Derek > > ------------------------------ > *From:* hotspot-dev on behalf of > amith pawar > *Sent:* Thursday, November 29, 2018 6:08 AM > *To:* thomas.schatzl at oracle.com > *Cc:* hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com > *Subject:* Re: [PATCH] JDK NUMA Interleaving issue > > External Email > > Hi Thomas, > > Sorry for the late reply and please check my inline comments below. > > On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl > wrote: > > > Hi Amith, > > > > welcome to the OpenJDK community! :) > > > Thanks > > > > > > On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: > > > Hi all, > > > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, > > > S1 and S2 region (if any other ) memory areas on requested Numa nodes > > > and it should not configure itself to access other Numa nodes. This > > > issue is observed only when Java is allowed to run on fewer NUMA > > > nodes than available on the system with numactl membind and > > > interleave options. Running on all the nodes does not have any > > > effect. This will cause some applications (objects residing in old > > > gen and survivor region) to run slower on system with large Numa > > > nodes. > > > > > [... long explanation...] > > > > Is it possible to summarize the problem and the changes with the > > following few lines: > > > > "NUMA interleaving of memory of old gen and survivor spaces (for > > parallel GC) tells the OS to interleave memory across all nodes of a > > NUMA system. However the VM process may be configured to be limited to > > run only on a few nodes, which means that large parts of the heap will > > be located on foreign nodes. This can incurs a large performance > > penalty. > > > > The proposed solution is to tell the OS to interleave memory only > > across available nodes when enabling NUMA." > > > OK. Since, I dont have write access to the defect link so can I request you > do the same. > > > > > We have had trouble understanding the problem statement and purpose of > > this patch when triaging (making sure the issue is understandable and > > can be worked on) as the text is rather technical. Having an easily > > understandable text also helps reviewers a lot. > > > > > Assuming my summary is appropriate, I have several other unrelated > > questions: > > > > - could you propose a better subject for this work? "JDK NUMA > > Interleaving issue" seems very generic. Something like "NUMA heap > > allocation does not respect VM membind/interleave settings" maybe? > > > This is also OK. > > > > > - there have been other NUMA related patches from AMD recently, in > > particular JDK-what is the relation to JDK-8205051? The text there > > reads awfully similar to this one, but I *think* JDK-8205051 is > > actually about making sure that the parallel gc eden is not put on > > inactive nodes. > > Can you confirm this (talk to your colleague) so that we can fix the > > description too? > > > JDK-8205051 is related to early GC. > JDK-8189922 specific to membind case where heap was allocated on non > requested NUMA nodes. > The proposed patch fixes two following issues > 1. Similar to JDK-8189922 but for interleave case. > 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer > NUMA nodes. > > > > - fyi, we are currently working on NUMA aware memory allocation support > > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to > > not step on each other's toes (see below). > > > We are not working anything related to G1. It may effect G1 if > numa_make_global function is called. > > > > > [... broken patch...] > > > > I tried to apply the patch to the jdk/jdk tree, which failed; I then > > started to manually apply it but stopped after not being able to find > > the context of some hunks. I do not think this change applies to the > > latest source tree. > > > > Please make sure that the patch applies to the latest jdk/jdk tree with > > errors. All changes generally must first go into the latest dev tree > > before you can apply for backporting. > > > > Could you please send the patch as attachment (not copy&pasted) to this > > list and cc me? Then I can create a webrev out of it. > > > > I did look a bit over the patch as much as I could (it's very hard > > trying to review a diff), some comments: > > > Sorry. Please check the attached patch. > > > > > - the _numa_interleave_memory_v2 function pointer is never assigned > > to in the patch in the CR, so it will not be used. Please make sure the > > patch is complete. > > Actually it is never defined anywhere, ie. the patch unlikely actually > > compiles even if I could apply it somewhere. > > > > Please avoid frustrating reviewers by sending incomplete patches. > > > Sorry again. Faced same issue when copied from the defect link but worked > for me from the mail. Now i have attached the patch. Can you please update > defect link with this patch ? > > > > > - I am not sure set_numa_interleave, i.e. the interleaving, should be > > done without UseNUMAInterleaving enabled. Some collectors may do their > > own interleaving in the future (JDK-8210473 / JEP-345) so this would > > massively interfere in how they work. (That issue may be because I am > > looking at a potentially incomplete diff, so forgive me if the patch > > already handles this). > > > Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect > when java externally invoked in interleave mode. It helps membind case. > > > > > - if some of the actions (interleaving/membind) fail, and it had been > > requested, it would be nice to print a log message. > > > Following two NUMA API's are used and they return nothing. Right now not > sure which cases to handle. Please suggest > void numa_tonode_memory(void *start, size_t size, int node); > void numa_interleave_memory(void *start, size_t size, struct bitmask > *nodemask); > > > > > Actually it would be nice to print information about e.g. the bitmask > > anyway in the log so that the user can easily verify that what he > > specified on the command line has been properly picked up by the VM - > > instead of looking through the proc filesystem. > > > As per the suggestion, patch is updated to print following information. > [0.001s][info][os] UseNUMA is enabled > [0.001s][info][os] Java is configured to run in interleave mode > [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, 2, > 3 > > > > > Thanks, > > Thomas > > > > > > > > > Thanks > Amit Pawar > Thanks, Amit Pawar From amith.pawar at gmail.com Mon Dec 3 06:21:17 2018 From: amith.pawar at gmail.com (amith pawar) Date: Mon, 3 Dec 2018 11:51:17 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> Message-ID: +Roshan Mangal On Mon, Dec 3, 2018 at 11:49 AM amith pawar wrote: > Hi Derek > > On Sat, Dec 1, 2018 at 4:10 AM White, Derek > wrote: > >> Hi Amith, >> >> >> Thanks for tackling this! >> >> >> I think that your patch may fix >> https://bugs.openjdk.java.net/browse/JDK-8205051, although it's probably >> fixing more than that. I was thinking that the problem was related to >> calling " _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr);", >> and you replaced that call with more specific node sets, which seems better >> to me. >> >> >> You can try the test cases listed in the bug to see if the patch fixes >> the problem. >> > Tested and didn't fix it. > >> >> The existing code disables UseNUMA if the process is bound to only one >> node, but it looks like your patch removes this. Is that a correct read of >> the patch, intentional, and/or a good idea? >> > I removed that after testing it with SPECjbb for single node case. > With patch : Enabling/Disabling UseNUMA gives same score. > Without patch: UseNUMA scores 10-12% lesser than without UseNUMA > >> >> Thanks! >> >> - Derek >> >> > ------------------------------ >> *From:* hotspot-dev on behalf of >> amith pawar >> *Sent:* Thursday, November 29, 2018 6:08 AM >> *To:* thomas.schatzl at oracle.com >> *Cc:* hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com >> *Subject:* Re: [PATCH] JDK NUMA Interleaving issue >> >> External Email >> >> Hi Thomas, >> >> Sorry for the late reply and please check my inline comments below. >> >> On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl > > >> wrote: >> >> > Hi Amith, >> > >> > welcome to the OpenJDK community! :) >> > >> Thanks >> >> > >> > >> On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: >> > > Hi all, >> > > >> > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, >> > > S1 and S2 region (if any other ) memory areas on requested Numa nodes >> > > and it should not configure itself to access other Numa nodes. This >> > > issue is observed only when Java is allowed to run on fewer NUMA >> > > nodes than available on the system with numactl membind and >> > > interleave options. Running on all the nodes does not have any >> > > effect. This will cause some applications (objects residing in old >> > > gen and survivor region) to run slower on system with large Numa >> > > nodes. >> > > >> > [... long explanation...] >> > >> > Is it possible to summarize the problem and the changes with the >> > following few lines: >> > >> > "NUMA interleaving of memory of old gen and survivor spaces (for >> > parallel GC) tells the OS to interleave memory across all nodes of a >> > NUMA system. However the VM process may be configured to be limited to >> > run only on a few nodes, which means that large parts of the heap will >> > be located on foreign nodes. This can incurs a large performance >> > penalty. >> > >> > The proposed solution is to tell the OS to interleave memory only >> > across available nodes when enabling NUMA." >> > >> OK. Since, I dont have write access to the defect link so can I request >> you >> do the same. >> >> > >> > We have had trouble understanding the problem statement and purpose of >> > this patch when triaging (making sure the issue is understandable and >> > can be worked on) as the text is rather technical. Having an easily >> > understandable text also helps reviewers a lot. >> > >> >> > Assuming my summary is appropriate, I have several other unrelated >> > questions: >> > >> > - could you propose a better subject for this work? "JDK NUMA >> > Interleaving issue" seems very generic. Something like "NUMA heap >> > allocation does not respect VM membind/interleave settings" maybe? >> > >> This is also OK. >> >> > >> > - there have been other NUMA related patches from AMD recently, in >> > particular JDK-what is the relation to JDK-8205051? The text there >> > reads awfully similar to this one, but I *think* JDK-8205051 is >> > actually about making sure that the parallel gc eden is not put on >> > inactive nodes. >> > Can you confirm this (talk to your colleague) so that we can fix the >> > description too? >> > >> JDK-8205051 is related to early GC. >> JDK-8189922 specific to membind case where heap was allocated on non >> requested NUMA nodes. >> The proposed patch fixes two following issues >> 1. Similar to JDK-8189922 but for interleave case. >> 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer >> NUMA nodes. >> >> >> > - fyi, we are currently working on NUMA aware memory allocation support >> > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to >> > not step on each other's toes (see below). >> > >> We are not working anything related to G1. It may effect G1 if >> numa_make_global function is called. >> >> > >> > [... broken patch...] >> > >> > I tried to apply the patch to the jdk/jdk tree, which failed; I then >> > started to manually apply it but stopped after not being able to find >> > the context of some hunks. I do not think this change applies to the >> > latest source tree. >> > >> > Please make sure that the patch applies to the latest jdk/jdk tree with >> > errors. All changes generally must first go into the latest dev tree >> > before you can apply for backporting. >> > >> > Could you please send the patch as attachment (not copy&pasted) to this >> > list and cc me? Then I can create a webrev out of it. >> > >> > I did look a bit over the patch as much as I could (it's very hard >> > trying to review a diff), some comments: >> > >> Sorry. Please check the attached patch. >> >> > >> > - the _numa_interleave_memory_v2 function pointer is never assigned >> > to in the patch in the CR, so it will not be used. Please make sure the >> > patch is complete. >> > Actually it is never defined anywhere, ie. the patch unlikely actually >> > compiles even if I could apply it somewhere. >> > >> > Please avoid frustrating reviewers by sending incomplete patches. >> > >> Sorry again. Faced same issue when copied from the defect link but worked >> for me from the mail. Now i have attached the patch. Can you please update >> defect link with this patch ? >> >> > >> > - I am not sure set_numa_interleave, i.e. the interleaving, should be >> > done without UseNUMAInterleaving enabled. Some collectors may do their >> > own interleaving in the future (JDK-8210473 / JEP-345) so this would >> > massively interfere in how they work. (That issue may be because I am >> > looking at a potentially incomplete diff, so forgive me if the patch >> > already handles this). >> > >> Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect >> when java externally invoked in interleave mode. It helps membind case. >> >> > >> > - if some of the actions (interleaving/membind) fail, and it had been >> > requested, it would be nice to print a log message. >> > >> Following two NUMA API's are used and they return nothing. Right now not >> sure which cases to handle. Please suggest >> void numa_tonode_memory(void *start, size_t size, int node); >> void numa_interleave_memory(void *start, size_t size, struct bitmask >> *nodemask); >> >> > >> > Actually it would be nice to print information about e.g. the bitmask >> > anyway in the log so that the user can easily verify that what he >> > specified on the command line has been properly picked up by the VM - >> > instead of looking through the proc filesystem. >> > >> As per the suggestion, patch is updated to print following information. >> [0.001s][info][os] UseNUMA is enabled >> [0.001s][info][os] Java is configured to run in interleave mode >> [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, >> 2, >> 3 >> >> > >> > Thanks, >> > Thomas >> > >> > >> > >> > >> Thanks >> Amit Pawar >> > > Thanks, > Amit Pawar > -- With best regards, amit pawar From roshanmangal at gmail.com Mon Dec 3 06:27:45 2018 From: roshanmangal at gmail.com (roshan mangal) Date: Mon, 3 Dec 2018 11:57:45 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> Message-ID: Hi Derek, The fix submitted by Amith is not fixing "https://bugs.openjdk.java.net/browse/JDK-8205051". Amith is fixing survivor space and old gen related issue, not a locality group(lgrp) related issue. We are addressing below two issues in "https://bugs.openjdk.java.net/browse/JDK-8205051" Issue 1: Whenever any locality group(lgrp) fills-in, the young collection occurs irrespective of other locality groups has enough memory. Issue 2: if an application has fewer threads than the number of locality groups (NUMA nodes). Locality groups, which are not having any home thread will be unused. Regards, Roshan Mangal On Mon, Dec 3, 2018 at 11:51 AM amith pawar wrote: > > +Roshan Mangal > > On Mon, Dec 3, 2018 at 11:49 AM amith pawar wrote: >> >> Hi Derek >> >> On Sat, Dec 1, 2018 at 4:10 AM White, Derek wrote: >>> >>> Hi Amith, >>> >>> >>> Thanks for tackling this! >>> >>> >>> I think that your patch may fix https://bugs.openjdk.java.net/browse/JDK-8205051, although it's probably fixing more than that. I was thinking that the problem was related to calling " _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr);", and you replaced that call with more specific node sets, which seems better to me. >>> >>> >>> You can try the test cases listed in the bug to see if the patch fixes the problem. >> >> Tested and didn't fix it. >>> >>> >>> The existing code disables UseNUMA if the process is bound to only one node, but it looks like your patch removes this. Is that a correct read of the patch, intentional, and/or a good idea? >> >> I removed that after testing it with SPECjbb for single node case. >> With patch : Enabling/Disabling UseNUMA gives same score. >> Without patch: UseNUMA scores 10-12% lesser than without UseNUMA >>> >>> >>> Thanks! >>> >>> - Derek >>> >>> >>> ________________________________ >>> From: hotspot-dev on behalf of amith pawar >>> Sent: Thursday, November 29, 2018 6:08 AM >>> To: thomas.schatzl at oracle.com >>> Cc: hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com >>> Subject: Re: [PATCH] JDK NUMA Interleaving issue >>> >>> External Email >>> >>> Hi Thomas, >>> >>> Sorry for the late reply and please check my inline comments below. >>> >>> On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl >>> wrote: >>> >>> > Hi Amith, >>> > >>> > welcome to the OpenJDK community! :) >>> > >>> Thanks >>> >>> > >>> > >>> On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: >>> > > Hi all, >>> > > >>> > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, >>> > > S1 and S2 region (if any other ) memory areas on requested Numa nodes >>> > > and it should not configure itself to access other Numa nodes. This >>> > > issue is observed only when Java is allowed to run on fewer NUMA >>> > > nodes than available on the system with numactl membind and >>> > > interleave options. Running on all the nodes does not have any >>> > > effect. This will cause some applications (objects residing in old >>> > > gen and survivor region) to run slower on system with large Numa >>> > > nodes. >>> > > >>> > [... long explanation...] >>> > >>> > Is it possible to summarize the problem and the changes with the >>> > following few lines: >>> > >>> > "NUMA interleaving of memory of old gen and survivor spaces (for >>> > parallel GC) tells the OS to interleave memory across all nodes of a >>> > NUMA system. However the VM process may be configured to be limited to >>> > run only on a few nodes, which means that large parts of the heap will >>> > be located on foreign nodes. This can incurs a large performance >>> > penalty. >>> > >>> > The proposed solution is to tell the OS to interleave memory only >>> > across available nodes when enabling NUMA." >>> > >>> OK. Since, I dont have write access to the defect link so can I request you >>> do the same. >>> >>> > >>> > We have had trouble understanding the problem statement and purpose of >>> > this patch when triaging (making sure the issue is understandable and >>> > can be worked on) as the text is rather technical. Having an easily >>> > understandable text also helps reviewers a lot. >>> > >>> >>> > Assuming my summary is appropriate, I have several other unrelated >>> > questions: >>> > >>> > - could you propose a better subject for this work? "JDK NUMA >>> > Interleaving issue" seems very generic. Something like "NUMA heap >>> > allocation does not respect VM membind/interleave settings" maybe? >>> > >>> This is also OK. >>> >>> > >>> > - there have been other NUMA related patches from AMD recently, in >>> > particular JDK-what is the relation to JDK-8205051? The text there >>> > reads awfully similar to this one, but I *think* JDK-8205051 is >>> > actually about making sure that the parallel gc eden is not put on >>> > inactive nodes. >>> > Can you confirm this (talk to your colleague) so that we can fix the >>> > description too? >>> > >>> JDK-8205051 is related to early GC. >>> JDK-8189922 specific to membind case where heap was allocated on non >>> requested NUMA nodes. >>> The proposed patch fixes two following issues >>> 1. Similar to JDK-8189922 but for interleave case. >>> 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer >>> NUMA nodes. >>> >>> >>> > - fyi, we are currently working on NUMA aware memory allocation support >>> > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to >>> > not step on each other's toes (see below). >>> > >>> We are not working anything related to G1. It may effect G1 if >>> numa_make_global function is called. >>> >>> > >>> > [... broken patch...] >>> > >>> > I tried to apply the patch to the jdk/jdk tree, which failed; I then >>> > started to manually apply it but stopped after not being able to find >>> > the context of some hunks. I do not think this change applies to the >>> > latest source tree. >>> > >>> > Please make sure that the patch applies to the latest jdk/jdk tree with >>> > errors. All changes generally must first go into the latest dev tree >>> > before you can apply for backporting. >>> > >>> > Could you please send the patch as attachment (not copy&pasted) to this >>> > list and cc me? Then I can create a webrev out of it. >>> > >>> > I did look a bit over the patch as much as I could (it's very hard >>> > trying to review a diff), some comments: >>> > >>> Sorry. Please check the attached patch. >>> >>> > >>> > - the _numa_interleave_memory_v2 function pointer is never assigned >>> > to in the patch in the CR, so it will not be used. Please make sure the >>> > patch is complete. >>> > Actually it is never defined anywhere, ie. the patch unlikely actually >>> > compiles even if I could apply it somewhere. >>> > >>> > Please avoid frustrating reviewers by sending incomplete patches. >>> > >>> Sorry again. Faced same issue when copied from the defect link but worked >>> for me from the mail. Now i have attached the patch. Can you please update >>> defect link with this patch ? >>> >>> > >>> > - I am not sure set_numa_interleave, i.e. the interleaving, should be >>> > done without UseNUMAInterleaving enabled. Some collectors may do their >>> > own interleaving in the future (JDK-8210473 / JEP-345) so this would >>> > massively interfere in how they work. (That issue may be because I am >>> > looking at a potentially incomplete diff, so forgive me if the patch >>> > already handles this). >>> > >>> Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect >>> when java externally invoked in interleave mode. It helps membind case. >>> >>> > >>> > - if some of the actions (interleaving/membind) fail, and it had been >>> > requested, it would be nice to print a log message. >>> > >>> Following two NUMA API's are used and they return nothing. Right now not >>> sure which cases to handle. Please suggest >>> void numa_tonode_memory(void *start, size_t size, int node); >>> void numa_interleave_memory(void *start, size_t size, struct bitmask >>> *nodemask); >>> >>> > >>> > Actually it would be nice to print information about e.g. the bitmask >>> > anyway in the log so that the user can easily verify that what he >>> > specified on the command line has been properly picked up by the VM - >>> > instead of looking through the proc filesystem. >>> > >>> As per the suggestion, patch is updated to print following information. >>> [0.001s][info][os] UseNUMA is enabled >>> [0.001s][info][os] Java is configured to run in interleave mode >>> [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, 2, >>> 3 >>> >>> > >>> > Thanks, >>> > Thomas >>> > >>> > >>> > >>> > >>> Thanks >>> Amit Pawar >> >> >> Thanks, >> Amit Pawar > > > > -- > With best regards, > amit pawar From david.holmes at oracle.com Mon Dec 3 06:29:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Dec 2018 16:29:08 +1000 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> Message-ID: <895ef766-9c96-7185-4222-178379629ce4@oracle.com> Hi Jc, I've been lurking on this one and have had a look through. I'm okay with the FatalError approach for the tests - we don't expect anything to go wrong in a well written test in a correctly functioning VM. Thanks, David On 3/12/2018 3:24 pm, JC Beyler wrote: > Hi all, > > Would someone on the GC or runtime team be motivated to give this a > review? :) > > It would be much appreciated! > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > Thanks for your help, > Jc > > On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > wrote: > > Hi Chris, > > Yes I was waiting for another review since you had explicitly asked :) > > And sounds good that when someone from GC or runtime gives a review, > I'll wait for your full review on the webrev.02! > > Thanks again for?your help, > Jc > > > On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer > > wrote: > > Hi JC, > > I think it would be good to get a review from the gc or runtime > teams, since this also affects their tests. > > Also, once we are settled on this FatalError approach, I still > need to give your webrev-02 a full review. I only skimmed over > parts of it (I did look at all the changes in webrevo-01). > > thanks, > > Chris > > On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com > wrote: >> Hi Jc, >> >> I've already reviewed this too. >> >> Thanks, >> Serguei >> >> >> On 11/27/18 06:56, JC Beyler wrote: >>> Thanks Chris, >>> >>> Anybody else motivated to look at this and review it? :) >>> Jc >>> >>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >>> > >>> wrote: >>> >>> Hi JC, >>> >>> I'm ok with the FatalError approach, but would like to >>> hear opinions from others also. >>> >>> thanks, >>> >>> Chris >>> >>> On 11/21/18 8:19 AM, JC Beyler wrote: >>>> Hi Chris, >>>> >>>> Thanks for taking the time to look at it and yes you >>>> have raised exactly why the webrev is between two >>>> worlds: in cases where a fatal error on failure is >>>> wanted, should we simplify the code to remove the return >>>> tests since we do them internally? Now that I've looked >>>> around for non-fatal cases, I think the answer is yes, >>>> it simplifies the code while maintaining the checks. >>>> >>>> I looked a bit and it seems that I can't find easily a >>>> case where the test accepts a JNI failure to then move >>>> on. Therefore, perhaps, for now, the fail with a Fatal >>>> is enough and we can work on the tests to clean them up? >>>> >>>> That means that this is the new webrev with only Fatal >>>> and cleans up the tests so that it is no longer in >>>> between two worlds: >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> >>>> (This passes testing on my dev machine for all the >>>> modified tests) >>>> >>>> with the example you provided, it now looks like: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> >>>> >>>> Where it does, to me at least, seem cleaner and less >>>> "noisy". >>>> >>>> Let me know what you think, >>>> Jc >>>> >>>> >>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >>>> >>> > wrote: >>>> >>>> Hi JC, >>>> >>>> Sorry about the delay. I had to go back an look at >>>> the initial 8210842 webrev and RFR thread to see >>>> what this was initially all about. >>>> >>>> In general the changes look good. >>>> >>>> I don't have a good answer to your >>>> FatalError/NonFatalError question. It makes the code >>>> a lot cleaner to use FatalError, but then it is a >>>> behavior change, and you also need to deal with >>>> tests that intentionally induce errors (do you have >>>> an example of that). >>>> >>>> In any case, right now your webrev seems to be >>>> between two worlds. You are producing FatalError, >>>> but still checking results. Here's a good example: >>>> >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> >>>> >>>> I'm not sure if this is just a temporary state until >>>> it was decided which approach to take. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> >>>> On 11/20/18 2:14 PM, JC Beyler wrote: >>>>> Hi all, >>>>> >>>>> Chris thought it made sense to have more eyes on >>>>> this change than just serviceability as it will >>>>> modify to tests that are not only serviceability >>>>> tests so I've moved this to conversation here :) >>>>> >>>>> For convenience, I've copy-pasted the initial RFR: >>>>> >>>>> Could I have a review for the extension and usage >>>>> of the ExceptionJniWrapper. This adds lines and >>>>> filenames to the end of the wrapper JNI methods, >>>>> adds tracing, and throws an error if need be. I've >>>>> ported the gc/lock files to use the new >>>>> TRACE_JNI_CALL add-on and I've ported a few of the >>>>> tests that were already changed for the assignment >>>>> webrev for JDK-8212884. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> >>>>> For illustration, if I force an error to the >>>>> AP04/ap04t03 test and set the verbosity on, I get >>>>> something like: >>>>> >>>>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>>>> >> Calling with these parameter(s): >>>>> ? ? ? ? java/lang/Threadd >>>>> Wait for thread to finish >>>>> << Called JNI method FindClass from ap04t003.cpp:343 >>>>> Exception in thread "Thread-0" >>>>> java.lang.NoClassDefFoundError: java/lang/Threadd >>>>> ? ? ? ? at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> Method) >>>>> ? ? ? ? at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> ? ? ? ? at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> Caused by: java.lang.ClassNotFoundException: >>>>> java.lang.Threadd >>>>> ? ? ? ? at >>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>> ? ? ? ? at >>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>> ? ? ? ? at >>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>> ? ? ? ? ... 3 more >>>>> FATAL ERROR in native method: JNI method FindClass >>>>> : internal error from ap04t003.cpp:343 >>>>> ? ? ? ? at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> Method) >>>>> ? ? ? ? at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> ? ? ? ? at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> >>>>> Questions/comments I have about this are: >>>>> ? - Do we want to force fatal errors when a JNI >>>>> call fails in general? Most of these tests do the >>>>> right thing and test the return of the JNI calls, >>>>> for example: >>>>> ? ? thrClass = jni->FindClass("java/lang/Threadd", >>>>> TRACE_JNI_CALL); >>>>> ? ? if (thrClass == NULL) { >>>>> >>>>> but now the wrapper actually would do a fatal if >>>>> the FindClass call would return a nullptr, so we >>>>> could remove that test altogether. What do you think? >>>>> ? ? - I prefer to leave them as the tests then >>>>> become closer to what real users would have in >>>>> their code and is the "recommended" way of doing it >>>>> >>>>> ? ?- The alternative is to use the NonFatalError I >>>>> added which then just prints out that something >>>>> went wrong, letting the test continue. Question >>>>> will be what should be the default? The fatal or >>>>> the non-fatal error handling? >>>>> >>>>> On a different subject: >>>>> ? - On the new tests, I've removed the >>>>> NSK_JNI_VERIFY since the JNI wrapper handles the >>>>> tracing and the verify in almost the same way; only >>>>> difference I can really tell is that the complain >>>>> method from NSK has a max complain before stopping >>>>> to "complain"; I have not added that part of the >>>>> code in this webrev >>>>> >>>>> Once we decide on these, I can continue on the >>>>> files from JDK-8212884 and then do both the >>>>> assignment in an if extraction followed-by this >>>>> type of webrev in an easier fashion. Depending on >>>>> decisions here, NSK*VERIFY can be deprecated as >>>>> well as we go forward. >>>>> >>>>> Thanks! >>>>> Jc >>>>> >>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer >>>>> >>>> > wrote: >>>>> >>>>> On 11/19/18 10:07 AM, JC Beyler wrote: >>>>>> Hi all, >>>>>> >>>>>> @David/Chris: should I then push this RFR to >>>>>> the hotspot mailing or the runtime one? For >>>>>> what it's worth, a lot of the tests under the >>>>>> vmTestbase are jvmti so the review also >>>>>> affects serviceability; it just turns out I >>>>>> started with the GC originally and then hit >>>>>> some other tests I had touched via the >>>>>> assignment extraction. >>>>> I think hotspot would be best. >>>>> >>>>> Chris >>>>>> >>>>>> @Serguei: Done for the method renaming, for >>>>>> the indent, are you talking about going from >>>>>> the 8-indent to 4-indent? If so, would it not >>>>>> just be better to do a new JBS bug and do the >>>>>> whole files in one go? I ask because >>>>>> otherwise, it will look a bit weird to have >>>>>> parts of the file as 8-indent and others 4-indent? >>>>>> >>>>>> Thanks for looking at it! >>>>>> Jc >>>>>> >>>>>> On Mon, Nov 19, 2018 at 1:25 AM >>>>>> serguei.spitsyn at oracle.com >>>>>> >>>>>> >>>>> > wrote: >>>>>> >>>>>> Hi Jc, >>>>>> >>>>>> We have to start this review anyway. :) >>>>>> It looks good to me in general. >>>>>> Thank you for your consistency in this >>>>>> refactoring! >>>>>> >>>>>> Some minor comments. >>>>>> >>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>>> +static const char* remove_folders(const >>>>>> char* fullname) { I'd suggest to rename >>>>>> the function name to something traditional >>>>>> like get_basename. Otherwise, it sounds >>>>>> like this function has to really remove >>>>>> folders. :) Also, all *Locker.cpp have >>>>>> wrong indent in the bodies of if and while >>>>>> statements. Could this be fixed with the >>>>>> refactoring? I did not look on how this >>>>>> impacts the tests other than >>>>>> serviceability. Thanks, Serguei >>>>>> >>>>>> >>>>>> On 11/16/18 19:43, JC Beyler wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Anybody motivated to review this? :) >>>>>>> Jc >>>>>>> >>>>>>> On Wed, Nov 7, 2018 at 9:53 PM JC Beyler >>>>>>> >>>>>> > wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Could I have a review for the >>>>>>> extension and usage of the >>>>>>> ExceptionJniWrapper. This adds lines >>>>>>> and filenames to the end of the >>>>>>> wrapper JNI methods, adds tracing, >>>>>>> and throws an error if need be. I've >>>>>>> ported the gc/lock files to use the >>>>>>> new TRACE_JNI_CALL add-on and I've >>>>>>> ported a few of the tests that were >>>>>>> already changed for the assignment >>>>>>> webrev for JDK-8212884. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>>> >>>>>>> For illustration, if I force an error >>>>>>> to the AP04/ap04t03 test and set the >>>>>>> verbosity on, I get something like: >>>>>>> >>>>>>> >> Calling JNI method FindClass from >>>>>>> ap04t003.cpp:343 >>>>>>> >> Calling with these parameter(s): >>>>>>> java/lang/Threadd >>>>>>> Wait for thread to finish >>>>>>> << Called JNI method FindClass from >>>>>>> ap04t003.cpp:343 >>>>>>> Exception in thread "Thread-0" >>>>>>> java.lang.NoClassDefFoundError: >>>>>>> java/lang/Threadd >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>> Method) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>> Caused by: >>>>>>> java.lang.ClassNotFoundException: >>>>>>> java.lang.Threadd >>>>>>> at >>>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>>> at >>>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>>> at >>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>>> ... 3 more >>>>>>> FATAL ERROR in native method: JNI >>>>>>> method FindClass : internal error >>>>>>> from ap04t003.cpp:343 >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>> Method) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>> at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>> >>>>>>> Questions/comments I have about this are: >>>>>>> ? - Do we want to force fatal errors >>>>>>> when a JNI call fails in general? >>>>>>> Most of these tests do the right >>>>>>> thing and test the return of the JNI >>>>>>> calls, for example: >>>>>>> thrClass = >>>>>>> jni->FindClass("java/lang/Threadd", >>>>>>> TRACE_JNI_CALL); >>>>>>> ? ? if (thrClass == NULL) { >>>>>>> >>>>>>> but now the wrapper actually would do >>>>>>> a fatal if the FindClass call would >>>>>>> return a nullptr, so we could remove >>>>>>> that test altogether. What do you think? >>>>>>> ? ? - I prefer to leave them as the >>>>>>> tests then become closer to what real >>>>>>> users would have in their code and is >>>>>>> the "recommended" way of doing it >>>>>>> >>>>>>> ? ?- The alternative is to use the >>>>>>> NonFatalError I added which then just >>>>>>> prints out that something went wrong, >>>>>>> letting the test continue. Question >>>>>>> will be what should be the default? >>>>>>> The fatal or the non-fatal error >>>>>>> handling? >>>>>>> >>>>>>> On a different subject: >>>>>>> ? - On the new tests, I've removed >>>>>>> the NSK_JNI_VERIFY since the JNI >>>>>>> wrapper handles the tracing and the >>>>>>> verify in almost the same way; only >>>>>>> difference I can really tell is that >>>>>>> the complain method from NSK has a >>>>>>> max complain before stopping to >>>>>>> "complain"; I have not added that >>>>>>> part of the code in this webrev >>>>>>> >>>>>>> Once we decide on these, I can >>>>>>> continue on the files from >>>>>>> JDK-8212884 and then do both the >>>>>>> assignment in an if extraction >>>>>>> followed-by this type of webrev in an >>>>>>> easier fashion. Depending on >>>>>>> decisions here, NSK*VERIFY can be >>>>>>> deprecated as well as we go forward. >>>>>>> >>>>>>> Thank you for the reviews/comments :) >>>>>>> Jc >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks, >>>>>>> Jc >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> Jc >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> Jc >>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> > > > > -- > > Thanks, > Jc > > > > -- > > Thanks, > Jc From kim.barrett at oracle.com Mon Dec 3 07:03:17 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Dec 2018 02:03:17 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> Message-ID: <396B4688-DF16-47D5-9F53-31FC2524C4F3@oracle.com> > On Nov 27, 2018, at 4:34 PM, David Holmes wrote: > > On 28/11/2018 4:12 am, Kim Barrett wrote: >> The "natural result" is that the derived class copy-assign code does >> what it does. There are things that can go badly wrong here regarding >> lifetimes of embedded object references. But the asserted restriction >> on the allocated location of the object being copy-assigned to does >> nothing that I can see to prevent those potential problems. >> Can you provide a use-case where the assertion actually does something >> useful? I removed it because I don?t think such a thing exists. > > AFAICS the only time it makes any sense to use assignment is with stack/embedded ResourceObj. The assert verifies that is the case. > > This seems a very clear cut use of an assert to ensure an API is not being misused. Does it prevent all possible misuse - no. But that's not a reason to remove what it does do. > > Do we ever need the assignment semantics? Maybe we should just prohibit it altogether? > > David I don't understand why you might think that, and don't agree. Maybe there are additional unstated and unenforced restrictions? But part of my rejection of such a position is that a commonly understood idiomatic operation like assignment should either just work, or it shouldn't be allowed at all. Having a strange restriction like this makes code much harder to understand. Make a new operation for this restricted assignment. Unfortunately, there are already many uses of ResourceObj assignment. I spent a bit of time exploring the idea of removing or replacing it, but that seems way too hard for the intended scope of this cleanup. (Note that for some of the existing uses it is not obvious whether the restriction applies. Presumably it doesn't, since we're not hitting the assert.) While I like the idea of prohibiting assignment (and possibly copy too, which has many of the same issues), I'm not going to pursue that beyond perhaps an RFE for future work. BTW, as part of that exploration I found such gems as this, from opto/cfgnode.cpp:962: Node_Array node_map = new Node_Array(a); Anyway, even though I think the assert prevents some things that would work (but should perhaps be avoided anyway), and doesn't prevent some things that won't work, I'm willing to reinstate it in order to make progress. New webrevs: full: http://cr.openjdk.java.net/~kbarrett/8213481/open.01/ incr: http://cr.openjdk.java.net/~kbarrett/8213481/open.01.inc/ From kim.barrett at oracle.com Mon Dec 3 07:04:35 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Dec 2018 02:04:35 -0500 Subject: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <4F249D17-A348-4BB2-A19C-ECFFC6AB80B8@oracle.com> Message-ID: <9D7B2528-B0F8-49E5-833E-69ACB52D9590@oracle.com> > On Nov 28, 2018, at 2:38 AM, Lindenmaier, Goetz wrote: > > Hi Kim, > > we have seen various issues with xlc and acc wrt. > to constructors/destructors. xlc still issues a row of warnings, > which we disabled so that the builds passes. Therefore I > thought I better make sure it works :) Hm, are any of those warnings ?interesting?? Maybe they should be reported? > All the other builds are fine, too. Thanks for the additional testing. From david.holmes at oracle.com Mon Dec 3 07:33:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Dec 2018 17:33:09 +1000 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <396B4688-DF16-47D5-9F53-31FC2524C4F3@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> <396B4688-DF16-47D5-9F53-31FC2524C4F3@oracle.com> Message-ID: <4f10af7b-a18a-ac95-6371-7b2c5c6f3a89@oracle.com> Hi Kim, On 3/12/2018 5:03 pm, Kim Barrett wrote: >> On Nov 27, 2018, at 4:34 PM, David Holmes wrote: >> >> On 28/11/2018 4:12 am, Kim Barrett wrote: >>> The "natural result" is that the derived class copy-assign code does >>> what it does. There are things that can go badly wrong here regarding >>> lifetimes of embedded object references. But the asserted restriction >>> on the allocated location of the object being copy-assigned to does >>> nothing that I can see to prevent those potential problems. >>> Can you provide a use-case where the assertion actually does something >>> useful? I removed it because I don?t think such a thing exists. >> >> AFAICS the only time it makes any sense to use assignment is with stack/embedded ResourceObj. The assert verifies that is the case. >> >> This seems a very clear cut use of an assert to ensure an API is not being misused. Does it prevent all possible misuse - no. But that's not a reason to remove what it does do. >> >> Do we ever need the assignment semantics? Maybe we should just prohibit it altogether? >> >> David > > I don't understand why you might think that, and don't agree. Maybe > there are additional unstated and unenforced restrictions? > > But part of my rejection of such a position is that a commonly > understood idiomatic operation like assignment should either just > work, or it shouldn't be allowed at all. Having a strange restriction > like this makes code much harder to understand. Make a new operation > for this restricted assignment. How do you make a special assignment operator? AFAIK there is only one op= ResourceObj is not a nice clean, well-defined class - it is one class that embodies the possibility of being one of three different kinds of "resource obj" (embedded/stack, arena-allocated, C-heap allocated) and the operations of copying and assignment don't make sense for all of these - hence the restriction. > Unfortunately, there are already many uses of ResourceObj assignment. > I spent a bit of time exploring the idea of removing or replacing it, > but that seems way too hard for the intended scope of this cleanup. > (Note that for some of the existing uses it is not obvious whether the > restriction applies. Presumably it doesn't, since we're not hitting > the assert.) While I like the idea of prohibiting assignment (and > possibly copy too, which has many of the same issues), I'm not going > to pursue that beyond perhaps an RFE for future work. > > BTW, as part of that exploration I found such gems as this, from > opto/cfgnode.cpp:962: > > Node_Array node_map = new Node_Array(a); I can certainly understand people being confused about C++ allocation, construction and assignment. So I think the above acts as: Node_Array node_map; // Stack object using default constructor node_map.operator=(new Node_Array(a)); Which means the whole "new Node_Array(a)" is completely useless (does it cause a leak?) as operator= ignores it. > Anyway, even though I think the assert prevents some things that would > work (but should perhaps be avoided anyway), and doesn't prevent some > things that won't work, I'm willing to reinstate it in order to make > progress. > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8213481/open.01/ > incr: http://cr.openjdk.java.net/~kbarrett/8213481/open.01.inc/ Thanks for doing that. David ----- From jini.george at oracle.com Mon Dec 3 07:57:36 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 3 Dec 2018 13:27:36 +0530 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <1da44101-954c-c560-c332-af82bac2abec@oracle.com> Hi Roman, A few comments on the SA changes: ==> Could you please add the following lines in src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/HSDB.java from line 1120 onwards to avoid the "[Unknown generation]" message with hsdb while displaying the Stack Memory for a mutator thread ? else if (collHeap instanceof ShenandoahHeap) { ShenandoahHeap heap = (ShenandoahHeap) collHeap; anno = "ShenandoahHeap "; bad = false; } ==> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java The printGCAlgorithm() method would need to be updated to read in the UseShenandoahGC flag to avoid the default "Mark Sweep Compact GC" being displayed with jhsdb jmap -heap. ==> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCName.java Could you please add "Shenandoah" to the GCName enum list ? ==> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java Could you please update the GCCause enum values to include these: _shenandoah_stop_vm, _shenandoah_allocation_failure_evac, _shenandoah_concurrent_gc, _shenandoah_traversal_gc, _shenandoah_upgrade_to_full_gc, ==> share/classes/sun/jvm/hotspot/runtime/VMOps.java It would be good to add 'ShenandoahOperation' to the VMOps enum (though it is probably not in sync now). Thank you, Jini. On 12/1/2018 2:30 AM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From per.liden at oracle.com Mon Dec 3 09:05:52 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 3 Dec 2018 10:05:52 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8d5b0cd0-d791-3c77-7302-eea97012897e@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <566619f4-87f5-a605-ca49-8b9083fc17eb@oracle.com> <20c6ccb0-4161-90e9-a6b7-f1aff4c05bef@redhat.com> <169dbd79-2ae1-8c9a-67b2-bc3a9f0f01a7@oracle.com> <8d5b0cd0-d791-3c77-7302-eea97012897e@redhat.com> Message-ID: Hi, On 11/28/18 3:24 PM, Roman Kennke wrote: >>> Hi Per, >>> >>>> Hi Roman, >>>> >>>> On 11/26/18 10:39 PM, Roman Kennke wrote: >>>> [...] >>>>> ???*) shared-serviceability >>>>> >>>>> >>>>> >>>>> ????? - The usual code to support another GC >>>> >>>> Just had a quick look at the SA part. I was thinking you'd have the same >>>> problem as ZGC here, with regards to parsing the heap and potentially >>>> reading garbage when you step on a Klass* which had been unloaded? >>> >>> Possible. I am myself not very familiar with SA. I guess it depends on >>> how SA does it: if it iterates objects via CH::object_iterate() (e.g. >>> same entry point as, e.g., heap-dumping code), then we should be fine. >>> We're kicking off a traversal rather than straight scan there. If >>> however SA somehow makes a raw scan itself, then we'd have the problem >>> you describe. >> >> The SA does a raw scan itself, which is the root of the problem. >> ObejctHeap.iterateLiveRegions() will locate the first object in a region >> by doing >> >> ? OopHandle handle = bottom.addOffsetToAsOopHandle(0); >> >> and to get the next object it does >> >> ? handle.addOffsetToAsOopHandle(obj.getObjectSize()); >> >> and you'll crash. So I'm afraid this will not work for Shenandoah either. > > Alright. I'll 'disable' it like you did with ZGC then. Thanks for > pointing it out. > > I'm wondering: this would crash with G1 and > +ClassUnloadingWithConcurrentMark too, then? That's a very good point and I think you're actually right. Would be good if someone from the G1 camp could confirm. cheers, Per From lutz.schmidt at sap.com Mon Dec 3 10:41:18 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Mon, 3 Dec 2018 10:41:18 +0000 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* Message-ID: Hi all, may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ Thanks, Lutz From tobias.hartmann at oracle.com Mon Dec 3 11:07:39 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 3 Dec 2018 12:07:39 +0100 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: References: Message-ID: <72acc6d8-55f9-cfb9-8558-a145b669ca15@oracle.com> Hi Lutz, this looks good to me. Best regards, Tobias On 03.12.18 11:41, Schmidt, Lutz wrote: > Hi all, > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > Thanks, > Lutz > > From thomas.stuefe at gmail.com Mon Dec 3 11:13:36 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 3 Dec 2018 12:13:36 +0100 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: References: Message-ID: This makes much more sense than control via UL and mirrors what we do with e.g. NMT. There, statistic printout is triggered by jcmd and with -XX:+PrintNMTStatistics. Change itself looks good. I guess you need either a CSR or make the switch diagnostic (again, like PrintNMTStatistics). ..Thomas On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > Hi all, > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > Thanks, > Lutz > > From lutz.schmidt at sap.com Mon Dec 3 11:42:18 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Mon, 3 Dec 2018 11:42:18 +0000 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: References: Message-ID: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> Hi Tobias, Thomas, thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. Thanks, Lutz ?On 03.12.18, 12:13, "Thomas St?fe" wrote: This makes much more sense than control via UL and mirrors what we do with e.g. NMT. There, statistic printout is triggered by jcmd and with -XX:+PrintNMTStatistics. Change itself looks good. I guess you need either a CSR or make the switch diagnostic (again, like PrintNMTStatistics). ..Thomas On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > Hi all, > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > Thanks, > Lutz > > From rkennke at redhat.com Mon Dec 3 12:30:19 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 3 Dec 2018 13:30:19 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <1da44101-954c-c560-c332-af82bac2abec@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <1da44101-954c-c560-c332-af82bac2abec@oracle.com> Message-ID: <479eaac8-73a9-4a07-0e7e-2c8bdb672bf4@redhat.com> Hi Jini, Thanks for your suggestions. I've added this to Shenandoah's dev: http://cr.openjdk.java.net/~rkennke/shenandoah-sa/webrev.01/ and it will show up in next round of webrevs. Thanks, Roman > A few comments on the SA changes: > > ==> Could you please add the following lines in > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/HSDB.java from line > 1120 onwards to avoid the "[Unknown generation]" message with hsdb while > displaying the Stack Memory for a mutator thread ? > > else if (collHeap instanceof ShenandoahHeap) { > ?? ShenandoahHeap heap = (ShenandoahHeap) collHeap; > ?? anno = "ShenandoahHeap "; > ?? bad = false; > } > > ==> > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java > > The printGCAlgorithm() method would need to be updated to read in the > UseShenandoahGC flag to avoid the default "Mark Sweep Compact GC" being > displayed with jhsdb jmap -heap. > > ==> > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCName.java > > Could you please add "Shenandoah" to the GCName enum list ? > > ==> > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java > > Could you please update the GCCause enum values to include these: > > ??? _shenandoah_stop_vm, > ??? _shenandoah_allocation_failure_evac, > ??? _shenandoah_concurrent_gc, > ??? _shenandoah_traversal_gc, > ??? _shenandoah_upgrade_to_full_gc, > > ==> share/classes/sun/jvm/hotspot/runtime/VMOps.java > > It would be good to add 'ShenandoahOperation' to the VMOps enum (though > it is probably not in sync now). > > Thank you, > Jini. > > On 12/1/2018 2:30 AM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> From thomas.schatzl at oracle.com Mon Dec 3 12:42:28 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 03 Dec 2018 13:42:28 +0100 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> Message-ID: <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> Hi, On Fri, 2018-11-30 at 11:11 +0100, Thomas Schatzl wrote: > Hi Amit, > > [...] > > > > Sorry. Please check the attached patch. > > I created a webrev at > http://cr.openjdk.java.net/~tschatzl/8213827/webrev/ . > > Let the reviews begin :) > style issues: - random whitespaces in many places. - all if's or if-then-else require braces - only class members have a leading underscore - do not declare local variables long before they are used. That makes the code hard to read. - do not introduce local variables for no reason. E.g. 2907 struct bitmask *bmp; [...unrelated code...] 2911 bmp = _numa_get_interleave_mask(); 2912 set_numa_interleave_ptr(&bmp); 2913 bmp = _numa_get_membind(); 2914 set_numa_membind_ptr(&bmp); is much less readable than set_numa_interleave_ptr(_numa_get_interleave_mask()); set_numa_membind_ptr(_numa_get_membind_mask()); - in the type declaration, please put the * next to the type. While this is kind-of wrong, the majority of existing code does so (not only in these files). code issues: - interleave/membind mode are mutually exclusive, which the code kind-of tries to establish (and check) using special values for eg. the value of Linux::_numa_interleave_ptr. I think the code gets a lot more readable if you introduce an enum and an explicit global variable holding the current NUMA node. If you do not want to do that, move the condition "Linux::_numa_interleave_ptr != 0" into a new method. - move the new NUMA initialization code in os::init_2() into a new method please. It's gotten very long now. - please avoid duplicating code; ie. 5039 // Check for membind mode. 5040 bmp = Linux::_numa_membind_ptr; 5041 for (int node = 0; node <= Linux::numa_max_node() ; node++) { 5042 if (Linux::_numa_bitmask_isbitset(bmp, node)) { 5043 _is_membind = true; 5044 } 5045 } 5048 bmp = Linux::_numa_interleave_ptr; 5049 for (int node = 0; node <= Linux::numa_max_node() ; node++) { 5050 if (Linux::_numa_bitmask_isbitset(bmp, node)) { 5051 _is_interleaved = true; 5052 // Set membind to false as interleave mode allows all nodes to be used. 5053 _is_membind = false; 5054 } 5055 } and right after: Something like: is_interleave = is_any_bit_in_bitmask_set(Linux::_numa_interleave_ptr); is_membind = !is_interleave && is_any_bit_in_bitmask_set(Linux::_numa_membind_ptr); with an appropriate is_any_bit_in_bitmask_set() helper method is much more clear. (Even better, as suggested above use a global "NUMA mode" enum that is filled in somewhere instead of multiple flags) - Use a LogStream to print the log messages instead of rolling your own code to create a string containing the node number. The code that is there exhibits a potential buffer overflow (with many NUMA nodes), and a potential out-of-bounds write. - the log message for selected NUMA mode and nodes should be a single log message. - partially pre-existing: the numa_set_xxx_ptr methods take a "struct bitmask**". Could you explain why "**"? All of those store the dereferenced result anyway, so why not pass the dereferenced pointer right away? This also simplifies the code at the callers. - partially pre-existing: I would prefer if the numa_set_xx_ptr methods were all renamed to numa_set_xx_bitmask. The "ptr" suffix seems redundant, and most importantly very unspecific. - in numa_interleave_memory(), why does the code not make the membind/interleave mode distinction when the v1 API is used? That method could probably cleaned up a bit, i.e. extracting common code. - Pre-existing: numa_interleave_memory() could be cleaned up; why do we have two pointers to _numa_interleave_memory_v2 and _numa_interleave_memory around if their parameters are the same and we only ever use one of them? That might be an additional cleanup that needs to be done. - please split out the change that disables UseNUMA if there is a single node only. It seems relatively unrelated, and so should imho be looked at (evaluated) separately. While reviewing I already fixed a few of the above issues, see http://cr.openjdk.java.net/~tschatzl/8213827/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8213827/webrev.1 (full) webrevs. You can use it as starting point. Thanks, Thomas From per.liden at oracle.com Mon Dec 3 12:45:31 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 3 Dec 2018 13:45:31 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <66f45e27-9e75-42ca-2a9b-c77166d38533@oracle.com> Hi Roman, On 11/30/18 10:00 PM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W Thanks for fixing. Looks good to me, except it looks like you missed adjusting the macro order in the following files: src/hotspot/share/gc/shared/gc_globals.hpp src/hotspot/share/gc/shared/vmStructs_gc.hpp cheers, Per > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From rkennke at redhat.com Mon Dec 3 13:24:19 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 3 Dec 2018 14:24:19 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <66f45e27-9e75-42ca-2a9b-c77166d38533@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <66f45e27-9e75-42ca-2a9b-c77166d38533@oracle.com> Message-ID: <11a96945-7b8e-f30c-0e29-cd532f6addef@redhat.com> Hi Per, Thanks for looking again. I've fixed the ordering in shenandoah-dev: http://cr.openjdk.java.net/~rkennke/fix-macro-order/webrev.00/ and it will apear in the next round of webrevs. Thanks, Roman > Hi Roman, > > On 11/30/18 10:00 PM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > Thanks for fixing. Looks good to me, except it looks like you missed > adjusting the macro order in the following files: > ?src/hotspot/share/gc/shared/gc_globals.hpp > ?src/hotspot/share/gc/shared/vmStructs_gc.hpp > > cheers, > Per > >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> From zgu at redhat.com Mon Dec 3 14:33:00 2018 From: zgu at redhat.com (zgu at redhat.com) Date: Mon, 03 Dec 2018 09:33:00 -0500 Subject: Extend NMT to JDK native libraries? In-Reply-To: References: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> <22785794-bee7-98e9-2151-c8487f65744a@redhat.com> Message-ID: <1543847580.22375.11.camel@redhat.com> On Mon, 2018-11-26 at 21:43 +0100, Thomas St?fe wrote: Hi Thomas, Sorry for replying so late. > Hi Zhengyu, > > > On Wed, Nov 21, 2018 at 5:26 PM Zhengyu Gu wrote: > > > > > > > > But note the unbalanced-malloc issue. The more we expose NMT to > > > the > > > JDK wild, the more probable are such bugs. Within the hotspot all > > > is > > > nice and tidy. > > > > Hotspot also has a few unbalanced-malloc instances. > > Oh, where? Really, raw malloc with os::free, or the other way around? Maybe I misunderstand what you mean of unbalanced-malloc. I was thinking something like [1], the library calls that potential realloc buffers? or could you point me some examples in JDK library code that you referred to? There is another hurdle in my mind, I am not sure how did you solve it with hashtable approach, that is GuardedMemory, although, it is debug only feature, it also installs a header. Could you elaborate? Thanks, -Zhengyu > > Would they not create the problem I described, corrupted C-heap etc? > > > And yes, we will > > find more in library, I would consider it as initial investment to > > fix > > them, especially, if we can do them module-by-module, no? > > Yes, they should be fixed if possible. My concern is that it is > realistic to assume that we miss some, or even if not that new ones > will creep in if the code changes. And that the penalties for these > unbalanced mallocs is severe - crashes and heap corruptions - if we > use malloc headers. > > Even worse, they would only show up, in release builds, if NMT is > switched on, since only NMT adds malloc headers in release builds. > That would be annoying, telling a customer to switch on NMT just for > him to hit such a code path and crash the VM. > > I don't know, I seem to miss something here. Do you not see this as > serious problem preventing NMT from covering code outside hotspot? > > > > > Mismatched statistics is quite annoying ... Did see people actually > > counting bytes and expecting to match :-) JDK-8191369 actually was > > driven by customers, who tried to match smap. > > > > I remember :) and yes, we have many curious customers too. But that > is > fine. We want NMT to be as exact as possible. > > Cheers, Thomas > > > > > Thanks, > > > > -Zhengyu > > > > > > > > If we wanted to avoid these bugs, we would have to remove malloc > > > headers from both os::malloc() and NMT MallocTracker and add a > > > malloc > > > pointer hashtable of some sorts to the latter. This is not very > > > difficult, but would still need an initial investment. > > > > > > > > > > > > Thanks, Thomas > > > > > > > > > > > Thanks, > > > > > > > > -Zhengyu > > > > > > > > > > > > > > (I think even if we were to instrument parts of the JDK - > > > > > e.g. just > > > > > NIO - this would already be very helpful. In parts we do this > > > > > already > > > > > for mtOther.). > > > > > > > > > > On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu > > > > > wrote: > > > > > > > > > > > > FYI: There was a phase 2 RFE: Native Memory Tracking (Phase > > > > > > 2) > > > > > > https://bugs.openjdk.java.net/browse/JDK-6995789 > > > > > > > > > > > > -Zhengyu > > > > > > > > > > > > On 11/21/18 9:28 AM, Thomas St?fe wrote: > > > > > > > Hi all, > > > > > > > > > > > > > > (yet again not sure if this is serviceablity-dev or not - > > > > > > > I start at > > > > > > > hs-dev, feel free to move this mail around.) > > > > > > > > > > > > > > Do we have any plans to extend NMT to cover native JDK > > > > > > > libaries too? > > > > > > > That would be a really cool feature. > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > We at SAP have done a similar thing in the past: > > > > > > > > > > > > > > We have a monitoring facility in our port which tracks C- > > > > > > > heap > > > > > > > allocations, non-imaginatively called "malloc statistic". > > > > > > > This feature > > > > > > > predates NMT somewhat - had we had NMT at that time, we > > > > > > > would not have > > > > > > > bothered. Our Malloc statistic is less powerful than NMT > > > > > > > and > > > > > > > implementation-wise completely at odds with it, so I > > > > > > > never felt the > > > > > > > urge to bring it upstream. However, one thing we did do > > > > > > > is we extended > > > > > > > its coverage to the JDK native code. > > > > > > > > > > > > > > This has been quite helpful in the past to find leaks in > > > > > > > JDK, see > > > > > > > e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 > > > > > > > > > > > > > > We did this by exposing os::malloc, os::free etc from > > > > > > > libjvm.so > > > > > > > ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK > > > > > > > native code, we > > > > > > > then either manually replaced calls to raw ::malloc(), > > > > > > > ::free() etc > > > > > > > with JVM_malloc(), JVM_free(). Or, in places where this > > > > > > > was possible, > > > > > > > we did this replacement stuff wholesale by employing a > > > > > > > header which > > > > > > > re-defined malloc(), free() etc JVM_malloc, JVM_free etc. > > > > > > > Of course, > > > > > > > we also had to add a number of linkage dependencies to > > > > > > > the libjvm.so. > > > > > > > > > > > > > > All this is pretty standard stuff. > > > > > > > > > > > > > > One detail stood out: malloc headers are evil. In our > > > > > > > experience, JDK > > > > > > > native code was more difficult to control and "unbalanced > > > > > > > malloc/frees" kept creeping in - especially with the > > > > > > > wholesale-redefinition technique. Unbalanced > > > > > > > mallocs/frees means cases > > > > > > > where malloc() is instrumented but ::free() stays raw, or > > > > > > > the other > > > > > > > way around. Both combinations are catastrophic since > > > > > > > os::malloc uses > > > > > > > malloc headers. We typically corrupted the C-Heap and > > > > > > > crashed, often > > > > > > > much later in completely unrelated places. > > > > > > > > > > > > > > These types of bugs were very hard to spot and hence very > > > > > > > expensive. > > > > > > > And they can creep in in many ways. One example, there > > > > > > > exist a > > > > > > > surprising number of system APIs which return results in > > > > > > > C-heap and > > > > > > > require the user to free that, which of course must > > > > > > > happen with raw > > > > > > > ::free(), not os::free(). > > > > > > > > > > > > > > We fixed this by not using malloc headers. That means a > > > > > > > pointer > > > > > > > returned by os::malloc() is compatible with raw ::free() > > > > > > > and vice > > > > > > > versa. The only bad thing happening would be our > > > > > > > statistic numbers > > > > > > > being slightly off. > > > > > > > > > > > > > > Instead of malloc headers we use a hand-groomed hash > > > > > > > table to track > > > > > > > the malloced memory. It is actually quite fast, fast > > > > > > > enough that this > > > > > > > malloc statistic feature is on-by-default in our port. > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Of course, if we extend NMT to JDK native code we also > > > > > > > would want to > > > > > > > extend it to mmap() etc - we never did this with our > > > > > > > statistic, since > > > > > > > it only tracked malloc. > > > > > > > > > > > > > > What do you think? Did anyone else play with similar > > > > > > > ideas? Would it > > > > > > > be worth the effort? > > > > > > > > > > > > > > Cheers, Thomas > > > > > > > From erik.osterlund at oracle.com Mon Dec 3 15:35:39 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 3 Dec 2018 16:35:39 +0100 Subject: RFR: 8214257: IC cache not clean after cleaning assertion failure In-Reply-To: <81c911f1-81a1-2187-1dc9-7990e4c4d0e7@oracle.com> References: <81c911f1-81a1-2187-1dc9-7990e4c4d0e7@oracle.com> Message-ID: <80f9edb3-3704-1aa9-08a9-1c8992b56abd@oracle.com> Hi Vladimir, Thanks for the review. /Erik On 2018-11-28 19:25, Vladimir Kozlov wrote: > Looks good to me. > > Thanks, > Vladimir > > On 11/27/18 5:00 AM, Erik ?sterlund wrote: >> Hi, >> >> Back in 8212681, an abstract CompiledICLocker was introduce to protect >> inline caches from concurrent patching. >> >> Unfortunately, not all patching of ICs started using the >> CompiledICLocker. There were still some callsites using the >> Patching_lock. Therefore, the assert may trigger sometimes saying the >> IC cache is not clean right after cleaning, due to other concurrent >> modifications under the Patching_lock. >> >> Also, in CompiledIC::set_to_clean(), the condition for safe_transition >> was altered incorrectly from is_at_safepoint() to >> CompiledICLocker::is_safe(), causing cleaning to not used transition, >> so I reinstated that in this patch. >> >> This patch depends on 8214338 by assuming that the CompiledICLocker >> does not use safepoint checks, as it is now used from leaf calls where >> safepointing is not allowed. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8214257 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8214257/webrev.00/ >> >> Thanks, >> /Erik From erik.osterlund at oracle.com Mon Dec 3 15:39:44 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 3 Dec 2018 16:39:44 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> Message-ID: <7d75efb2-337b-9722-6636-97d1152c656c@oracle.com> Any more takers? I need one more review. I have passed hs-tier1-6 twice with this now. /Erik On 2018-11-30 12:19, Erik ?sterlund wrote: > Hi Dean, > > Thanks for reviewing this! > > On 2018-11-27 23:55, dean.long at oracle.com wrote: >> There's a typo in the new guarantee message: "tranisition". > > Fixed in-place. > >> Regarding the infinite loops, can we detect if progress isn't being >> made and assert/panic, rather than hanging? > > I'm afraid not. It's a similar analogy to the problem of progress > guarantees of a lock-free algorithm. They have a global progress > guarantee for the system, but no local guarantee of progress for each > operation, that can in theory be starved indefinitely due to other > operations winning every race. The situation is similar here, at least > in theory, that the thread refilling the IC stubs didn't get a single > stub back, because another thread started depleted the IC stubs again. > Naturally, if this was ever to happen, we would get time outs in our > tests though, to indicate that there is a problem. > > Thanks, > /Erik > >> dl >> >> On 11/27/18 1:29 PM, Erik ?sterlund wrote: >>> Hi Dean, >>> >>> Thank you for reviewing this. >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >>> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >>> >>> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>>> Hi Erik.? Can you explain why you removed the pre-allocated "next >>>> stub"?? I guess it was no longer necessary?? If so, then you should >>>> update the "always has sentinel" comment in is_empty. >>> >>> Updated. Indeed, the sentinel no longer serves a purpose so I removed >>> it. Found some more traces of the sentinel that I removed in the last >>> webrev. >>> >>>> In your guarantee messages, some use "should not fail" and some use >>>> "should never fail".? It's not a big deal but maybe they should be >>>> the same. >>> >>> Sure. Updated. >>> >>>> You introduced a couple of infinite loops.? Are we guaranteed to >>>> exit these loops eventually?? Is there an upper bound on how many >>>> iterations are needed? >>> >>> In my latest webrev I removed some unnecessary set_to_clean() where >>> IC caches are already is_clean(). With that in place, there should be >>> global progress guarantees and a single IC stub in the buffer should >>> be all you really "need". Although, you might want more. In fact, I'm >>> questioning if the 10K size buffer is big enough today, but I'll >>> leave that for another day. >>> >>> Thanks, >>> /Erik >>> >>>> The rest looks good. >>>> >>>> dl >>>> >>>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> IC stub refilling currently requires a safepoint operation. When >>>>> done right at the point where an CompiledIC is about to get patched >>>>> to a transitional state using an IC stub, locks may already be >>>>> held, causing a bunch of locking issues - especially for concurrent >>>>> class unloading. Therefore, the IC stub refilling ought to be moved >>>>> out so that IC cache cleaning and transitioning may be done without >>>>> any safepoints, and the locks in the path ought to not perform >>>>> safepoint checking. >>>>> >>>>> This is implemented by allowing IC transitions to fail when they >>>>> require IC stubs, and we run out of them. This propages back to a >>>>> higher level where IC stubs are refilled after having released the >>>>> IC locker. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>>> >>>>> Thanks, >>>>> /Erik >>>> >> From vladimir.kozlov at oracle.com Mon Dec 3 15:44:54 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Dec 2018 07:44:54 -0800 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> Message-ID: Please, fix comments which describe usage in codeHeapState.cpp Thanks, Vladimir On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > Hi Tobias, Thomas, > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > Thanks, > Lutz > > ?On 03.12.18, 12:13, "Thomas St?fe" wrote: > > This makes much more sense than control via UL and mirrors what we do > with e.g. NMT. There, statistic printout is triggered by jcmd and with > -XX:+PrintNMTStatistics. > > Change itself looks good. I guess you need either a CSR or make the > switch diagnostic (again, like PrintNMTStatistics). > > ..Thomas > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > Hi all, > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > Thanks, > > Lutz > > > > > > From lutz.schmidt at sap.com Mon Dec 3 15:54:57 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Mon, 3 Dec 2018 15:54:57 +0000 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> Message-ID: <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> Oooohhh yes, good point. Working on it... Lutz ?On 03.12.18, 16:44, "Vladimir Kozlov" wrote: Please, fix comments which describe usage in codeHeapState.cpp Thanks, Vladimir On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > Hi Tobias, Thomas, > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > Thanks, > Lutz > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > This makes much more sense than control via UL and mirrors what we do > with e.g. NMT. There, statistic printout is triggered by jcmd and with > -XX:+PrintNMTStatistics. > > Change itself looks good. I guess you need either a CSR or make the > switch diagnostic (again, like PrintNMTStatistics). > > ..Thomas > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > Hi all, > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > Thanks, > > Lutz > > > > > > From lutz.schmidt at sap.com Mon Dec 3 16:49:12 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Mon, 3 Dec 2018 16:49:12 +0000 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> Message-ID: <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> Hi, description in codeHeapState.cpp is adapted. In addition, I changed the name of the command line option to PrintCodeHeapAnalytics to better match the jcmd function name. New webrev is here: http://cr.openjdk.java.net/~lucy/webrevs/8214526.01/ Thanks, Lutz ?On 03.12.18, 16:54, "Schmidt, Lutz" wrote: Oooohhh yes, good point. Working on it... Lutz On 03.12.18, 16:44, "Vladimir Kozlov" wrote: Please, fix comments which describe usage in codeHeapState.cpp Thanks, Vladimir On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > Hi Tobias, Thomas, > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > Thanks, > Lutz > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > This makes much more sense than control via UL and mirrors what we do > with e.g. NMT. There, statistic printout is triggered by jcmd and with > -XX:+PrintNMTStatistics. > > Change itself looks good. I guess you need either a CSR or make the > switch diagnostic (again, like PrintNMTStatistics). > > ..Thomas > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > Hi all, > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > Thanks, > > Lutz > > > > > > From vladimir.kozlov at oracle.com Mon Dec 3 16:54:52 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Dec 2018 08:54:52 -0800 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> Message-ID: <7f47f405-8f6b-9527-6aeb-62f442c31531@oracle.com> Good. Thanks, Vladimir On 12/3/18 8:49 AM, Schmidt, Lutz wrote: > Hi, > description in codeHeapState.cpp is adapted. In addition, I changed the name of the command line option to PrintCodeHeapAnalytics to better match the jcmd function name. > New webrev is here: http://cr.openjdk.java.net/~lucy/webrevs/8214526.01/ > Thanks, > Lutz > > ?On 03.12.18, 16:54, "Schmidt, Lutz" wrote: > > Oooohhh yes, good point. > Working on it... > Lutz > > On 03.12.18, 16:44, "Vladimir Kozlov" wrote: > > Please, fix comments which describe usage in codeHeapState.cpp > > Thanks, > Vladimir > > On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > > Hi Tobias, Thomas, > > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > > Thanks, > > Lutz > > > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > > > This makes much more sense than control via UL and mirrors what we do > > with e.g. NMT. There, statistic printout is triggered by jcmd and with > > -XX:+PrintNMTStatistics. > > > > Change itself looks good. I guess you need either a CSR or make the > > switch diagnostic (again, like PrintNMTStatistics). > > > > ..Thomas > > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > > > Hi all, > > > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > > > Thanks, > > > Lutz > > > > > > > > > > > > > > From tobias.hartmann at oracle.com Mon Dec 3 17:00:28 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 3 Dec 2018 18:00:28 +0100 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> Message-ID: Looks good to me too. Best regards, Tobias On 03.12.18 17:49, Schmidt, Lutz wrote: > Hi, > description in codeHeapState.cpp is adapted. In addition, I changed the name of the command line option to PrintCodeHeapAnalytics to better match the jcmd function name. > New webrev is here: http://cr.openjdk.java.net/~lucy/webrevs/8214526.01/ > Thanks, > Lutz > > ?On 03.12.18, 16:54, "Schmidt, Lutz" wrote: > > Oooohhh yes, good point. > Working on it... > Lutz > > On 03.12.18, 16:44, "Vladimir Kozlov" wrote: > > Please, fix comments which describe usage in codeHeapState.cpp > > Thanks, > Vladimir > > On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > > Hi Tobias, Thomas, > > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > > Thanks, > > Lutz > > > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > > > This makes much more sense than control via UL and mirrors what we do > > with e.g. NMT. There, statistic printout is triggered by jcmd and with > > -XX:+PrintNMTStatistics. > > > > Change itself looks good. I guess you need either a CSR or make the > > switch diagnostic (again, like PrintNMTStatistics). > > > > ..Thomas > > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > > > Hi all, > > > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > > > Thanks, > > > Lutz > > > > > > > > > > > > > > From thomas.stuefe at gmail.com Mon Dec 3 17:20:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 3 Dec 2018 18:20:10 +0100 Subject: Extend NMT to JDK native libraries? In-Reply-To: <1543847580.22375.11.camel@redhat.com> References: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> <22785794-bee7-98e9-2151-c8487f65744a@redhat.com> <1543847580.22375.11.camel@redhat.com> Message-ID: Hi Zhengyu, On Mon, Dec 3, 2018 at 3:33 PM wrote: > > On Mon, 2018-11-26 at 21:43 +0100, Thomas St?fe wrote: > > Hi Thomas, > > Sorry for replying so late. > no problem, this is a backburner discussion :) > > Oh, where? Really, raw malloc with os::free, or the other way around? > > Maybe I misunderstand what you mean of unbalanced-malloc. I was > thinking something like [1], the library calls that potential realloc > buffers? or could you point me some examples in JDK library code that > you referred to? I do not have concreate examples, but our experience, when we did this, was that it was very difficult to get right 100% of the time. And it bitrots very easily. For example, if you go the wholesale-redefine-malloc-road, and someone accidentally changes the includes and one of the c files now misses your redefinition includes and falls back to raw malloc. Or if you miss a system API which returns malloced space. Basically, this is like herding cats, very difficult to catch every single one :) All this is solvable given enough manpower, but after trying a bit we went another road and just avoided malloc headers. That way, we can live with 99% coverage - the remaining 1% are annoying but do not crash the system, and we can hunt them down at our leisure. Hotspot is less of a problem, because the project scope is small and there is a limited number of devs which all are all experienced and know to avoid raw malloc() and free(). Plus I think we have safeguards at build time now. > > There is another hurdle in my mind, I am not sure how did you solve it > with hashtable approach, that is GuardedMemory, although, it is debug > only feature, it also installs a header. Could you elaborate? We just did not use it. Basically, we rolled our own os::malloc/os::free implementation. To catch errors, we use two techniques: - we still have trailing guards - quite heavyweight but powerful: we have a mode were every single allocation gets a full page and is preceeded/succeeded by a protected page (a bit smarter than that, we need just 2 pages per allocation). Overflowing boundaries will instantly crash. Also, we protect the payload page after its allocation has been free()d, for a certain amount of time, and therefore can catch use-after-free errors. I always found the mechanism neat, but it is completely at odds with the current hotspot implementation of os::malloc and NMT, so we never attempted to bring it upstream. Cheers, Thomas > > Thanks, > > -Zhengyu > > > > Would they not create the problem I described, corrupted C-heap etc? > > > > > And yes, we will > > > find more in library, I would consider it as initial investment to > > > fix > > > them, especially, if we can do them module-by-module, no? > > > > Yes, they should be fixed if possible. My concern is that it is > > realistic to assume that we miss some, or even if not that new ones > > will creep in if the code changes. And that the penalties for these > > unbalanced mallocs is severe - crashes and heap corruptions - if we > > use malloc headers. > > > > Even worse, they would only show up, in release builds, if NMT is > > switched on, since only NMT adds malloc headers in release builds. > > That would be annoying, telling a customer to switch on NMT just for > > him to hit such a code path and crash the VM. > > > > I don't know, I seem to miss something here. Do you not see this as > > serious problem preventing NMT from covering code outside hotspot? > > > > > > > > Mismatched statistics is quite annoying ... Did see people actually > > > counting bytes and expecting to match :-) JDK-8191369 actually was > > > driven by customers, who tried to match smap. > > > > > > > I remember :) and yes, we have many curious customers too. But that > > is > > fine. We want NMT to be as exact as possible. > > > > Cheers, Thomas > > > > > > > > Thanks, > > > > > > -Zhengyu > > > > > > > > > > > If we wanted to avoid these bugs, we would have to remove malloc > > > > headers from both os::malloc() and NMT MallocTracker and add a > > > > malloc > > > > pointer hashtable of some sorts to the latter. This is not very > > > > difficult, but would still need an initial investment. > > > > > > > > > > > > > > > > > Thanks, Thomas > > > > > > > > > > > > > > Thanks, > > > > > > > > > > -Zhengyu > > > > > > > > > > > > > > > > > (I think even if we were to instrument parts of the JDK - > > > > > > e.g. just > > > > > > NIO - this would already be very helpful. In parts we do this > > > > > > already > > > > > > for mtOther.). > > > > > > > > > > > > On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu > > > > > > wrote: > > > > > > > > > > > > > > FYI: There was a phase 2 RFE: Native Memory Tracking (Phase > > > > > > > 2) > > > > > > > https://bugs.openjdk.java.net/browse/JDK-6995789 > > > > > > > > > > > > > > -Zhengyu > > > > > > > > > > > > > > On 11/21/18 9:28 AM, Thomas St?fe wrote: > > > > > > > > Hi all, > > > > > > > > > > > > > > > > (yet again not sure if this is serviceablity-dev or not - > > > > > > > > I start at > > > > > > > > hs-dev, feel free to move this mail around.) > > > > > > > > > > > > > > > > Do we have any plans to extend NMT to cover native JDK > > > > > > > > libaries too? > > > > > > > > That would be a really cool feature. > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > We at SAP have done a similar thing in the past: > > > > > > > > > > > > > > > > We have a monitoring facility in our port which tracks C- > > > > > > > > heap > > > > > > > > allocations, non-imaginatively called "malloc statistic". > > > > > > > > This feature > > > > > > > > predates NMT somewhat - had we had NMT at that time, we > > > > > > > > would not have > > > > > > > > bothered. Our Malloc statistic is less powerful than NMT > > > > > > > > and > > > > > > > > implementation-wise completely at odds with it, so I > > > > > > > > never felt the > > > > > > > > urge to bring it upstream. However, one thing we did do > > > > > > > > is we extended > > > > > > > > its coverage to the JDK native code. > > > > > > > > > > > > > > > > This has been quite helpful in the past to find leaks in > > > > > > > > JDK, see > > > > > > > > e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 > > > > > > > > > > > > > > > > We did this by exposing os::malloc, os::free etc from > > > > > > > > libjvm.so > > > > > > > > ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK > > > > > > > > native code, we > > > > > > > > then either manually replaced calls to raw ::malloc(), > > > > > > > > ::free() etc > > > > > > > > with JVM_malloc(), JVM_free(). Or, in places where this > > > > > > > > was possible, > > > > > > > > we did this replacement stuff wholesale by employing a > > > > > > > > header which > > > > > > > > re-defined malloc(), free() etc JVM_malloc, JVM_free etc. > > > > > > > > Of course, > > > > > > > > we also had to add a number of linkage dependencies to > > > > > > > > the libjvm.so. > > > > > > > > > > > > > > > > All this is pretty standard stuff. > > > > > > > > > > > > > > > > One detail stood out: malloc headers are evil. In our > > > > > > > > experience, JDK > > > > > > > > native code was more difficult to control and "unbalanced > > > > > > > > malloc/frees" kept creeping in - especially with the > > > > > > > > wholesale-redefinition technique. Unbalanced > > > > > > > > mallocs/frees means cases > > > > > > > > where malloc() is instrumented but ::free() stays raw, or > > > > > > > > the other > > > > > > > > way around. Both combinations are catastrophic since > > > > > > > > os::malloc uses > > > > > > > > malloc headers. We typically corrupted the C-Heap and > > > > > > > > crashed, often > > > > > > > > much later in completely unrelated places. > > > > > > > > > > > > > > > > These types of bugs were very hard to spot and hence very > > > > > > > > expensive. > > > > > > > > And they can creep in in many ways. One example, there > > > > > > > > exist a > > > > > > > > surprising number of system APIs which return results in > > > > > > > > C-heap and > > > > > > > > require the user to free that, which of course must > > > > > > > > happen with raw > > > > > > > > ::free(), not os::free(). > > > > > > > > > > > > > > > > We fixed this by not using malloc headers. That means a > > > > > > > > pointer > > > > > > > > returned by os::malloc() is compatible with raw ::free() > > > > > > > > and vice > > > > > > > > versa. The only bad thing happening would be our > > > > > > > > statistic numbers > > > > > > > > being slightly off. > > > > > > > > > > > > > > > > Instead of malloc headers we use a hand-groomed hash > > > > > > > > table to track > > > > > > > > the malloced memory. It is actually quite fast, fast > > > > > > > > enough that this > > > > > > > > malloc statistic feature is on-by-default in our port. > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > Of course, if we extend NMT to JDK native code we also > > > > > > > > would want to > > > > > > > > extend it to mmap() etc - we never did this with our > > > > > > > > statistic, since > > > > > > > > it only tracked malloc. > > > > > > > > > > > > > > > > What do you think? Did anyone else play with similar > > > > > > > > ideas? Would it > > > > > > > > be worth the effort? > > > > > > > > > > > > > > > > Cheers, Thomas > > > > > > > > From coleen.phillimore at oracle.com Mon Dec 3 18:58:11 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 3 Dec 2018 13:58:11 -0500 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> Message-ID: <9c193e47-748d-8978-b4c4-47c9316d3fad@oracle.com> Thank you for fixing this! Coleen On 12/3/18 11:49 AM, Schmidt, Lutz wrote: > Hi, > description in codeHeapState.cpp is adapted. In addition, I changed the name of the command line option to PrintCodeHeapAnalytics to better match the jcmd function name. > New webrev is here: http://cr.openjdk.java.net/~lucy/webrevs/8214526.01/ > Thanks, > Lutz > > ?On 03.12.18, 16:54, "Schmidt, Lutz" wrote: > > Oooohhh yes, good point. > Working on it... > Lutz > > On 03.12.18, 16:44, "Vladimir Kozlov" wrote: > > Please, fix comments which describe usage in codeHeapState.cpp > > Thanks, > Vladimir > > On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > > Hi Tobias, Thomas, > > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > > Thanks, > > Lutz > > > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > > > This makes much more sense than control via UL and mirrors what we do > > with e.g. NMT. There, statistic printout is triggered by jcmd and with > > -XX:+PrintNMTStatistics. > > > > Change itself looks good. I guess you need either a CSR or make the > > switch diagnostic (again, like PrintNMTStatistics). > > > > ..Thomas > > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > > > Hi all, > > > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > > > Thanks, > > > Lutz > > > > > > > > > > > > > > From thomas.stuefe at gmail.com Mon Dec 3 19:05:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 3 Dec 2018 20:05:10 +0100 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> Message-ID: +1 ..Thomas On Mon, Dec 3, 2018 at 5:49 PM Schmidt, Lutz wrote: > > Hi, > description in codeHeapState.cpp is adapted. In addition, I changed the name of the command line option to PrintCodeHeapAnalytics to better match the jcmd function name. > New webrev is here: http://cr.openjdk.java.net/~lucy/webrevs/8214526.01/ > Thanks, > Lutz > > ?On 03.12.18, 16:54, "Schmidt, Lutz" wrote: > > Oooohhh yes, good point. > Working on it... > Lutz > > On 03.12.18, 16:44, "Vladimir Kozlov" wrote: > > Please, fix comments which describe usage in codeHeapState.cpp > > Thanks, > Vladimir > > On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > > Hi Tobias, Thomas, > > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > > Thanks, > > Lutz > > > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > > > This makes much more sense than control via UL and mirrors what we do > > with e.g. NMT. There, statistic printout is triggered by jcmd and with > > -XX:+PrintNMTStatistics. > > > > Change itself looks good. I guess you need either a CSR or make the > > switch diagnostic (again, like PrintNMTStatistics). > > > > ..Thomas > > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > > > Hi all, > > > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > > > Thanks, > > > Lutz > > > > > > > > > > > > > > From rkennke at redhat.com Mon Dec 3 19:27:04 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 3 Dec 2018 20:27:04 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Round 5 of Shenandoah review includes: - A fix for the @requires tag in TestFullGCCountTest.java. It should be correct now. We believe the CMS @requires was also not quite right and fixed it the same. It reads now: Don't run this test if: - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is true, as set by harness - Actual GC set by harness is Shenandoah *and* ExplicitGCInvokesConcurrent is not set false by harness (it's true by default in Shenandoah, so this needs to be double-inverteed). The @requires for CMS was wrong before (we think), because it would also filter defaultGC + ExplicitGCInvokesConcurrent. - Sorting of macros was fixed, as was pointed out by Per - Some stuff was added to SA, as suggested by Jini - Rebased on most current jdk/jdk code Webrevs: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ I also need reviews from GC reviewers for the CSR: https://bugs.openjdk.java.net/browse/JDK-8214349 I already got reviews for: [x] shared-runtime (coleenp) [x] shared-compiler (kvn) I got reviews for shared-build, but an earlier version, so maybe makes sense to look over this again. Erik J, Magnus? I still need approvals for: [ ] shared-build (kvn, erikj, ihse, pliden) [ ] shared-gc (pliden, kbarrett) [ ] shared-serviceability (jgeorge, pliden) [ ] shared-tests (lmesnik, pliden) [ ] shenandoah-gc [ ] shenandoah-tests Thanks for your patience and ongoing support! Cheers, Roman > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From Derek.White at cavium.com Mon Dec 3 21:55:59 2018 From: Derek.White at cavium.com (White, Derek) Date: Mon, 3 Dec 2018 21:55:59 +0000 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> , Message-ID: Hi Amith, Thanks for running the test cases for JDK-8205051. My theory was incorrect - there's a reason I didn't try to fix that bug :-) As far as clearing the UseNUMA flag goes, there's a few positions to take when UseNUMA was specified, but the system has only 1 node, or the JVM is bound to 1 node: 1. Pretend the user never set UseNUMA - turn it off. (Note that this can easily occur using JVM configs in containers). 2. Set up the JVM to use resources that it can't access - don't pay attention to binding. 3. Pay attention to binding. Single-node JVM goes through the UseNUMA paths, but correctly acts as a non-NUMA system. Currently the JVM does a combination of 1 & 2. And yes, 2 is a bug. Your patch seems to be implementing 3, which is fine, and it's great that it actually works! But it's riskier in that it has to be correct for all uses of UseNUMA (including various collectors), and for future uses of UseNUMA. I had an expectation of what UseNUMA did in this case, but GC isn't my area any more so I'll stay out of it now. - Derek ________________________________ From: amith pawar Sent: Monday, December 3, 2018 1:21 AM To: White, Derek Cc: thomas.schatzl at oracle.com; hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com; roshanmanagal at gmail.com; swatibits14 at gmail.com; roshanmangal at gmail.com Subject: Re: [PATCH] JDK NUMA Interleaving issue +Roshan Mangal On Mon, Dec 3, 2018 at 11:49 AM amith pawar > wrote: Hi Derek On Sat, Dec 1, 2018 at 4:10 AM White, Derek > wrote: Hi Amith, Thanks for tackling this! I think that your patch may fix https://bugs.openjdk.java.net/browse/JDK-8205051, although it's probably fixing more than that. I was thinking that the problem was related to calling " _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr);", and you replaced that call with more specific node sets, which seems better to me. You can try the test cases listed in the bug to see if the patch fixes the problem. Tested and didn't fix it. The existing code disables UseNUMA if the process is bound to only one node, but it looks like your patch removes this. Is that a correct read of the patch, intentional, and/or a good idea? I removed that after testing it with SPECjbb for single node case. With patch : Enabling/Disabling UseNUMA gives same score. Without patch: UseNUMA scores 10-12% lesser than without UseNUMA Thanks! - Derek ________________________________ From: hotspot-dev > on behalf of amith pawar > Sent: Thursday, November 29, 2018 6:08 AM To: thomas.schatzl at oracle.com Cc: hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com Subject: Re: [PATCH] JDK NUMA Interleaving issue External Email Hi Thomas, Sorry for the late reply and please check my inline comments below. On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl > wrote: > Hi Amith, > > welcome to the OpenJDK community! :) > Thanks > > On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: > > Hi all, > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, > > S1 and S2 region (if any other ) memory areas on requested Numa nodes > > and it should not configure itself to access other Numa nodes. This > > issue is observed only when Java is allowed to run on fewer NUMA > > nodes than available on the system with numactl membind and > > interleave options. Running on all the nodes does not have any > > effect. This will cause some applications (objects residing in old > > gen and survivor region) to run slower on system with large Numa > > nodes. > > > [... long explanation...] > > Is it possible to summarize the problem and the changes with the > following few lines: > > "NUMA interleaving of memory of old gen and survivor spaces (for > parallel GC) tells the OS to interleave memory across all nodes of a > NUMA system. However the VM process may be configured to be limited to > run only on a few nodes, which means that large parts of the heap will > be located on foreign nodes. This can incurs a large performance > penalty. > > The proposed solution is to tell the OS to interleave memory only > across available nodes when enabling NUMA." > OK. Since, I dont have write access to the defect link so can I request you do the same. > > We have had trouble understanding the problem statement and purpose of > this patch when triaging (making sure the issue is understandable and > can be worked on) as the text is rather technical. Having an easily > understandable text also helps reviewers a lot. > > Assuming my summary is appropriate, I have several other unrelated > questions: > > - could you propose a better subject for this work? "JDK NUMA > Interleaving issue" seems very generic. Something like "NUMA heap > allocation does not respect VM membind/interleave settings" maybe? > This is also OK. > > - there have been other NUMA related patches from AMD recently, in > particular JDK-what is the relation to JDK-8205051? The text there > reads awfully similar to this one, but I *think* JDK-8205051 is > actually about making sure that the parallel gc eden is not put on > inactive nodes. > Can you confirm this (talk to your colleague) so that we can fix the > description too? > JDK-8205051 is related to early GC. JDK-8189922 specific to membind case where heap was allocated on non requested NUMA nodes. The proposed patch fixes two following issues 1. Similar to JDK-8189922 but for interleave case. 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer NUMA nodes. > - fyi, we are currently working on NUMA aware memory allocation support > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to > not step on each other's toes (see below). > We are not working anything related to G1. It may effect G1 if numa_make_global function is called. > > [... broken patch...] > > I tried to apply the patch to the jdk/jdk tree, which failed; I then > started to manually apply it but stopped after not being able to find > the context of some hunks. I do not think this change applies to the > latest source tree. > > Please make sure that the patch applies to the latest jdk/jdk tree with > errors. All changes generally must first go into the latest dev tree > before you can apply for backporting. > > Could you please send the patch as attachment (not copy&pasted) to this > list and cc me? Then I can create a webrev out of it. > > I did look a bit over the patch as much as I could (it's very hard > trying to review a diff), some comments: > Sorry. Please check the attached patch. > > - the _numa_interleave_memory_v2 function pointer is never assigned > to in the patch in the CR, so it will not be used. Please make sure the > patch is complete. > Actually it is never defined anywhere, ie. the patch unlikely actually > compiles even if I could apply it somewhere. > > Please avoid frustrating reviewers by sending incomplete patches. > Sorry again. Faced same issue when copied from the defect link but worked for me from the mail. Now i have attached the patch. Can you please update defect link with this patch ? > > - I am not sure set_numa_interleave, i.e. the interleaving, should be > done without UseNUMAInterleaving enabled. Some collectors may do their > own interleaving in the future (JDK-8210473 / JEP-345) so this would > massively interfere in how they work. (That issue may be because I am > looking at a potentially incomplete diff, so forgive me if the patch > already handles this). > Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect when java externally invoked in interleave mode. It helps membind case. > > - if some of the actions (interleaving/membind) fail, and it had been > requested, it would be nice to print a log message. > Following two NUMA API's are used and they return nothing. Right now not sure which cases to handle. Please suggest void numa_tonode_memory(void *start, size_t size, int node); void numa_interleave_memory(void *start, size_t size, struct bitmask *nodemask); > > Actually it would be nice to print information about e.g. the bitmask > anyway in the log so that the user can easily verify that what he > specified on the command line has been properly picked up by the VM - > instead of looking through the proc filesystem. > As per the suggestion, patch is updated to print following information. [0.001s][info][os] UseNUMA is enabled [0.001s][info][os] Java is configured to run in interleave mode [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, 2, 3 > > Thanks, > Thomas > > > > Thanks Amit Pawar Thanks, Amit Pawar -- With best regards, amit pawar From Derek.White at cavium.com Mon Dec 3 21:56:50 2018 From: Derek.White at cavium.com (White, Derek) Date: Mon, 3 Dec 2018 21:56:50 +0000 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> , Message-ID: OK Roshan, Thanks for the update! - Derek ________________________________ From: roshan mangal Sent: Monday, December 3, 2018 1:27 AM To: amith.pawar at gmail.com Cc: White, Derek; Thomas Schatzl; hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com; roshanmanagal at gmail.com; Swati Sharma Subject: Re: [PATCH] JDK NUMA Interleaving issue External Email Hi Derek, The fix submitted by Amith is not fixing "https://bugs.openjdk.java.net/browse/JDK-8205051". Amith is fixing survivor space and old gen related issue, not a locality group(lgrp) related issue. We are addressing below two issues in "https://bugs.openjdk.java.net/browse/JDK-8205051" Issue 1: Whenever any locality group(lgrp) fills-in, the young collection occurs irrespective of other locality groups has enough memory. Issue 2: if an application has fewer threads than the number of locality groups (NUMA nodes). Locality groups, which are not having any home thread will be unused. Regards, Roshan Mangal On Mon, Dec 3, 2018 at 11:51 AM amith pawar wrote: > > +Roshan Mangal > > On Mon, Dec 3, 2018 at 11:49 AM amith pawar wrote: >> >> Hi Derek >> >> On Sat, Dec 1, 2018 at 4:10 AM White, Derek wrote: >>> >>> Hi Amith, >>> >>> >>> Thanks for tackling this! >>> >>> >>> I think that your patch may fix https://bugs.openjdk.java.net/browse/JDK-8205051, although it's probably fixing more than that. I was thinking that the problem was related to calling " _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr);", and you replaced that call with more specific node sets, which seems better to me. >>> >>> >>> You can try the test cases listed in the bug to see if the patch fixes the problem. >> >> Tested and didn't fix it. >>> >>> >>> The existing code disables UseNUMA if the process is bound to only one node, but it looks like your patch removes this. Is that a correct read of the patch, intentional, and/or a good idea? >> >> I removed that after testing it with SPECjbb for single node case. >> With patch : Enabling/Disabling UseNUMA gives same score. >> Without patch: UseNUMA scores 10-12% lesser than without UseNUMA >>> >>> >>> Thanks! >>> >>> - Derek >>> >>> >>> ________________________________ >>> From: hotspot-dev on behalf of amith pawar >>> Sent: Thursday, November 29, 2018 6:08 AM >>> To: thomas.schatzl at oracle.com >>> Cc: hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com >>> Subject: Re: [PATCH] JDK NUMA Interleaving issue >>> >>> External Email >>> >>> Hi Thomas, >>> >>> Sorry for the late reply and please check my inline comments below. >>> >>> On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl >>> wrote: >>> >>> > Hi Amith, >>> > >>> > welcome to the OpenJDK community! :) >>> > >>> Thanks >>> >>> > >>> > >>> On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: >>> > > Hi all, >>> > > >>> > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, >>> > > S1 and S2 region (if any other ) memory areas on requested Numa nodes >>> > > and it should not configure itself to access other Numa nodes. This >>> > > issue is observed only when Java is allowed to run on fewer NUMA >>> > > nodes than available on the system with numactl membind and >>> > > interleave options. Running on all the nodes does not have any >>> > > effect. This will cause some applications (objects residing in old >>> > > gen and survivor region) to run slower on system with large Numa >>> > > nodes. >>> > > >>> > [... long explanation...] >>> > >>> > Is it possible to summarize the problem and the changes with the >>> > following few lines: >>> > >>> > "NUMA interleaving of memory of old gen and survivor spaces (for >>> > parallel GC) tells the OS to interleave memory across all nodes of a >>> > NUMA system. However the VM process may be configured to be limited to >>> > run only on a few nodes, which means that large parts of the heap will >>> > be located on foreign nodes. This can incurs a large performance >>> > penalty. >>> > >>> > The proposed solution is to tell the OS to interleave memory only >>> > across available nodes when enabling NUMA." >>> > >>> OK. Since, I dont have write access to the defect link so can I request you >>> do the same. >>> >>> > >>> > We have had trouble understanding the problem statement and purpose of >>> > this patch when triaging (making sure the issue is understandable and >>> > can be worked on) as the text is rather technical. Having an easily >>> > understandable text also helps reviewers a lot. >>> > >>> >>> > Assuming my summary is appropriate, I have several other unrelated >>> > questions: >>> > >>> > - could you propose a better subject for this work? "JDK NUMA >>> > Interleaving issue" seems very generic. Something like "NUMA heap >>> > allocation does not respect VM membind/interleave settings" maybe? >>> > >>> This is also OK. >>> >>> > >>> > - there have been other NUMA related patches from AMD recently, in >>> > particular JDK-what is the relation to JDK-8205051? The text there >>> > reads awfully similar to this one, but I *think* JDK-8205051 is >>> > actually about making sure that the parallel gc eden is not put on >>> > inactive nodes. >>> > Can you confirm this (talk to your colleague) so that we can fix the >>> > description too? >>> > >>> JDK-8205051 is related to early GC. >>> JDK-8189922 specific to membind case where heap was allocated on non >>> requested NUMA nodes. >>> The proposed patch fixes two following issues >>> 1. Similar to JDK-8189922 but for interleave case. >>> 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer >>> NUMA nodes. >>> >>> >>> > - fyi, we are currently working on NUMA aware memory allocation support >>> > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to >>> > not step on each other's toes (see below). >>> > >>> We are not working anything related to G1. It may effect G1 if >>> numa_make_global function is called. >>> >>> > >>> > [... broken patch...] >>> > >>> > I tried to apply the patch to the jdk/jdk tree, which failed; I then >>> > started to manually apply it but stopped after not being able to find >>> > the context of some hunks. I do not think this change applies to the >>> > latest source tree. >>> > >>> > Please make sure that the patch applies to the latest jdk/jdk tree with >>> > errors. All changes generally must first go into the latest dev tree >>> > before you can apply for backporting. >>> > >>> > Could you please send the patch as attachment (not copy&pasted) to this >>> > list and cc me? Then I can create a webrev out of it. >>> > >>> > I did look a bit over the patch as much as I could (it's very hard >>> > trying to review a diff), some comments: >>> > >>> Sorry. Please check the attached patch. >>> >>> > >>> > - the _numa_interleave_memory_v2 function pointer is never assigned >>> > to in the patch in the CR, so it will not be used. Please make sure the >>> > patch is complete. >>> > Actually it is never defined anywhere, ie. the patch unlikely actually >>> > compiles even if I could apply it somewhere. >>> > >>> > Please avoid frustrating reviewers by sending incomplete patches. >>> > >>> Sorry again. Faced same issue when copied from the defect link but worked >>> for me from the mail. Now i have attached the patch. Can you please update >>> defect link with this patch ? >>> >>> > >>> > - I am not sure set_numa_interleave, i.e. the interleaving, should be >>> > done without UseNUMAInterleaving enabled. Some collectors may do their >>> > own interleaving in the future (JDK-8210473 / JEP-345) so this would >>> > massively interfere in how they work. (That issue may be because I am >>> > looking at a potentially incomplete diff, so forgive me if the patch >>> > already handles this). >>> > >>> Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect >>> when java externally invoked in interleave mode. It helps membind case. >>> >>> > >>> > - if some of the actions (interleaving/membind) fail, and it had been >>> > requested, it would be nice to print a log message. >>> > >>> Following two NUMA API's are used and they return nothing. Right now not >>> sure which cases to handle. Please suggest >>> void numa_tonode_memory(void *start, size_t size, int node); >>> void numa_interleave_memory(void *start, size_t size, struct bitmask >>> *nodemask); >>> >>> > >>> > Actually it would be nice to print information about e.g. the bitmask >>> > anyway in the log so that the user can easily verify that what he >>> > specified on the command line has been properly picked up by the VM - >>> > instead of looking through the proc filesystem. >>> > >>> As per the suggestion, patch is updated to print following information. >>> [0.001s][info][os] UseNUMA is enabled >>> [0.001s][info][os] Java is configured to run in interleave mode >>> [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, 2, >>> 3 >>> >>> > >>> > Thanks, >>> > Thomas >>> > >>> > >>> > >>> > >>> Thanks >>> Amit Pawar >> >> >> Thanks, >> Amit Pawar > > > > -- > With best regards, > amit pawar From vladimir.x.ivanov at oracle.com Tue Dec 4 01:06:59 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 3 Dec 2018 17:06:59 -0800 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot Message-ID: http://cr.openjdk.java.net/~vlivanov/8146090/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8146090 The test doesn't work with -XX:+DeoptimizeALot: the test expects execution to continue in generated code after System.gc() call, but JVM constantly deoptimizes it at that point in -XX:+DeoptimizeALot mode, so execution continues in interpreter and it breaks the invariant the test checks. The fix is to disable executing the test with -XX:+DeoptimizeALot. Testing: manually verified that the test isn't executed when -XX:+DeoptimizeALot is specified. Best regards, Vladimir Ivanov From david.holmes at oracle.com Tue Dec 4 03:38:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 13:38:00 +1000 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot In-Reply-To: References: Message-ID: <896662e4-fd36-a466-0e4d-2c69c59e8889@oracle.com> Hi Vladimir, On 4/12/2018 11:06 am, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8146090/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8146090 > > The test doesn't work with -XX:+DeoptimizeALot: the test expects > execution to continue in generated code after System.gc() call, but JVM > constantly deoptimizes it at that point in -XX:+DeoptimizeALot mode, so > execution continues in interpreter and it breaks the invariant the test > checks. > > The fix is to disable executing the test with -XX:+DeoptimizeALot. I just added a query to the bug report. I can't tell what other flags get tested in same run that uses DeoptimizeALot. Is it better to exclude the test from all such runs, or to force DeoptimizeALot off in the @run lines? PS. Please update the copyright line to add 2018. Thanks, David > Testing: manually verified that the test isn't executed when > -XX:+DeoptimizeALot is specified. > > Best regards, > Vladimir Ivanov From igor.ignatyev at oracle.com Tue Dec 4 03:40:27 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 3 Dec 2018 19:40:27 -0800 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot In-Reply-To: References: Message-ID: <4305BACA-9303-4784-9B1D-E696C863E521@oracle.com> Hi Vladimir, the fix looks good to me. Thanks, -- Igor > On Dec 3, 2018, at 5:06 PM, Vladimir Ivanov wrote: > > http://cr.openjdk.java.net/~vlivanov/8146090/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8146090 > > The test doesn't work with -XX:+DeoptimizeALot: the test expects execution to continue in generated code after System.gc() call, but JVM constantly deoptimizes it at that point in -XX:+DeoptimizeALot mode, so execution continues in interpreter and it breaks the invariant the test checks. > > The fix is to disable executing the test with -XX:+DeoptimizeALot. > > Testing: manually verified that the test isn't executed when -XX:+DeoptimizeALot is specified. > > Best regards, > Vladimir Ivanov From chris.plummer at oracle.com Tue Dec 4 04:15:05 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 3 Dec 2018 20:15:05 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <895ef766-9c96-7185-4222-178379629ce4@oracle.com> References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> Message-ID: <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> Hi JC, Overall it looks good. A few naming nits thought: In bi01t001.cpp, why have you declared the ExceptionCheckingJniEnvPtr using jni_env(jni). Elsewhere you use jni(jni_env) and rename the method argument passed in from jni to jni_env. Related to this, I also noticed in some files that already are using ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, you delcared it as env(jni_env). So that means there are 3 different names you have used for the ExceptionCheckingJniEnvPtr local variable. They should be consistent. Also, can you rename get_basename() to get_dirname()? I know Serguei suggested get_basename() a while back, but unless "basename" is commonly used for this purpose, I think "dirname" is more self explanatory. thanks, Chris On 12/2/18 10:29 PM, David Holmes wrote: > Hi Jc, > > I've been lurking on this one and have had a look through. I'm okay > with the FatalError approach for the tests - we don't expect anything > to go wrong in a well written test in a correctly functioning VM. > > Thanks, > David > > > > On 3/12/2018 3:24 pm, JC Beyler wrote: >> Hi all, >> >> Would someone on the GC or runtime team be motivated to give this a >> review? :) >> >> It would be much appreciated! >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> >> Thanks for your help, >> Jc >> >> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > > wrote: >> >> ??? Hi Chris, >> >> ??? Yes I was waiting for another review since you had explicitly >> asked :) >> >> ??? And sounds good that when someone from GC or runtime gives a review, >> ??? I'll wait for your full review on the webrev.02! >> >> ??? Thanks again for?your help, >> ??? Jc >> >> >> ??? On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >> ??? > wrote: >> >> ??????? Hi JC, >> >> ??????? I think it would be good to get a review from the gc or runtime >> ??????? teams, since this also affects their tests. >> >> ??????? Also, once we are settled on this FatalError approach, I still >> ??????? need to give your webrev-02 a full review. I only skimmed over >> ??????? parts of it (I did look at all the changes in webrevo-01). >> >> ??????? thanks, >> >> ??????? Chris >> >> ??????? On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >> ??????? wrote: >>> ??????? Hi Jc, >>> >>> ??????? I've already reviewed this too. >>> >>> ??????? Thanks, >>> ??????? Serguei >>> >>> >>> ??????? On 11/27/18 06:56, JC Beyler wrote: >>>> ??????? Thanks Chris, >>>> >>>> ??????? Anybody else motivated to look at this and review it? :) >>>> ??????? Jc >>>> >>>> ??????? On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >>>> ??????? > >>>> ??????? wrote: >>>> >>>> ??????????? Hi JC, >>>> >>>> ??????????? I'm ok with the FatalError approach, but would like to >>>> ??????????? hear opinions from others also. >>>> >>>> ??????????? thanks, >>>> >>>> ??????????? Chris >>>> >>>> ??????????? On 11/21/18 8:19 AM, JC Beyler wrote: >>>>> ??????????? Hi Chris, >>>>> >>>>> ??????????? Thanks for taking the time to look at it and yes you >>>>> ??????????? have raised exactly why the webrev is between two >>>>> ??????????? worlds: in cases where a fatal error on failure is >>>>> ??????????? wanted, should we simplify the code to remove the return >>>>> ??????????? tests since we do them internally? Now that I've looked >>>>> ??????????? around for non-fatal cases, I think the answer is yes, >>>>> ??????????? it simplifies the code while maintaining the checks. >>>>> >>>>> ??????????? I looked a bit and it seems that I can't find easily a >>>>> ??????????? case where the test accepts a JNI failure to then move >>>>> ??????????? on. Therefore, perhaps, for now, the fail with a Fatal >>>>> ??????????? is enough and we can work on the tests to clean them up? >>>>> >>>>> ??????????? That means that this is the new webrev with only Fatal >>>>> ??????????? and cleans up the tests so that it is no longer in >>>>> ??????????? between two worlds: >>>>> >>>>> ??????????? Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>> >>>>> ??????????? Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> >>>>> ??????????? (This passes testing on my dev machine for all the >>>>> ??????????? modified tests) >>>>> >>>>> ??????????? with the example you provided, it now looks like: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> >>>>> >>>>> ??????????? Where it does, to me at least, seem cleaner and less >>>>> ??????????? "noisy". >>>>> >>>>> ??????????? Let me know what you think, >>>>> ??????????? Jc >>>>> >>>>> >>>>> ??????????? On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >>>>> ??????????? >>>> ??????????? > wrote: >>>>> >>>>> ??????????????? Hi JC, >>>>> >>>>> ??????????????? Sorry about the delay. I had to go back an look at >>>>> ??????????????? the initial 8210842 webrev and RFR thread to see >>>>> ??????????????? what this was initially all about. >>>>> >>>>> ??????????????? In general the changes look good. >>>>> >>>>> ??????????????? I don't have a good answer to your >>>>> ??????????????? FatalError/NonFatalError question. It makes the code >>>>> ??????????????? a lot cleaner to use FatalError, but then it is a >>>>> ??????????????? behavior change, and you also need to deal with >>>>> ??????????????? tests that intentionally induce errors (do you have >>>>> ??????????????? an example of that). >>>>> >>>>> ??????????????? In any case, right now your webrev seems to be >>>>> ??????????????? between two worlds. You are producing FatalError, >>>>> ??????????????? but still checking results. Here's a good example: >>>>> >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> >>>>> >>>>> ??????????????? I'm not sure if this is just a temporary state until >>>>> ??????????????? it was decided which approach to take. >>>>> >>>>> ??????????????? thanks, >>>>> >>>>> ??????????????? Chris >>>>> >>>>> >>>>> ??????????????? On 11/20/18 2:14 PM, JC Beyler wrote: >>>>>> ??????????????? Hi all, >>>>>> >>>>>> ??????????????? Chris thought it made sense to have more eyes on >>>>>> ??????????????? this change than just serviceability as it will >>>>>> ??????????????? modify to tests that are not only serviceability >>>>>> ??????????????? tests so I've moved this to conversation here :) >>>>>> >>>>>> ??????????????? For convenience, I've copy-pasted the initial RFR: >>>>>> >>>>>> ??????????????? Could I have a review for the extension and usage >>>>>> ??????????????? of the ExceptionJniWrapper. This adds lines and >>>>>> ??????????????? filenames to the end of the wrapper JNI methods, >>>>>> ??????????????? adds tracing, and throws an error if need be. I've >>>>>> ??????????????? ported the gc/lock files to use the new >>>>>> ??????????????? TRACE_JNI_CALL add-on and I've ported a few of the >>>>>> ??????????????? tests that were already changed for the assignment >>>>>> ??????????????? webrev for JDK-8212884. >>>>>> >>>>>> ??????????????? Webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>>>> >>>>>> ??????????????? Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> >>>>>> ??????????????? For illustration, if I force an error to the >>>>>> ??????????????? AP04/ap04t03 test and set the verbosity on, I get >>>>>> ??????????????? something like: >>>>>> >>>>>> ??????????????? >> Calling JNI method FindClass from >>>>>> ap04t003.cpp:343 >>>>>> ??????????????? >> Calling with these parameter(s): >>>>>> ??????????????? ? ? ? ? java/lang/Threadd >>>>>> ??????????????? Wait for thread to finish >>>>>> ??????????????? << Called JNI method FindClass from ap04t003.cpp:343 >>>>>> ??????????????? Exception in thread "Thread-0" >>>>>> ??????????????? java.lang.NoClassDefFoundError: java/lang/Threadd >>>>>> ??????????????? ? ? ? ? at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> >>>>>> ??????????????? Method) >>>>>> ??????????????? ? ? ? ? at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> ??????????????? ? ? ? ? at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> ??????????????? Caused by: java.lang.ClassNotFoundException: >>>>>> ??????????????? java.lang.Threadd >>>>>> ??????????????? ? ? ? ? at >>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>> ??????????????? ? ? ? ? at >>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>> ??????????????? ? ? ? ? at >>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>> ??????????????? ? ? ? ? ... 3 more >>>>>> ??????????????? FATAL ERROR in native method: JNI method FindClass >>>>>> ??????????????? : internal error from ap04t003.cpp:343 >>>>>> ??????????????? ? ? ? ? at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> >>>>>> ??????????????? Method) >>>>>> ??????????????? ? ? ? ? at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> ??????????????? ? ? ? ? at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> >>>>>> ??????????????? Questions/comments I have about this are: >>>>>> ??????????????? ? - Do we want to force fatal errors when a JNI >>>>>> ??????????????? call fails in general? Most of these tests do the >>>>>> ??????????????? right thing and test the return of the JNI calls, >>>>>> ??????????????? for example: >>>>>> ??????????????? ? ? thrClass = jni->FindClass("java/lang/Threadd", >>>>>> ??????????????? TRACE_JNI_CALL); >>>>>> ??????????????? ? ? if (thrClass == NULL) { >>>>>> >>>>>> ??????????????? but now the wrapper actually would do a fatal if >>>>>> ??????????????? the FindClass call would return a nullptr, so we >>>>>> ??????????????? could remove that test altogether. What do you >>>>>> think? >>>>>> ??????????????? ? ? - I prefer to leave them as the tests then >>>>>> ??????????????? become closer to what real users would have in >>>>>> ??????????????? their code and is the "recommended" way of doing it >>>>>> >>>>>> ??????????????? ? ?- The alternative is to use the NonFatalError I >>>>>> ??????????????? added which then just prints out that something >>>>>> ??????????????? went wrong, letting the test continue. Question >>>>>> ??????????????? will be what should be the default? The fatal or >>>>>> ??????????????? the non-fatal error handling? >>>>>> >>>>>> ??????????????? On a different subject: >>>>>> ??????????????? ? - On the new tests, I've removed the >>>>>> ??????????????? NSK_JNI_VERIFY since the JNI wrapper handles the >>>>>> ??????????????? tracing and the verify in almost the same way; only >>>>>> ??????????????? difference I can really tell is that the complain >>>>>> ??????????????? method from NSK has a max complain before stopping >>>>>> ??????????????? to "complain"; I have not added that part of the >>>>>> ??????????????? code in this webrev >>>>>> >>>>>> ??????????????? Once we decide on these, I can continue on the >>>>>> ??????????????? files from JDK-8212884 and then do both the >>>>>> ??????????????? assignment in an if extraction followed-by this >>>>>> ??????????????? type of webrev in an easier fashion. Depending on >>>>>> ??????????????? decisions here, NSK*VERIFY can be deprecated as >>>>>> ??????????????? well as we go forward. >>>>>> >>>>>> ??????????????? Thanks! >>>>>> ??????????????? Jc >>>>>> >>>>>> ??????????????? On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer >>>>>> ??????????????? >>>>> > wrote: >>>>>> >>>>>> ??????????????????? On 11/19/18 10:07 AM, JC Beyler wrote: >>>>>>> ??????????????????? Hi all, >>>>>>> >>>>>>> ??????????????????? @David/Chris: should I then push this RFR to >>>>>>> ??????????????????? the hotspot mailing or the runtime one? For >>>>>>> ??????????????????? what it's worth, a lot of the tests under the >>>>>>> ??????????????????? vmTestbase are jvmti so the review also >>>>>>> ??????????????????? affects serviceability; it just turns out I >>>>>>> ??????????????????? started with the GC originally and then hit >>>>>>> ??????????????????? some other tests I had touched via the >>>>>>> ??????????????????? assignment extraction. >>>>>> ??????????????????? I think hotspot would be best. >>>>>> >>>>>> ??????????????????? Chris >>>>>>> >>>>>>> ??????????????????? @Serguei: Done for the method renaming, for >>>>>>> ??????????????????? the indent, are you talking about going from >>>>>>> ??????????????????? the 8-indent to 4-indent? If so, would it not >>>>>>> ??????????????????? just be better to do a new JBS bug and do the >>>>>>> ??????????????????? whole files in one go? I ask because >>>>>>> ??????????????????? otherwise, it will look a bit weird to have >>>>>>> ??????????????????? parts of the file as 8-indent and others >>>>>>> 4-indent? >>>>>>> >>>>>>> ??????????????????? Thanks for looking at it! >>>>>>> ??????????????????? Jc >>>>>>> >>>>>>> ??????????????????? On Mon, Nov 19, 2018 at 1:25 AM >>>>>>> ??????????????????? serguei.spitsyn at oracle.com >>>>>>> >>>>>>> ??????????????????? >>>>>> > wrote: >>>>>>> >>>>>>> ??????????????????????? Hi Jc, >>>>>>> >>>>>>> ??????????????????????? We have to start this review anyway. :) >>>>>>> ??????????????????????? It looks good to me in general. >>>>>>> ??????????????????????? Thank you for your consistency in this >>>>>>> ??????????????????????? refactoring! >>>>>>> >>>>>>> ??????????????????????? Some minor comments. >>>>>>> >>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>>>> ??????????????????????? +static const char* remove_folders(const >>>>>>> ??????????????????????? char* fullname) { I'd suggest to rename >>>>>>> ??????????????????????? the function name to something traditional >>>>>>> ??????????????????????? like get_basename. Otherwise, it sounds >>>>>>> ??????????????????????? like this function has to really remove >>>>>>> ??????????????????????? folders. :) Also, all *Locker.cpp have >>>>>>> ??????????????????????? wrong indent in the bodies of if and while >>>>>>> ??????????????????????? statements. Could this be fixed with the >>>>>>> ??????????????????????? refactoring? I did not look on how this >>>>>>> ??????????????????????? impacts the tests other than >>>>>>> ??????????????????????? serviceability. Thanks, Serguei >>>>>>> >>>>>>> >>>>>>> ??????????????????????? On 11/16/18 19:43, JC Beyler wrote: >>>>>>>> ??????????????????????? Hi all, >>>>>>>> >>>>>>>> ??????????????????????? Anybody motivated to review this? :) >>>>>>>> ??????????????????????? Jc >>>>>>>> >>>>>>>> ??????????????????????? On Wed, Nov 7, 2018 at 9:53 PM JC Beyler >>>>>>>> ??????????????????????? >>>>>>> > wrote: >>>>>>>> >>>>>>>> ??????????????????????????? Hi all, >>>>>>>> >>>>>>>> ??????????????????????????? Could I have a review for the >>>>>>>> ??????????????????????????? extension and usage of the >>>>>>>> ??????????????????????????? ExceptionJniWrapper. This adds lines >>>>>>>> ??????????????????????????? and filenames to the end of the >>>>>>>> ??????????????????????????? wrapper JNI methods, adds tracing, >>>>>>>> ??????????????????????????? and throws an error if need be. I've >>>>>>>> ??????????????????????????? ported the gc/lock files to use the >>>>>>>> ??????????????????????????? new TRACE_JNI_CALL add-on and I've >>>>>>>> ??????????????????????????? ported a few of the tests that were >>>>>>>> ??????????????????????????? already changed for the assignment >>>>>>>> ??????????????????????????? webrev for JDK-8212884. >>>>>>>> >>>>>>>> ??????????????????????????? Webrev: >>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>>>>> >>>>>>>> ??????????????????????????? Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>>>> >>>>>>>> ??????????????????????????? For illustration, if I force an error >>>>>>>> ??????????????????????????? to the AP04/ap04t03 test and set the >>>>>>>> ??????????????????????????? verbosity on, I get something like: >>>>>>>> >>>>>>>> ??????????????????????????? >> Calling JNI method FindClass from >>>>>>>> ??????????????????????????? ap04t003.cpp:343 >>>>>>>> ??????????????????????????? >> Calling with these parameter(s): >>>>>>>> ??????????????????????????? java/lang/Threadd >>>>>>>> ??????????????????????????? Wait for thread to finish >>>>>>>> ??????????????????????????? << Called JNI method FindClass from >>>>>>>> ??????????????????????????? ap04t003.cpp:343 >>>>>>>> ??????????????????????????? Exception in thread "Thread-0" >>>>>>>> java.lang.NoClassDefFoundError: >>>>>>>> ??????????????????????????? java/lang/Threadd >>>>>>>> ??????????????????????????? at >>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>>> >>>>>>>> ??????????????????????????? Method) >>>>>>>> ??????????????????????????? at >>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>>> ??????????????????????????? at >>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>>> ??????????????????????????? Caused by: >>>>>>>> java.lang.ClassNotFoundException: >>>>>>>> ??????????????????????????? java.lang.Threadd >>>>>>>> ??????????????????????????? at >>>>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>>>> ??????????????????????????? at >>>>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>>>> ??????????????????????????? at >>>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>>>> ??????????????????????????? ... 3 more >>>>>>>> ??????????????????????????? FATAL ERROR in native method: JNI >>>>>>>> ??????????????????????????? method FindClass : internal error >>>>>>>> ??????????????????????????? from ap04t003.cpp:343 >>>>>>>> ??????????????????????????? at >>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>>> >>>>>>>> ??????????????????????????? Method) >>>>>>>> ??????????????????????????? at >>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>>> ??????????????????????????? at >>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>>> >>>>>>>> ??????????????????????????? Questions/comments I have about >>>>>>>> this are: >>>>>>>> ??????????????????????????? ? - Do we want to force fatal errors >>>>>>>> ??????????????????????????? when a JNI call fails in general? >>>>>>>> ??????????????????????????? Most of these tests do the right >>>>>>>> ??????????????????????????? thing and test the return of the JNI >>>>>>>> ??????????????????????????? calls, for example: >>>>>>>> ??????????????????????????? thrClass = >>>>>>>> jni->FindClass("java/lang/Threadd", >>>>>>>> ??????????????????????????? TRACE_JNI_CALL); >>>>>>>> ??????????????????????????? ? ? if (thrClass == NULL) { >>>>>>>> >>>>>>>> ??????????????????????????? but now the wrapper actually would do >>>>>>>> ??????????????????????????? a fatal if the FindClass call would >>>>>>>> ??????????????????????????? return a nullptr, so we could remove >>>>>>>> ??????????????????????????? that test altogether. What do you >>>>>>>> think? >>>>>>>> ??????????????????????????? ? ? - I prefer to leave them as the >>>>>>>> ??????????????????????????? tests then become closer to what real >>>>>>>> ??????????????????????????? users would have in their code and is >>>>>>>> ??????????????????????????? the "recommended" way of doing it >>>>>>>> >>>>>>>> ??????????????????????????? ? ?- The alternative is to use the >>>>>>>> ??????????????????????????? NonFatalError I added which then just >>>>>>>> ??????????????????????????? prints out that something went wrong, >>>>>>>> ??????????????????????????? letting the test continue. Question >>>>>>>> ??????????????????????????? will be what should be the default? >>>>>>>> ??????????????????????????? The fatal or the non-fatal error >>>>>>>> ??????????????????????????? handling? >>>>>>>> >>>>>>>> ??????????????????????????? On a different subject: >>>>>>>> ??????????????????????????? ? - On the new tests, I've removed >>>>>>>> ??????????????????????????? the NSK_JNI_VERIFY since the JNI >>>>>>>> ??????????????????????????? wrapper handles the tracing and the >>>>>>>> ??????????????????????????? verify in almost the same way; only >>>>>>>> ??????????????????????????? difference I can really tell is that >>>>>>>> ??????????????????????????? the complain method from NSK has a >>>>>>>> ??????????????????????????? max complain before stopping to >>>>>>>> ??????????????????????????? "complain"; I have not added that >>>>>>>> ??????????????????????????? part of the code in this webrev >>>>>>>> >>>>>>>> ??????????????????????????? Once we decide on these, I can >>>>>>>> ??????????????????????????? continue on the files from >>>>>>>> ??????????????????????????? JDK-8212884 and then do both the >>>>>>>> ??????????????????????????? assignment in an if extraction >>>>>>>> ??????????????????????????? followed-by this type of webrev in an >>>>>>>> ??????????????????????????? easier fashion. Depending on >>>>>>>> ??????????????????????????? decisions here, NSK*VERIFY can be >>>>>>>> ??????????????????????????? deprecated as well as we go forward. >>>>>>>> >>>>>>>> ??????????????????????????? Thank you for the reviews/comments :) >>>>>>>> ??????????????????????????? Jc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ??????????????????????? -- >>>>>>>> ??????????????????????? Thanks, >>>>>>>> ??????????????????????? Jc >>>>>>> >>>>>>> >>>>>>> >>>>>>> ??????????????????? -- >>>>>>> ??????????????????? Thanks, >>>>>>> ??????????????????? Jc >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ??????????????? -- >>>>>> ??????????????? Thanks, >>>>>> ??????????????? Jc >>>>> >>>>> >>>>> >>>>> >>>>> ??????????? -- >>>>> ??????????? Thanks, >>>>> ??????????? Jc >>>> >>>> >>>> >>>> >>>> ??????? -- >>>> ??????? Thanks, >>>> ??????? Jc >>> >> >> >> >> ??? -- >> ??? Thanks, >> ??? Jc >> >> >> >> -- >> >> Thanks, >> Jc From serguei.spitsyn at oracle.com Tue Dec 4 04:40:49 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Dec 2018 20:40:49 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> Message-ID: <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> On 12/3/18 20:15, Chris Plummer wrote: > Hi JC, > > Overall it looks good. A few naming nits thought: > > In bi01t001.cpp, why have you declared the ExceptionCheckingJniEnvPtr > using jni_env(jni). Elsewhere you use jni(jni_env) and rename the > method argument passed in from jni to jni_env. > > Related to this, I also noticed in some files that already are using > ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, you > delcared it as env(jni_env). So that means there are 3 different names > you have used for the ExceptionCheckingJniEnvPtr local variable. They > should be consistent. > > Also, can you rename get_basename() to get_dirname()? I know Serguei > suggested get_basename() a while back, but unless "basename" is > commonly used for this purpose, I think "dirname" is more self > explanatory. In general, I'm Okay with get_dirname(). Just to mention dirname can be both short or full, so it is a little confusing as well. It is the reason why the get_basename() was suggested. However, I do not insist on get_basename() nor get_full_dirname(). :) Thanks, Serguei > thanks, > > Chris > > On 12/2/18 10:29 PM, David Holmes wrote: >> Hi Jc, >> >> I've been lurking on this one and have had a look through. I'm okay >> with the FatalError approach for the tests - we don't expect anything >> to go wrong in a well written test in a correctly functioning VM. >> >> Thanks, >> David >> >> >> >> On 3/12/2018 3:24 pm, JC Beyler wrote: >>> Hi all, >>> >>> Would someone on the GC or runtime team be motivated to give this a >>> review? :) >>> >>> It would be much appreciated! >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> >>> Thanks for your help, >>> Jc >>> >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler >> > wrote: >>> >>> ??? Hi Chris, >>> >>> ??? Yes I was waiting for another review since you had explicitly >>> asked :) >>> >>> ??? And sounds good that when someone from GC or runtime gives a >>> review, >>> ??? I'll wait for your full review on the webrev.02! >>> >>> ??? Thanks again for?your help, >>> ??? Jc >>> >>> >>> ??? On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >>> ??? > wrote: >>> >>> ??????? Hi JC, >>> >>> ??????? I think it would be good to get a review from the gc or runtime >>> ??????? teams, since this also affects their tests. >>> >>> ??????? Also, once we are settled on this FatalError approach, I still >>> ??????? need to give your webrev-02 a full review. I only skimmed over >>> ??????? parts of it (I did look at all the changes in webrevo-01). >>> >>> ??????? thanks, >>> >>> ??????? Chris >>> >>> ??????? On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >>> ??????? wrote: >>>> ??????? Hi Jc, >>>> >>>> ??????? I've already reviewed this too. >>>> >>>> ??????? Thanks, >>>> ??????? Serguei >>>> >>>> >>>> ??????? On 11/27/18 06:56, JC Beyler wrote: >>>>> ??????? Thanks Chris, >>>>> >>>>> ??????? Anybody else motivated to look at this and review it? :) >>>>> ??????? Jc >>>>> >>>>> ??????? On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >>>>> ??????? > >>>>> ??????? wrote: >>>>> >>>>> ??????????? Hi JC, >>>>> >>>>> ??????????? I'm ok with the FatalError approach, but would like to >>>>> ??????????? hear opinions from others also. >>>>> >>>>> ??????????? thanks, >>>>> >>>>> ??????????? Chris >>>>> >>>>> ??????????? On 11/21/18 8:19 AM, JC Beyler wrote: >>>>>> ??????????? Hi Chris, >>>>>> >>>>>> ??????????? Thanks for taking the time to look at it and yes you >>>>>> ??????????? have raised exactly why the webrev is between two >>>>>> ??????????? worlds: in cases where a fatal error on failure is >>>>>> ??????????? wanted, should we simplify the code to remove the return >>>>>> ??????????? tests since we do them internally? Now that I've looked >>>>>> ??????????? around for non-fatal cases, I think the answer is yes, >>>>>> ??????????? it simplifies the code while maintaining the checks. >>>>>> >>>>>> ??????????? I looked a bit and it seems that I can't find easily a >>>>>> ??????????? case where the test accepts a JNI failure to then move >>>>>> ??????????? on. Therefore, perhaps, for now, the fail with a Fatal >>>>>> ??????????? is enough and we can work on the tests to clean them up? >>>>>> >>>>>> ??????????? That means that this is the new webrev with only Fatal >>>>>> ??????????? and cleans up the tests so that it is no longer in >>>>>> ??????????? between two worlds: >>>>>> >>>>>> ??????????? Webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>>> >>>>>> ??????????? Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> >>>>>> ??????????? (This passes testing on my dev machine for all the >>>>>> ??????????? modified tests) >>>>>> >>>>>> ??????????? with the example you provided, it now looks like: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ??????????? Where it does, to me at least, seem cleaner and less >>>>>> ??????????? "noisy". >>>>>> >>>>>> ??????????? Let me know what you think, >>>>>> ??????????? Jc >>>>>> >>>>>> >>>>>> ??????????? On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >>>>>> ??????????? >>>>> ??????????? > wrote: >>>>>> >>>>>> ??????????????? Hi JC, >>>>>> >>>>>> ??????????????? Sorry about the delay. I had to go back an look at >>>>>> ??????????????? the initial 8210842 webrev and RFR thread to see >>>>>> ??????????????? what this was initially all about. >>>>>> >>>>>> ??????????????? In general the changes look good. >>>>>> >>>>>> ??????????????? I don't have a good answer to your >>>>>> ??????????????? FatalError/NonFatalError question. It makes the code >>>>>> ??????????????? a lot cleaner to use FatalError, but then it is a >>>>>> ??????????????? behavior change, and you also need to deal with >>>>>> ??????????????? tests that intentionally induce errors (do you have >>>>>> ??????????????? an example of that). >>>>>> >>>>>> ??????????????? In any case, right now your webrev seems to be >>>>>> ??????????????? between two worlds. You are producing FatalError, >>>>>> ??????????????? but still checking results. Here's a good example: >>>>>> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ??????????????? I'm not sure if this is just a temporary state until >>>>>> ??????????????? it was decided which approach to take. >>>>>> >>>>>> ??????????????? thanks, >>>>>> >>>>>> ??????????????? Chris >>>>>> >>>>>> >>>>>> ??????????????? On 11/20/18 2:14 PM, JC Beyler wrote: >>>>>>> ??????????????? Hi all, >>>>>>> >>>>>>> ??????????????? Chris thought it made sense to have more eyes on >>>>>>> ??????????????? this change than just serviceability as it will >>>>>>> ??????????????? modify to tests that are not only serviceability >>>>>>> ??????????????? tests so I've moved this to conversation here :) >>>>>>> >>>>>>> ??????????????? For convenience, I've copy-pasted the initial RFR: >>>>>>> >>>>>>> ??????????????? Could I have a review for the extension and usage >>>>>>> ??????????????? of the ExceptionJniWrapper. This adds lines and >>>>>>> ??????????????? filenames to the end of the wrapper JNI methods, >>>>>>> ??????????????? adds tracing, and throws an error if need be. I've >>>>>>> ??????????????? ported the gc/lock files to use the new >>>>>>> ??????????????? TRACE_JNI_CALL add-on and I've ported a few of the >>>>>>> ??????????????? tests that were already changed for the assignment >>>>>>> ??????????????? webrev for JDK-8212884. >>>>>>> >>>>>>> ??????????????? Webrev: >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>>>>> >>>>>>> ??????????????? Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>>> >>>>>>> ??????????????? For illustration, if I force an error to the >>>>>>> ??????????????? AP04/ap04t03 test and set the verbosity on, I get >>>>>>> ??????????????? something like: >>>>>>> >>>>>>> ??????????????? >> Calling JNI method FindClass from >>>>>>> ap04t003.cpp:343 >>>>>>> ??????????????? >> Calling with these parameter(s): >>>>>>> ??????????????? ? ? ? ? java/lang/Threadd >>>>>>> ??????????????? Wait for thread to finish >>>>>>> ??????????????? << Called JNI method FindClass from >>>>>>> ap04t003.cpp:343 >>>>>>> ??????????????? Exception in thread "Thread-0" >>>>>>> ??????????????? java.lang.NoClassDefFoundError: java/lang/Threadd >>>>>>> ??????????????? ? ? ? ? at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>> >>>>>>> ??????????????? Method) >>>>>>> ??????????????? ? ? ? ? at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>> >>>>>>> ??????????????? ? ? ? ? at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>> >>>>>>> ??????????????? Caused by: java.lang.ClassNotFoundException: >>>>>>> ??????????????? java.lang.Threadd >>>>>>> ??????????????? ? ? ? ? at >>>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>>> >>>>>>> ??????????????? ? ? ? ? at >>>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>>> >>>>>>> ??????????????? ? ? ? ? at >>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>>> ??????????????? ? ? ? ? ... 3 more >>>>>>> ??????????????? FATAL ERROR in native method: JNI method FindClass >>>>>>> ??????????????? : internal error from ap04t003.cpp:343 >>>>>>> ??????????????? ? ? ? ? at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>> >>>>>>> ??????????????? Method) >>>>>>> ??????????????? ? ? ? ? at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>> >>>>>>> ??????????????? ? ? ? ? at >>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>> >>>>>>> >>>>>>> ??????????????? Questions/comments I have about this are: >>>>>>> ??????????????? ? - Do we want to force fatal errors when a JNI >>>>>>> ??????????????? call fails in general? Most of these tests do the >>>>>>> ??????????????? right thing and test the return of the JNI calls, >>>>>>> ??????????????? for example: >>>>>>> ??????????????? ? ? thrClass = jni->FindClass("java/lang/Threadd", >>>>>>> ??????????????? TRACE_JNI_CALL); >>>>>>> ??????????????? ? ? if (thrClass == NULL) { >>>>>>> >>>>>>> ??????????????? but now the wrapper actually would do a fatal if >>>>>>> ??????????????? the FindClass call would return a nullptr, so we >>>>>>> ??????????????? could remove that test altogether. What do you >>>>>>> think? >>>>>>> ??????????????? ? ? - I prefer to leave them as the tests then >>>>>>> ??????????????? become closer to what real users would have in >>>>>>> ??????????????? their code and is the "recommended" way of doing it >>>>>>> >>>>>>> ??????????????? ? ?- The alternative is to use the NonFatalError I >>>>>>> ??????????????? added which then just prints out that something >>>>>>> ??????????????? went wrong, letting the test continue. Question >>>>>>> ??????????????? will be what should be the default? The fatal or >>>>>>> ??????????????? the non-fatal error handling? >>>>>>> >>>>>>> ??????????????? On a different subject: >>>>>>> ??????????????? ? - On the new tests, I've removed the >>>>>>> ??????????????? NSK_JNI_VERIFY since the JNI wrapper handles the >>>>>>> ??????????????? tracing and the verify in almost the same way; only >>>>>>> ??????????????? difference I can really tell is that the complain >>>>>>> ??????????????? method from NSK has a max complain before stopping >>>>>>> ??????????????? to "complain"; I have not added that part of the >>>>>>> ??????????????? code in this webrev >>>>>>> >>>>>>> ??????????????? Once we decide on these, I can continue on the >>>>>>> ??????????????? files from JDK-8212884 and then do both the >>>>>>> ??????????????? assignment in an if extraction followed-by this >>>>>>> ??????????????? type of webrev in an easier fashion. Depending on >>>>>>> ??????????????? decisions here, NSK*VERIFY can be deprecated as >>>>>>> ??????????????? well as we go forward. >>>>>>> >>>>>>> ??????????????? Thanks! >>>>>>> ??????????????? Jc >>>>>>> >>>>>>> ??????????????? On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer >>>>>>> ??????????????? >>>>>> > wrote: >>>>>>> >>>>>>> ??????????????????? On 11/19/18 10:07 AM, JC Beyler wrote: >>>>>>>> ??????????????????? Hi all, >>>>>>>> >>>>>>>> ??????????????????? @David/Chris: should I then push this RFR to >>>>>>>> ??????????????????? the hotspot mailing or the runtime one? For >>>>>>>> ??????????????????? what it's worth, a lot of the tests under the >>>>>>>> ??????????????????? vmTestbase are jvmti so the review also >>>>>>>> ??????????????????? affects serviceability; it just turns out I >>>>>>>> ??????????????????? started with the GC originally and then hit >>>>>>>> ??????????????????? some other tests I had touched via the >>>>>>>> ??????????????????? assignment extraction. >>>>>>> ??????????????????? I think hotspot would be best. >>>>>>> >>>>>>> ??????????????????? Chris >>>>>>>> >>>>>>>> ??????????????????? @Serguei: Done for the method renaming, for >>>>>>>> ??????????????????? the indent, are you talking about going from >>>>>>>> ??????????????????? the 8-indent to 4-indent? If so, would it not >>>>>>>> ??????????????????? just be better to do a new JBS bug and do the >>>>>>>> ??????????????????? whole files in one go? I ask because >>>>>>>> ??????????????????? otherwise, it will look a bit weird to have >>>>>>>> ??????????????????? parts of the file as 8-indent and others >>>>>>>> 4-indent? >>>>>>>> >>>>>>>> ??????????????????? Thanks for looking at it! >>>>>>>> ??????????????????? Jc >>>>>>>> >>>>>>>> ??????????????????? On Mon, Nov 19, 2018 at 1:25 AM >>>>>>>> ??????????????????? serguei.spitsyn at oracle.com >>>>>>>> >>>>>>>> ??????????????????? >>>>>>> > wrote: >>>>>>>> >>>>>>>> ??????????????????????? Hi Jc, >>>>>>>> >>>>>>>> ??????????????????????? We have to start this review anyway. :) >>>>>>>> ??????????????????????? It looks good to me in general. >>>>>>>> ??????????????????????? Thank you for your consistency in this >>>>>>>> ??????????????????????? refactoring! >>>>>>>> >>>>>>>> ??????????????????????? Some minor comments. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>>>>> >>>>>>>> ??????????????????????? +static const char* remove_folders(const >>>>>>>> ??????????????????????? char* fullname) { I'd suggest to rename >>>>>>>> ??????????????????????? the function name to something traditional >>>>>>>> ??????????????????????? like get_basename. Otherwise, it sounds >>>>>>>> ??????????????????????? like this function has to really remove >>>>>>>> ??????????????????????? folders. :) Also, all *Locker.cpp have >>>>>>>> ??????????????????????? wrong indent in the bodies of if and while >>>>>>>> ??????????????????????? statements. Could this be fixed with the >>>>>>>> ??????????????????????? refactoring? I did not look on how this >>>>>>>> ??????????????????????? impacts the tests other than >>>>>>>> ??????????????????????? serviceability. Thanks, Serguei >>>>>>>> >>>>>>>> >>>>>>>> ??????????????????????? On 11/16/18 19:43, JC Beyler wrote: >>>>>>>>> ??????????????????????? Hi all, >>>>>>>>> >>>>>>>>> ??????????????????????? Anybody motivated to review this? :) >>>>>>>>> ??????????????????????? Jc >>>>>>>>> >>>>>>>>> ??????????????????????? On Wed, Nov 7, 2018 at 9:53 PM JC Beyler >>>>>>>>> ??????????????????????? >>>>>>>> > wrote: >>>>>>>>> >>>>>>>>> ??????????????????????????? Hi all, >>>>>>>>> >>>>>>>>> ??????????????????????????? Could I have a review for the >>>>>>>>> ??????????????????????????? extension and usage of the >>>>>>>>> ??????????????????????????? ExceptionJniWrapper. This adds lines >>>>>>>>> ??????????????????????????? and filenames to the end of the >>>>>>>>> ??????????????????????????? wrapper JNI methods, adds tracing, >>>>>>>>> ??????????????????????????? and throws an error if need be. I've >>>>>>>>> ??????????????????????????? ported the gc/lock files to use the >>>>>>>>> ??????????????????????????? new TRACE_JNI_CALL add-on and I've >>>>>>>>> ??????????????????????????? ported a few of the tests that were >>>>>>>>> ??????????????????????????? already changed for the assignment >>>>>>>>> ??????????????????????????? webrev for JDK-8212884. >>>>>>>>> >>>>>>>>> ??????????????????????????? Webrev: >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>>>>>> >>>>>>>>> ??????????????????????????? Bug: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>>>>> >>>>>>>>> ??????????????????????????? For illustration, if I force an error >>>>>>>>> ??????????????????????????? to the AP04/ap04t03 test and set the >>>>>>>>> ??????????????????????????? verbosity on, I get something like: >>>>>>>>> >>>>>>>>> ??????????????????????????? >> Calling JNI method FindClass from >>>>>>>>> ??????????????????????????? ap04t003.cpp:343 >>>>>>>>> ??????????????????????????? >> Calling with these parameter(s): >>>>>>>>> ??????????????????????????? java/lang/Threadd >>>>>>>>> ??????????????????????????? Wait for thread to finish >>>>>>>>> ??????????????????????????? << Called JNI method FindClass from >>>>>>>>> ??????????????????????????? ap04t003.cpp:343 >>>>>>>>> ??????????????????????????? Exception in thread "Thread-0" >>>>>>>>> java.lang.NoClassDefFoundError: >>>>>>>>> ??????????????????????????? java/lang/Threadd >>>>>>>>> ??????????????????????????? at >>>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>>>> >>>>>>>>> ??????????????????????????? Method) >>>>>>>>> ??????????????????????????? at >>>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>>>> >>>>>>>>> ??????????????????????????? at >>>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>>>> >>>>>>>>> ??????????????????????????? Caused by: >>>>>>>>> java.lang.ClassNotFoundException: >>>>>>>>> ??????????????????????????? java.lang.Threadd >>>>>>>>> ??????????????????????????? at >>>>>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>>>>> >>>>>>>>> ??????????????????????????? at >>>>>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>>>>> >>>>>>>>> ??????????????????????????? at >>>>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>>>>> ??????????????????????????? ... 3 more >>>>>>>>> ??????????????????????????? FATAL ERROR in native method: JNI >>>>>>>>> ??????????????????????????? method FindClass : internal error >>>>>>>>> ??????????????????????????? from ap04t003.cpp:343 >>>>>>>>> ??????????????????????????? at >>>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>>>>> >>>>>>>>> ??????????????????????????? Method) >>>>>>>>> ??????????????????????????? at >>>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>>>>> >>>>>>>>> ??????????????????????????? at >>>>>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>>>>> >>>>>>>>> >>>>>>>>> ??????????????????????????? Questions/comments I have about >>>>>>>>> this are: >>>>>>>>> ??????????????????????????? ? - Do we want to force fatal errors >>>>>>>>> ??????????????????????????? when a JNI call fails in general? >>>>>>>>> ??????????????????????????? Most of these tests do the right >>>>>>>>> ??????????????????????????? thing and test the return of the JNI >>>>>>>>> ??????????????????????????? calls, for example: >>>>>>>>> ??????????????????????????? thrClass = >>>>>>>>> jni->FindClass("java/lang/Threadd", >>>>>>>>> ??????????????????????????? TRACE_JNI_CALL); >>>>>>>>> ??????????????????????????? ? ? if (thrClass == NULL) { >>>>>>>>> >>>>>>>>> ??????????????????????????? but now the wrapper actually would do >>>>>>>>> ??????????????????????????? a fatal if the FindClass call would >>>>>>>>> ??????????????????????????? return a nullptr, so we could remove >>>>>>>>> ??????????????????????????? that test altogether. What do you >>>>>>>>> think? >>>>>>>>> ??????????????????????????? ? ? - I prefer to leave them as the >>>>>>>>> ??????????????????????????? tests then become closer to what real >>>>>>>>> ??????????????????????????? users would have in their code and is >>>>>>>>> ??????????????????????????? the "recommended" way of doing it >>>>>>>>> >>>>>>>>> ??????????????????????????? ? ?- The alternative is to use the >>>>>>>>> ??????????????????????????? NonFatalError I added which then just >>>>>>>>> ??????????????????????????? prints out that something went wrong, >>>>>>>>> ??????????????????????????? letting the test continue. Question >>>>>>>>> ??????????????????????????? will be what should be the default? >>>>>>>>> ??????????????????????????? The fatal or the non-fatal error >>>>>>>>> ??????????????????????????? handling? >>>>>>>>> >>>>>>>>> ??????????????????????????? On a different subject: >>>>>>>>> ??????????????????????????? ? - On the new tests, I've removed >>>>>>>>> ??????????????????????????? the NSK_JNI_VERIFY since the JNI >>>>>>>>> ??????????????????????????? wrapper handles the tracing and the >>>>>>>>> ??????????????????????????? verify in almost the same way; only >>>>>>>>> ??????????????????????????? difference I can really tell is that >>>>>>>>> ??????????????????????????? the complain method from NSK has a >>>>>>>>> ??????????????????????????? max complain before stopping to >>>>>>>>> ??????????????????????????? "complain"; I have not added that >>>>>>>>> ??????????????????????????? part of the code in this webrev >>>>>>>>> >>>>>>>>> ??????????????????????????? Once we decide on these, I can >>>>>>>>> ??????????????????????????? continue on the files from >>>>>>>>> ??????????????????????????? JDK-8212884 and then do both the >>>>>>>>> ??????????????????????????? assignment in an if extraction >>>>>>>>> ??????????????????????????? followed-by this type of webrev in an >>>>>>>>> ??????????????????????????? easier fashion. Depending on >>>>>>>>> ??????????????????????????? decisions here, NSK*VERIFY can be >>>>>>>>> ??????????????????????????? deprecated as well as we go forward. >>>>>>>>> >>>>>>>>> ??????????????????????????? Thank you for the reviews/comments :) >>>>>>>>> ??????????????????????????? Jc >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ??????????????????????? -- >>>>>>>>> ??????????????????????? Thanks, >>>>>>>>> ??????????????????????? Jc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ??????????????????? -- >>>>>>>> ??????????????????? Thanks, >>>>>>>> ??????????????????? Jc >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> ??????????????? -- >>>>>>> ??????????????? Thanks, >>>>>>> ??????????????? Jc >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ??????????? -- >>>>>> ??????????? Thanks, >>>>>> ??????????? Jc >>>>> >>>>> >>>>> >>>>> >>>>> ??????? -- >>>>> ??????? Thanks, >>>>> ??????? Jc >>>> >>> >>> >>> >>> ??? -- >>> ??? Thanks, >>> ??? Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc > > From vladimir.x.ivanov at oracle.com Tue Dec 4 05:10:52 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 3 Dec 2018 21:10:52 -0800 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot In-Reply-To: <896662e4-fd36-a466-0e4d-2c69c59e8889@oracle.com> References: <896662e4-fd36-a466-0e4d-2c69c59e8889@oracle.com> Message-ID: > I just added a query to the bug report. I can't tell what other flags > get tested in same run that uses DeoptimizeALot. Is it better to exclude > the test from all such runs, or to force DeoptimizeALot off in the @run > lines? The flag is used in hs-comp testing (tier7-8) and it turns on heavyweight stress mode. The actual flag combinations being tested are: -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation [-Xcomp] -XX:+DeoptimizeALot I'm in favor of excluding the test, since I don't see much value in running the test with the flag explicitly disabled: it duplicates testing in other configurations. > PS. Please update the copyright line to add 2018. Sure, will do. Best regards, Vladimir Ivanov From david.holmes at oracle.com Tue Dec 4 05:55:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 15:55:02 +1000 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot In-Reply-To: References: <896662e4-fd36-a466-0e4d-2c69c59e8889@oracle.com> Message-ID: Hi Vladimir, On 4/12/2018 3:10 pm, Vladimir Ivanov wrote: > >> I just added a query to the bug report. I can't tell what other flags >> get tested in same run that uses DeoptimizeALot. Is it better to >> exclude the test from all such runs, or to force DeoptimizeALot off in >> the @run lines? > > The flag is used in hs-comp testing (tier7-8) and it turns on > heavyweight stress mode. The actual flag combinations being tested are: > > ? -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation > [-Xcomp] -XX:+DeoptimizeALot > > I'm in favor of excluding the test, since I don't see much value in > running the test with the flag explicitly disabled: it duplicates > testing in other configurations. Which other tier config would this duplicate? As long as we are stress testing the reachability fence then I'm okay with this, but I want to be sure we are stress testing it. Ensuring reachability fences are kept in place and maintain object liveness is exactly what we need to check under heavy compilation etc. Thanks, David >> PS. Please update the copyright line to add 2018. > Sure, will do. > > Best regards, > Vladimir Ivanov From jcbeyler at google.com Tue Dec 4 06:04:01 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 3 Dec 2018 22:04:01 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> Message-ID: Hi both, Thanks for the reviews! Since Serguei did not insist on get_basename, I went for get_dirname since the method is a local static method and won't have its name start spreading, I think it's ok too. For the naming of the local variable, the idea initially was to use the same name as the local variable for JNIEnv already used to reduce the code change. Since I'm now adding the line macro at the end anyway, this does not matter anymore so I converged all local variables to "jni". So, without further ado, here is the new version: Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 This passes the various tests changed by the webrev on my dev machine. Let me know what you think, Jc On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > On 12/3/18 20:15, Chris Plummer wrote: > > Hi JC, > > > > Overall it looks good. A few naming nits thought: > > > > In bi01t001.cpp, why have you declared the ExceptionCheckingJniEnvPtr > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename the > > method argument passed in from jni to jni_env. > > > > Related to this, I also noticed in some files that already are using > > ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, you > > delcared it as env(jni_env). So that means there are 3 different names > > you have used for the ExceptionCheckingJniEnvPtr local variable. They > > should be consistent. > > > > Also, can you rename get_basename() to get_dirname()? I know Serguei > > suggested get_basename() a while back, but unless "basename" is > > commonly used for this purpose, I think "dirname" is more self > > explanatory. > > In general, I'm Okay with get_dirname(). > Just to mention dirname can be both short or full, so it is a little > confusing as well. > It is the reason why the get_basename() was suggested. > However, I do not insist on get_basename() nor get_full_dirname(). :) > > Thanks, > Serguei > > > > thanks, > > > > Chris > > > > On 12/2/18 10:29 PM, David Holmes wrote: > >> Hi Jc, > >> > >> I've been lurking on this one and have had a look through. I'm okay > >> with the FatalError approach for the tests - we don't expect anything > >> to go wrong in a well written test in a correctly functioning VM. > >> > >> Thanks, > >> David > >> > >> > >> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: > >>> Hi all, > >>> > >>> Would someone on the GC or runtime team be motivated to give this a > >>> review? :) > >>> > >>> It would be much appreciated! > >>> > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > >>> > >>> Thanks for your help, > >>> Jc > >>> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler >>> > wrote: > >>> > >>> Hi Chris, > >>> > >>> Yes I was waiting for another review since you had explicitly > >>> asked :) > >>> > >>> And sounds good that when someone from GC or runtime gives a > >>> review, > >>> I'll wait for your full review on the webrev.02! > >>> > >>> Thanks again for your help, > >>> Jc > >>> > >>> > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer > >>> > > wrote: > >>> > >>> Hi JC, > >>> > >>> I think it would be good to get a review from the gc or runtime > >>> teams, since this also affects their tests. > >>> > >>> Also, once we are settled on this FatalError approach, I still > >>> need to give your webrev-02 a full review. I only skimmed over > >>> parts of it (I did look at all the changes in webrevo-01). > >>> > >>> thanks, > >>> > >>> Chris > >>> > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com > >>> wrote: > >>>> Hi Jc, > >>>> > >>>> I've already reviewed this too. > >>>> > >>>> Thanks, > >>>> Serguei > >>>> > >>>> > >>>> On 11/27/18 06:56, JC Beyler wrote: > >>>>> Thanks Chris, > >>>>> > >>>>> Anybody else motivated to look at this and review it? :) > >>>>> Jc > >>>>> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer > >>>>> > > >>>>> wrote: > >>>>> > >>>>> Hi JC, > >>>>> > >>>>> I'm ok with the FatalError approach, but would like to > >>>>> hear opinions from others also. > >>>>> > >>>>> thanks, > >>>>> > >>>>> Chris > >>>>> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: > >>>>>> Hi Chris, > >>>>>> > >>>>>> Thanks for taking the time to look at it and yes you > >>>>>> have raised exactly why the webrev is between two > >>>>>> worlds: in cases where a fatal error on failure is > >>>>>> wanted, should we simplify the code to remove the return > >>>>>> tests since we do them internally? Now that I've looked > >>>>>> around for non-fatal cases, I think the answer is yes, > >>>>>> it simplifies the code while maintaining the checks. > >>>>>> > >>>>>> I looked a bit and it seems that I can't find easily a > >>>>>> case where the test accepts a JNI failure to then move > >>>>>> on. Therefore, perhaps, for now, the fail with a Fatal > >>>>>> is enough and we can work on the tests to clean them up? > >>>>>> > >>>>>> That means that this is the new webrev with only Fatal > >>>>>> and cleans up the tests so that it is no longer in > >>>>>> between two worlds: > >>>>>> > >>>>>> Webrev: > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > >>>>>> > >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> > >>>>>> (This passes testing on my dev machine for all the > >>>>>> modified tests) > >>>>>> > >>>>>> with the example you provided, it now looks like: > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > >>>>>> > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> > > >>>>>> > >>>>>> > >>>>>> Where it does, to me at least, seem cleaner and less > >>>>>> "noisy". > >>>>>> > >>>>>> Let me know what you think, > >>>>>> Jc > >>>>>> > >>>>>> > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer > >>>>>> >>>>>> > wrote: > >>>>>> > >>>>>> Hi JC, > >>>>>> > >>>>>> Sorry about the delay. I had to go back an look at > >>>>>> the initial 8210842 webrev and RFR thread to see > >>>>>> what this was initially all about. > >>>>>> > >>>>>> In general the changes look good. > >>>>>> > >>>>>> I don't have a good answer to your > >>>>>> FatalError/NonFatalError question. It makes the code > >>>>>> a lot cleaner to use FatalError, but then it is a > >>>>>> behavior change, and you also need to deal with > >>>>>> tests that intentionally induce errors (do you have > >>>>>> an example of that). > >>>>>> > >>>>>> In any case, right now your webrev seems to be > >>>>>> between two worlds. You are producing FatalError, > >>>>>> but still checking results. Here's a good example: > >>>>>> > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > >>>>>> > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> > > >>>>>> > >>>>>> > >>>>>> I'm not sure if this is just a temporary state until > >>>>>> it was decided which approach to take. > >>>>>> > >>>>>> thanks, > >>>>>> > >>>>>> Chris > >>>>>> > >>>>>> > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: > >>>>>>> Hi all, > >>>>>>> > >>>>>>> Chris thought it made sense to have more eyes on > >>>>>>> this change than just serviceability as it will > >>>>>>> modify to tests that are not only serviceability > >>>>>>> tests so I've moved this to conversation here :) > >>>>>>> > >>>>>>> For convenience, I've copy-pasted the initial RFR: > >>>>>>> > >>>>>>> Could I have a review for the extension and usage > >>>>>>> of the ExceptionJniWrapper. This adds lines and > >>>>>>> filenames to the end of the wrapper JNI methods, > >>>>>>> adds tracing, and throws an error if need be. I've > >>>>>>> ported the gc/lock files to use the new > >>>>>>> TRACE_JNI_CALL add-on and I've ported a few of the > >>>>>>> tests that were already changed for the assignment > >>>>>>> webrev for JDK-8212884. > >>>>>>> > >>>>>>> Webrev: > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 > >>>>>>> > >>>>>>> Bug: > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>>> > >>>>>>> For illustration, if I force an error to the > >>>>>>> AP04/ap04t03 test and set the verbosity on, I get > >>>>>>> something like: > >>>>>>> > >>>>>>> >> Calling JNI method FindClass from > >>>>>>> ap04t003.cpp:343 > >>>>>>> >> Calling with these parameter(s): > >>>>>>> java/lang/Threadd > >>>>>>> Wait for thread to finish > >>>>>>> << Called JNI method FindClass from > >>>>>>> ap04t003.cpp:343 > >>>>>>> Exception in thread "Thread-0" > >>>>>>> java.lang.NoClassDefFoundError: java/lang/Threadd > >>>>>>> at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>> > >>>>>>> Method) > >>>>>>> at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>> > >>>>>>> at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>>> > >>>>>>> Caused by: java.lang.ClassNotFoundException: > >>>>>>> java.lang.Threadd > >>>>>>> at > >>>>>>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>>>>> > >>>>>>> at > >>>>>>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>>>>> > >>>>>>> at > >>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >>>>>>> ... 3 more > >>>>>>> FATAL ERROR in native method: JNI method FindClass > >>>>>>> : internal error from ap04t003.cpp:343 > >>>>>>> at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>> > >>>>>>> Method) > >>>>>>> at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>> > >>>>>>> at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>>> > >>>>>>> > >>>>>>> Questions/comments I have about this are: > >>>>>>> - Do we want to force fatal errors when a JNI > >>>>>>> call fails in general? Most of these tests do the > >>>>>>> right thing and test the return of the JNI calls, > >>>>>>> for example: > >>>>>>> thrClass = jni->FindClass("java/lang/Threadd", > >>>>>>> TRACE_JNI_CALL); > >>>>>>> if (thrClass == NULL) { > >>>>>>> > >>>>>>> but now the wrapper actually would do a fatal if > >>>>>>> the FindClass call would return a nullptr, so we > >>>>>>> could remove that test altogether. What do you > >>>>>>> think? > >>>>>>> - I prefer to leave them as the tests then > >>>>>>> become closer to what real users would have in > >>>>>>> their code and is the "recommended" way of doing it > >>>>>>> > >>>>>>> - The alternative is to use the NonFatalError I > >>>>>>> added which then just prints out that something > >>>>>>> went wrong, letting the test continue. Question > >>>>>>> will be what should be the default? The fatal or > >>>>>>> the non-fatal error handling? > >>>>>>> > >>>>>>> On a different subject: > >>>>>>> - On the new tests, I've removed the > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper handles the > >>>>>>> tracing and the verify in almost the same way; only > >>>>>>> difference I can really tell is that the complain > >>>>>>> method from NSK has a max complain before stopping > >>>>>>> to "complain"; I have not added that part of the > >>>>>>> code in this webrev > >>>>>>> > >>>>>>> Once we decide on these, I can continue on the > >>>>>>> files from JDK-8212884 and then do both the > >>>>>>> assignment in an if extraction followed-by this > >>>>>>> type of webrev in an easier fashion. Depending on > >>>>>>> decisions here, NSK*VERIFY can be deprecated as > >>>>>>> well as we go forward. > >>>>>>> > >>>>>>> Thanks! > >>>>>>> Jc > >>>>>>> > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer > >>>>>>> >>>>>>> > wrote: > >>>>>>> > >>>>>>> On 11/19/18 10:07 AM, JC Beyler wrote: > >>>>>>>> Hi all, > >>>>>>>> > >>>>>>>> @David/Chris: should I then push this RFR to > >>>>>>>> the hotspot mailing or the runtime one? For > >>>>>>>> what it's worth, a lot of the tests under the > >>>>>>>> vmTestbase are jvmti so the review also > >>>>>>>> affects serviceability; it just turns out I > >>>>>>>> started with the GC originally and then hit > >>>>>>>> some other tests I had touched via the > >>>>>>>> assignment extraction. > >>>>>>> I think hotspot would be best. > >>>>>>> > >>>>>>> Chris > >>>>>>>> > >>>>>>>> @Serguei: Done for the method renaming, for > >>>>>>>> the indent, are you talking about going from > >>>>>>>> the 8-indent to 4-indent? If so, would it not > >>>>>>>> just be better to do a new JBS bug and do the > >>>>>>>> whole files in one go? I ask because > >>>>>>>> otherwise, it will look a bit weird to have > >>>>>>>> parts of the file as 8-indent and others > >>>>>>>> 4-indent? > >>>>>>>> > >>>>>>>> Thanks for looking at it! > >>>>>>>> Jc > >>>>>>>> > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM > >>>>>>>> serguei.spitsyn at oracle.com > >>>>>>>> > >>>>>>>> >>>>>>>> > wrote: > >>>>>>>> > >>>>>>>> Hi Jc, > >>>>>>>> > >>>>>>>> We have to start this review anyway. :) > >>>>>>>> It looks good to me in general. > >>>>>>>> Thank you for your consistency in this > >>>>>>>> refactoring! > >>>>>>>> > >>>>>>>> Some minor comments. > >>>>>>>> > >>>>>>>> > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > >>>>>>>> > >>>>>>>> +static const char* remove_folders(const > >>>>>>>> char* fullname) { I'd suggest to rename > >>>>>>>> the function name to something traditional > >>>>>>>> like get_basename. Otherwise, it sounds > >>>>>>>> like this function has to really remove > >>>>>>>> folders. :) Also, all *Locker.cpp have > >>>>>>>> wrong indent in the bodies of if and while > >>>>>>>> statements. Could this be fixed with the > >>>>>>>> refactoring? I did not look on how this > >>>>>>>> impacts the tests other than > >>>>>>>> serviceability. Thanks, Serguei > >>>>>>>> > >>>>>>>> > >>>>>>>> On 11/16/18 19:43, JC Beyler wrote: > >>>>>>>>> Hi all, > >>>>>>>>> > >>>>>>>>> Anybody motivated to review this? :) > >>>>>>>>> Jc > >>>>>>>>> > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 PM JC Beyler > >>>>>>>>> >>>>>>>>> > wrote: > >>>>>>>>> > >>>>>>>>> Hi all, > >>>>>>>>> > >>>>>>>>> Could I have a review for the > >>>>>>>>> extension and usage of the > >>>>>>>>> ExceptionJniWrapper. This adds lines > >>>>>>>>> and filenames to the end of the > >>>>>>>>> wrapper JNI methods, adds tracing, > >>>>>>>>> and throws an error if need be. I've > >>>>>>>>> ported the gc/lock files to use the > >>>>>>>>> new TRACE_JNI_CALL add-on and I've > >>>>>>>>> ported a few of the tests that were > >>>>>>>>> already changed for the assignment > >>>>>>>>> webrev for JDK-8212884. > >>>>>>>>> > >>>>>>>>> Webrev: > >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ > >>>>>>>>> > >>>>>>>>> Bug: > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>>>>> > >>>>>>>>> For illustration, if I force an error > >>>>>>>>> to the AP04/ap04t03 test and set the > >>>>>>>>> verbosity on, I get something like: > >>>>>>>>> > >>>>>>>>> >> Calling JNI method FindClass from > >>>>>>>>> ap04t003.cpp:343 > >>>>>>>>> >> Calling with these parameter(s): > >>>>>>>>> java/lang/Threadd > >>>>>>>>> Wait for thread to finish > >>>>>>>>> << Called JNI method FindClass from > >>>>>>>>> ap04t003.cpp:343 > >>>>>>>>> Exception in thread "Thread-0" > >>>>>>>>> java.lang.NoClassDefFoundError: > >>>>>>>>> java/lang/Threadd > >>>>>>>>> at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>>>> > >>>>>>>>> Method) > >>>>>>>>> at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>>>> > >>>>>>>>> at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>>>>> > >>>>>>>>> Caused by: > >>>>>>>>> java.lang.ClassNotFoundException: > >>>>>>>>> java.lang.Threadd > >>>>>>>>> at > >>>>>>>>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>>>>>>> > >>>>>>>>> at > >>>>>>>>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>>>>>>> > >>>>>>>>> at > >>>>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >>>>>>>>> ... 3 more > >>>>>>>>> FATAL ERROR in native method: JNI > >>>>>>>>> method FindClass : internal error > >>>>>>>>> from ap04t003.cpp:343 > >>>>>>>>> at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>>>> > >>>>>>>>> Method) > >>>>>>>>> at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>>>> > >>>>>>>>> at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Questions/comments I have about > >>>>>>>>> this are: > >>>>>>>>> - Do we want to force fatal errors > >>>>>>>>> when a JNI call fails in general? > >>>>>>>>> Most of these tests do the right > >>>>>>>>> thing and test the return of the JNI > >>>>>>>>> calls, for example: > >>>>>>>>> thrClass = > >>>>>>>>> jni->FindClass("java/lang/Threadd", > >>>>>>>>> TRACE_JNI_CALL); > >>>>>>>>> if (thrClass == NULL) { > >>>>>>>>> > >>>>>>>>> but now the wrapper actually would do > >>>>>>>>> a fatal if the FindClass call would > >>>>>>>>> return a nullptr, so we could remove > >>>>>>>>> that test altogether. What do you > >>>>>>>>> think? > >>>>>>>>> - I prefer to leave them as the > >>>>>>>>> tests then become closer to what real > >>>>>>>>> users would have in their code and is > >>>>>>>>> the "recommended" way of doing it > >>>>>>>>> > >>>>>>>>> - The alternative is to use the > >>>>>>>>> NonFatalError I added which then just > >>>>>>>>> prints out that something went wrong, > >>>>>>>>> letting the test continue. Question > >>>>>>>>> will be what should be the default? > >>>>>>>>> The fatal or the non-fatal error > >>>>>>>>> handling? > >>>>>>>>> > >>>>>>>>> On a different subject: > >>>>>>>>> - On the new tests, I've removed > >>>>>>>>> the NSK_JNI_VERIFY since the JNI > >>>>>>>>> wrapper handles the tracing and the > >>>>>>>>> verify in almost the same way; only > >>>>>>>>> difference I can really tell is that > >>>>>>>>> the complain method from NSK has a > >>>>>>>>> max complain before stopping to > >>>>>>>>> "complain"; I have not added that > >>>>>>>>> part of the code in this webrev > >>>>>>>>> > >>>>>>>>> Once we decide on these, I can > >>>>>>>>> continue on the files from > >>>>>>>>> JDK-8212884 and then do both the > >>>>>>>>> assignment in an if extraction > >>>>>>>>> followed-by this type of webrev in an > >>>>>>>>> easier fashion. Depending on > >>>>>>>>> decisions here, NSK*VERIFY can be > >>>>>>>>> deprecated as well as we go forward. > >>>>>>>>> > >>>>>>>>> Thank you for the reviews/comments :) > >>>>>>>>> Jc > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Thanks, > >>>>>>>>> Jc > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Thanks, > >>>>>>>> Jc > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Thanks, > >>>>>>> Jc > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Thanks, > >>>>>> Jc > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Thanks, > >>>>> Jc > >>>> > >>> > >>> > >>> > >>> -- > >>> Thanks, > >>> Jc > >>> > >>> > >>> > >>> -- > >>> > >>> Thanks, > >>> Jc > > > > > > -- Thanks, Jc From rkennke at redhat.com Tue Dec 4 07:10:28 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 08:10:28 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Round 5 of Shenandoah review includes: - A fix for the @requires tag in TestFullGCCountTest.java. It should be correct now. We believe the CMS @requires was also not quite right and fixed it the same. It reads now: Don't run this test if: - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is true, as set by harness - Actual GC set by harness is Shenandoah *and* ExplicitGCInvokesConcurrent is not set false by harness (it's true by default in Shenandoah, so this needs to be double-inverteed). The @requires for CMS was wrong before (we think), because it would also filter defaultGC + ExplicitGCInvokesConcurrent. - Sorting of macros was fixed, as was pointed out by Per - Some stuff was added to SA, as suggested by Jini - Rebased on most current jdk/jdk code Webrevs: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ I also need reviews from GC reviewers for the CSR: https://bugs.openjdk.java.net/browse/JDK-8214349 I already got reviews for: [x] shared-runtime (coleenp) [x] shared-compiler (kvn) I got reviews for shared-build, but an earlier version, so maybe makes sense to look over this again. Erik J, Magnus? I still need approvals for: [ ] shared-build (kvn, erikj, ihse, pliden) [ ] shared-gc (pliden, kbarrett) [ ] shared-serviceability (jgeorge, pliden) [ ] shared-tests (lmesnik, pliden) [ ] shenandoah-gc [ ] shenandoah-tests Thanks for your patience and ongoing support! Cheers, Roman > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From magnus.ihse.bursie at oracle.com Tue Dec 4 07:14:25 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 4 Dec 2018 08:14:25 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: > 3 dec. 2018 kl. 20:27 skrev Roman Kennke : > > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? Build changes look good. /Magnus > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> *) shenandoah-gc >>>>> >>>>> - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> *) shared-gc >>>>> >>>>> - This is mostly boilerplate that is common to any GC >>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> *) shared-serviceability >>>>> >>>>> - The usual code to support another GC >>>>> >>>>> *) shared-runtime >>>>> >>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> e.g. ClassLoaderData, CodeCache, etc >>>>> - Warning and disabling JFR LeakProfiler >>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> - Locks initialization in mutexLocker.cpp as usual >>>>> - VM operations defines for Shenandoah's VM ops >>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> - The usual macros in macro.hpp >>>>> >>>>> *) shared-build >>>>> >>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> don't) >>>>> >>>>> *) shared-tests >>>>> >>>>> - Test infrastructure to support Shenandoah >>>>> - Shenandoah test groups >>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> *) shenandoah-tests >>>>> >>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> - {Windows} x {x86_64}, >>>>> - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> - the new Shenandoah tests >>>>> - jcstress with/without aggressive Shenandoah verification >>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman > From jini.george at oracle.com Tue Dec 4 07:23:55 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 4 Dec 2018 12:53:55 +0530 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi Roman, Thank you for making the changes. The SA portion looks good to me. One nit though: In src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java, in printGCAlgorithm(), does displaying the nbr of Parallel GC threads not make sense for Shenandoah (like it is for G1, ZGC, etc)? Thank you, Jini. On 12/4/2018 12:57 AM, Roman Kennke wrote: > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ?*) shenandoah-gc >>>>> >>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ?*) shared-gc >>>>> >>>>> ??? - This is mostly boilerplate that is common to any GC >>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ?*) shared-serviceability >>>>> >>>>> ??? - The usual code to support another GC >>>>> >>>>> ?*) shared-runtime >>>>> >>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>> ??? - Warning and disabling JFR LeakProfiler >>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ??? - The usual macros in macro.hpp >>>>> >>>>> ?*) shared-build >>>>> >>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> ????? don't) >>>>> >>>>> ?*) shared-tests >>>>> >>>>> ??? - Test infrastructure to support Shenandoah >>>>> ??? - Shenandoah test groups >>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> ?*) shenandoah-tests >>>>> >>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> ??? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ? - {Windows} x {x86_64}, >>>>> ? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ? - the new Shenandoah tests >>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > From david.holmes at oracle.com Tue Dec 4 07:29:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 17:29:03 +1000 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> Message-ID: Looks fine to me. Thanks, David On 4/12/2018 4:04 pm, JC Beyler wrote: > Hi both, > > Thanks for the reviews! Since Serguei did not insist on get_basename, I > went for get_dirname since the method is a local static method and won't > have its name start spreading, I think it's ok too. > > For the naming of the local variable, the idea initially was to use the > same name as the local variable for JNIEnv already used to reduce the > code change. Since I'm now adding the line macro at the end anyway, this > does not matter anymore so I converged all local variables to "jni". > > So, without further ado, here is the new version: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > This passes the various tests changed by the webrev on my dev machine. > > Let me know what you think, > Jc > > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com > > wrote: > > On 12/3/18 20:15, Chris Plummer wrote: > > Hi JC, > > > > Overall it looks good. A few naming nits thought: > > > > In bi01t001.cpp, why have you declared the > ExceptionCheckingJniEnvPtr > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename the > > method argument passed in from jni to jni_env. > > > > Related to this, I also noticed in some files that already are using > > ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, you > > delcared it as env(jni_env). So that means there are 3 different > names > > you have used for the ExceptionCheckingJniEnvPtr local variable. > They > > should be consistent. > > > > Also, can you rename get_basename() to get_dirname()? I know Serguei > > suggested get_basename() a while back, but unless "basename" is > > commonly used for this purpose, I think "dirname" is more self > > explanatory. > > In general, I'm Okay with get_dirname(). > Just to mention dirname can be both short or full, so it is a little > confusing as well. > It is the reason why the get_basename() was suggested. > However, I do not insist on get_basename() nor get_full_dirname(). :) > > Thanks, > Serguei > > > > thanks, > > > > Chris > > > > On 12/2/18 10:29 PM, David Holmes wrote: > >> Hi Jc, > >> > >> I've been lurking on this one and have had a look through. I'm okay > >> with the FatalError approach for the tests - we don't expect > anything > >> to go wrong in a well written test in a correctly functioning VM. > >> > >> Thanks, > >> David > >> > >> > >> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: > >>> Hi all, > >>> > >>> Would someone on the GC or runtime team be motivated to give > this a > >>> review? :) > >>> > >>> It would be much appreciated! > >>> > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > >>> > >>> Thanks for your help, > >>> Jc > >>> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > >>> >> wrote: > >>> > >>> ??? Hi Chris, > >>> > >>> ??? Yes I was waiting for another review since you had explicitly > >>> asked :) > >>> > >>> ??? And sounds good that when someone from GC or runtime gives a > >>> review, > >>> ??? I'll wait for your full review on the webrev.02! > >>> > >>> ??? Thanks again for?your help, > >>> ??? Jc > >>> > >>> > >>> ??? On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer > >>> ??? > >> > wrote: > >>> > >>> ??????? Hi JC, > >>> > >>> ??????? I think it would be good to get a review from the gc or > runtime > >>> ??????? teams, since this also affects their tests. > >>> > >>> ??????? Also, once we are settled on this FatalError approach, > I still > >>> ??????? need to give your webrev-02 a full review. I only > skimmed over > >>> ??????? parts of it (I did look at all the changes in webrevo-01). > >>> > >>> ??????? thanks, > >>> > >>> ??????? Chris > >>> > >>> ??????? On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com > > >>> ??????? > wrote: > >>>> ??????? Hi Jc, > >>>> > >>>> ??????? I've already reviewed this too. > >>>> > >>>> ??????? Thanks, > >>>> ??????? Serguei > >>>> > >>>> > >>>> ??????? On 11/27/18 06:56, JC Beyler wrote: > >>>>> ??????? Thanks Chris, > >>>>> > >>>>> ??????? Anybody else motivated to look at this and review it? :) > >>>>> ??????? Jc > >>>>> > >>>>> ??????? On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer > >>>>> ??????? >> > >>>>> ??????? wrote: > >>>>> > >>>>> ??????????? Hi JC, > >>>>> > >>>>> ??????????? I'm ok with the FatalError approach, but would > like to > >>>>> ??????????? hear opinions from others also. > >>>>> > >>>>> ??????????? thanks, > >>>>> > >>>>> ??????????? Chris > >>>>> > >>>>> ??????????? On 11/21/18 8:19 AM, JC Beyler wrote: > >>>>>> ??????????? Hi Chris, > >>>>>> > >>>>>> ??????????? Thanks for taking the time to look at it and yes you > >>>>>> ??????????? have raised exactly why the webrev is between two > >>>>>> ??????????? worlds: in cases where a fatal error on failure is > >>>>>> ??????????? wanted, should we simplify the code to remove > the return > >>>>>> ??????????? tests since we do them internally? Now that I've > looked > >>>>>> ??????????? around for non-fatal cases, I think the answer > is yes, > >>>>>> ??????????? it simplifies the code while maintaining the checks. > >>>>>> > >>>>>> ??????????? I looked a bit and it seems that I can't find > easily a > >>>>>> ??????????? case where the test accepts a JNI failure to > then move > >>>>>> ??????????? on. Therefore, perhaps, for now, the fail with a > Fatal > >>>>>> ??????????? is enough and we can work on the tests to clean > them up? > >>>>>> > >>>>>> ??????????? That means that this is the new webrev with only > Fatal > >>>>>> ??????????? and cleans up the tests so that it is no longer in > >>>>>> ??????????? between two worlds: > >>>>>> > >>>>>> ??????????? Webrev: > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > >>>>>> > >>>>>> ??????????? Bug: > https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> > >>>>>> ??????????? (This passes testing on my dev machine for all the > >>>>>> ??????????? modified tests) > >>>>>> > >>>>>> ??????????? with the example you provided, it now looks like: > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > >>>>>> > >>>>>> > > > >>>>>> > >>>>>> > >>>>>> ??????????? Where it does, to me at least, seem cleaner and less > >>>>>> ??????????? "noisy". > >>>>>> > >>>>>> ??????????? Let me know what you think, > >>>>>> ??????????? Jc > >>>>>> > >>>>>> > >>>>>> ??????????? On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer > >>>>>> ??????????? > >>>>>> ??????????? >> wrote: > >>>>>> > >>>>>> ??????????????? Hi JC, > >>>>>> > >>>>>> ??????????????? Sorry about the delay. I had to go back an > look at > >>>>>> ??????????????? the initial 8210842 webrev and RFR thread to see > >>>>>> ??????????????? what this was initially all about. > >>>>>> > >>>>>> ??????????????? In general the changes look good. > >>>>>> > >>>>>> ??????????????? I don't have a good answer to your > >>>>>> ??????????????? FatalError/NonFatalError question. It makes > the code > >>>>>> ??????????????? a lot cleaner to use FatalError, but then it > is a > >>>>>> ??????????????? behavior change, and you also need to deal with > >>>>>> ??????????????? tests that intentionally induce errors (do > you have > >>>>>> ??????????????? an example of that). > >>>>>> > >>>>>> ??????????????? In any case, right now your webrev seems to be > >>>>>> ??????????????? between two worlds. You are producing > FatalError, > >>>>>> ??????????????? but still checking results. Here's a good > example: > >>>>>> > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > >>>>>> > >>>>>> > > > >>>>>> > >>>>>> > >>>>>> ??????????????? I'm not sure if this is just a temporary > state until > >>>>>> ??????????????? it was decided which approach to take. > >>>>>> > >>>>>> ??????????????? thanks, > >>>>>> > >>>>>> ??????????????? Chris > >>>>>> > >>>>>> > >>>>>> ??????????????? On 11/20/18 2:14 PM, JC Beyler wrote: > >>>>>>> ??????????????? Hi all, > >>>>>>> > >>>>>>> ??????????????? Chris thought it made sense to have more > eyes on > >>>>>>> ??????????????? this change than just serviceability as it will > >>>>>>> ??????????????? modify to tests that are not only > serviceability > >>>>>>> ??????????????? tests so I've moved this to conversation > here :) > >>>>>>> > >>>>>>> ??????????????? For convenience, I've copy-pasted the > initial RFR: > >>>>>>> > >>>>>>> ??????????????? Could I have a review for the extension and > usage > >>>>>>> ??????????????? of the ExceptionJniWrapper. This adds lines and > >>>>>>> ??????????????? filenames to the end of the wrapper JNI > methods, > >>>>>>> ??????????????? adds tracing, and throws an error if need > be. I've > >>>>>>> ??????????????? ported the gc/lock files to use the new > >>>>>>> ??????????????? TRACE_JNI_CALL add-on and I've ported a few > of the > >>>>>>> ??????????????? tests that were already changed for the > assignment > >>>>>>> ??????????????? webrev for JDK-8212884. > >>>>>>> > >>>>>>> ??????????????? Webrev: > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 > >>>>>>> > >>>>>>> ??????????????? Bug: > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>>> > >>>>>>> ??????????????? For illustration, if I force an error to the > >>>>>>> ??????????????? AP04/ap04t03 test and set the verbosity on, > I get > >>>>>>> ??????????????? something like: > >>>>>>> > >>>>>>> ??????????????? >> Calling JNI method FindClass from > >>>>>>> ap04t003.cpp:343 > >>>>>>> ??????????????? >> Calling with these parameter(s): > >>>>>>> ??????????????? ? ? ? ? java/lang/Threadd > >>>>>>> ??????????????? Wait for thread to finish > >>>>>>> ??????????????? << Called JNI method FindClass from > >>>>>>> ap04t003.cpp:343 > >>>>>>> ??????????????? Exception in thread "Thread-0" > >>>>>>> ??????????????? java.lang.NoClassDefFoundError: > java/lang/Threadd > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>> > >>>>>>> ??????????????? Method) > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>> > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>>>>> > >>>>>>> ??????????????? Caused by: java.lang.ClassNotFoundException: > >>>>>>> ??????????????? java.lang.Threadd > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>>>>> > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>>>>> > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >>>>>>> ??????????????? ? ? ? ? ... 3 more > >>>>>>> ??????????????? FATAL ERROR in native method: JNI method > FindClass > >>>>>>> ??????????????? : internal error from ap04t003.cpp:343 > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>> > >>>>>>> ??????????????? Method) > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>> > >>>>>>> ??????????????? ? ? ? ? at > >>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>>>>> > >>>>>>> > >>>>>>> ??????????????? Questions/comments I have about this are: > >>>>>>> ??????????????? ? - Do we want to force fatal errors when a JNI > >>>>>>> ??????????????? call fails in general? Most of these tests > do the > >>>>>>> ??????????????? right thing and test the return of the JNI > calls, > >>>>>>> ??????????????? for example: > >>>>>>> ??????????????? ? ? thrClass = > jni->FindClass("java/lang/Threadd", > >>>>>>> ??????????????? TRACE_JNI_CALL); > >>>>>>> ??????????????? ? ? if (thrClass == NULL) { > >>>>>>> > >>>>>>> ??????????????? but now the wrapper actually would do a > fatal if > >>>>>>> ??????????????? the FindClass call would return a nullptr, > so we > >>>>>>> ??????????????? could remove that test altogether. What do you > >>>>>>> think? > >>>>>>> ??????????????? ? ? - I prefer to leave them as the tests then > >>>>>>> ??????????????? become closer to what real users would have in > >>>>>>> ??????????????? their code and is the "recommended" way of > doing it > >>>>>>> > >>>>>>> ??????????????? ? ?- The alternative is to use the > NonFatalError I > >>>>>>> ??????????????? added which then just prints out that something > >>>>>>> ??????????????? went wrong, letting the test continue. Question > >>>>>>> ??????????????? will be what should be the default? The > fatal or > >>>>>>> ??????????????? the non-fatal error handling? > >>>>>>> > >>>>>>> ??????????????? On a different subject: > >>>>>>> ??????????????? ? - On the new tests, I've removed the > >>>>>>> ??????????????? NSK_JNI_VERIFY since the JNI wrapper > handles the > >>>>>>> ??????????????? tracing and the verify in almost the same > way; only > >>>>>>> ??????????????? difference I can really tell is that the > complain > >>>>>>> ??????????????? method from NSK has a max complain before > stopping > >>>>>>> ??????????????? to "complain"; I have not added that part > of the > >>>>>>> ??????????????? code in this webrev > >>>>>>> > >>>>>>> ??????????????? Once we decide on these, I can continue on the > >>>>>>> ??????????????? files from JDK-8212884 and then do both the > >>>>>>> ??????????????? assignment in an if extraction followed-by this > >>>>>>> ??????????????? type of webrev in an easier fashion. > Depending on > >>>>>>> ??????????????? decisions here, NSK*VERIFY can be deprecated as > >>>>>>> ??????????????? well as we go forward. > >>>>>>> > >>>>>>> ??????????????? Thanks! > >>>>>>> ??????????????? Jc > >>>>>>> > >>>>>>> ??????????????? On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer > >>>>>>> ??????????????? > >>>>>>> >> wrote: > >>>>>>> > >>>>>>> ??????????????????? On 11/19/18 10:07 AM, JC Beyler wrote: > >>>>>>>> ??????????????????? Hi all, > >>>>>>>> > >>>>>>>> ??????????????????? @David/Chris: should I then push this > RFR to > >>>>>>>> ??????????????????? the hotspot mailing or the runtime > one? For > >>>>>>>> ??????????????????? what it's worth, a lot of the tests > under the > >>>>>>>> ??????????????????? vmTestbase are jvmti so the review also > >>>>>>>> ??????????????????? affects serviceability; it just turns > out I > >>>>>>>> ??????????????????? started with the GC originally and > then hit > >>>>>>>> ??????????????????? some other tests I had touched via the > >>>>>>>> ??????????????????? assignment extraction. > >>>>>>> ??????????????????? I think hotspot would be best. > >>>>>>> > >>>>>>> ??????????????????? Chris > >>>>>>>> > >>>>>>>> ??????????????????? @Serguei: Done for the method > renaming, for > >>>>>>>> ??????????????????? the indent, are you talking about > going from > >>>>>>>> ??????????????????? the 8-indent to 4-indent? If so, would > it not > >>>>>>>> ??????????????????? just be better to do a new JBS bug and > do the > >>>>>>>> ??????????????????? whole files in one go? I ask because > >>>>>>>> ??????????????????? otherwise, it will look a bit weird to > have > >>>>>>>> ??????????????????? parts of the file as 8-indent and others > >>>>>>>> 4-indent? > >>>>>>>> > >>>>>>>> ??????????????????? Thanks for looking at it! > >>>>>>>> ??????????????????? Jc > >>>>>>>> > >>>>>>>> ??????????????????? On Mon, Nov 19, 2018 at 1:25 AM > >>>>>>>> serguei.spitsyn at oracle.com > >>>>>>>> > > >>>>>>>> ??????????????????? > >>>>>>>> >> wrote: > >>>>>>>> > >>>>>>>> ??????????????????????? Hi Jc, > >>>>>>>> > >>>>>>>> ??????????????????????? We have to start this review > anyway. :) > >>>>>>>> ??????????????????????? It looks good to me in general. > >>>>>>>> ??????????????????????? Thank you for your consistency in this > >>>>>>>> ??????????????????????? refactoring! > >>>>>>>> > >>>>>>>> ??????????????????????? Some minor comments. > >>>>>>>> > >>>>>>>> > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > > >>>>>>>> > >>>>>>>> ??????????????????????? +static const char* > remove_folders(const > >>>>>>>> ??????????????????????? char* fullname) { I'd suggest to > rename > >>>>>>>> ??????????????????????? the function name to something > traditional > >>>>>>>> ??????????????????????? like get_basename. Otherwise, it > sounds > >>>>>>>> ??????????????????????? like this function has to really > remove > >>>>>>>> ??????????????????????? folders. :) Also, all *Locker.cpp have > >>>>>>>> ??????????????????????? wrong indent in the bodies of if > and while > >>>>>>>> ??????????????????????? statements. Could this be fixed > with the > >>>>>>>> ??????????????????????? refactoring? I did not look on how > this > >>>>>>>> ??????????????????????? impacts the tests other than > >>>>>>>> ??????????????????????? serviceability. Thanks, Serguei > >>>>>>>> > >>>>>>>> > >>>>>>>> ??????????????????????? On 11/16/18 19:43, JC Beyler wrote: > >>>>>>>>> ??????????????????????? Hi all, > >>>>>>>>> > >>>>>>>>> ??????????????????????? Anybody motivated to review this? :) > >>>>>>>>> ??????????????????????? Jc > >>>>>>>>> > >>>>>>>>> ??????????????????????? On Wed, Nov 7, 2018 at 9:53 PM JC > Beyler > >>>>>>>>> ??????????????????????? > >>>>>>>>> >> wrote: > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Hi all, > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Could I have a review for the > >>>>>>>>> ??????????????????????????? extension and usage of the > >>>>>>>>> ??????????????????????????? ExceptionJniWrapper. This > adds lines > >>>>>>>>> ??????????????????????????? and filenames to the end of the > >>>>>>>>> ??????????????????????????? wrapper JNI methods, adds > tracing, > >>>>>>>>> ??????????????????????????? and throws an error if need > be. I've > >>>>>>>>> ??????????????????????????? ported the gc/lock files to > use the > >>>>>>>>> ??????????????????????????? new TRACE_JNI_CALL add-on and > I've > >>>>>>>>> ??????????????????????????? ported a few of the tests > that were > >>>>>>>>> ??????????????????????????? already changed for the > assignment > >>>>>>>>> ??????????????????????????? webrev for JDK-8212884. > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Webrev: > >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Bug: > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>>>>> > >>>>>>>>> ??????????????????????????? For illustration, if I force > an error > >>>>>>>>> ??????????????????????????? to the AP04/ap04t03 test and > set the > >>>>>>>>> ??????????????????????????? verbosity on, I get something > like: > >>>>>>>>> > >>>>>>>>> ??????????????????????????? >> Calling JNI method > FindClass from > >>>>>>>>> ??????????????????????????? ap04t003.cpp:343 > >>>>>>>>> ??????????????????????????? >> Calling with these > parameter(s): > >>>>>>>>> ??????????????????????????? java/lang/Threadd > >>>>>>>>> ??????????????????????????? Wait for thread to finish > >>>>>>>>> ??????????????????????????? << Called JNI method > FindClass from > >>>>>>>>> ??????????????????????????? ap04t003.cpp:343 > >>>>>>>>> ??????????????????????????? Exception in thread "Thread-0" > >>>>>>>>> java.lang.NoClassDefFoundError: > >>>>>>>>> ??????????????????????????? java/lang/Threadd > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Method) > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>>>> > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Caused by: > >>>>>>>>> java.lang.ClassNotFoundException: > >>>>>>>>> ??????????????????????????? java.lang.Threadd > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>>>>>>> > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>>>>>>> > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >>>>>>>>> ??????????????????????????? ... 3 more > >>>>>>>>> ??????????????????????????? FATAL ERROR in native method: JNI > >>>>>>>>> ??????????????????????????? method FindClass : internal error > >>>>>>>>> ??????????????????????????? from ap04t003.cpp:343 > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Method) > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>>>>> > >>>>>>>>> ??????????????????????????? at > >>>>>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Questions/comments I have about > >>>>>>>>> this are: > >>>>>>>>> ??????????????????????????? ? - Do we want to force fatal > errors > >>>>>>>>> ??????????????????????????? when a JNI call fails in general? > >>>>>>>>> ??????????????????????????? Most of these tests do the right > >>>>>>>>> ??????????????????????????? thing and test the return of > the JNI > >>>>>>>>> ??????????????????????????? calls, for example: > >>>>>>>>> ??????????????????????????? thrClass = > >>>>>>>>> jni->FindClass("java/lang/Threadd", > >>>>>>>>> ??????????????????????????? TRACE_JNI_CALL); > >>>>>>>>> ??????????????????????????? ? ? if (thrClass == NULL) { > >>>>>>>>> > >>>>>>>>> ??????????????????????????? but now the wrapper actually > would do > >>>>>>>>> ??????????????????????????? a fatal if the FindClass call > would > >>>>>>>>> ??????????????????????????? return a nullptr, so we could > remove > >>>>>>>>> ??????????????????????????? that test altogether. What do > you > >>>>>>>>> think? > >>>>>>>>> ??????????????????????????? ? ? - I prefer to leave them > as the > >>>>>>>>> ??????????????????????????? tests then become closer to > what real > >>>>>>>>> ??????????????????????????? users would have in their > code and is > >>>>>>>>> ??????????????????????????? the "recommended" way of doing it > >>>>>>>>> > >>>>>>>>> ??????????????????????????? ? ?- The alternative is to > use the > >>>>>>>>> ??????????????????????????? NonFatalError I added which > then just > >>>>>>>>> ??????????????????????????? prints out that something > went wrong, > >>>>>>>>> ??????????????????????????? letting the test continue. > Question > >>>>>>>>> ??????????????????????????? will be what should be the > default? > >>>>>>>>> ??????????????????????????? The fatal or the non-fatal error > >>>>>>>>> ??????????????????????????? handling? > >>>>>>>>> > >>>>>>>>> ??????????????????????????? On a different subject: > >>>>>>>>> ??????????????????????????? ? - On the new tests, I've > removed > >>>>>>>>> ??????????????????????????? the NSK_JNI_VERIFY since the JNI > >>>>>>>>> ??????????????????????????? wrapper handles the tracing > and the > >>>>>>>>> ??????????????????????????? verify in almost the same > way; only > >>>>>>>>> ??????????????????????????? difference I can really tell > is that > >>>>>>>>> ??????????????????????????? the complain method from NSK > has a > >>>>>>>>> ??????????????????????????? max complain before stopping to > >>>>>>>>> ??????????????????????????? "complain"; I have not added that > >>>>>>>>> ??????????????????????????? part of the code in this webrev > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Once we decide on these, I can > >>>>>>>>> ??????????????????????????? continue on the files from > >>>>>>>>> ??????????????????????????? JDK-8212884 and then do both the > >>>>>>>>> ??????????????????????????? assignment in an if extraction > >>>>>>>>> ??????????????????????????? followed-by this type of > webrev in an > >>>>>>>>> ??????????????????????????? easier fashion. Depending on > >>>>>>>>> ??????????????????????????? decisions here, NSK*VERIFY can be > >>>>>>>>> ??????????????????????????? deprecated as well as we go > forward. > >>>>>>>>> > >>>>>>>>> ??????????????????????????? Thank you for the > reviews/comments :) > >>>>>>>>> ??????????????????????????? Jc > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ??????????????????????? -- > >>>>>>>>> ??????????????????????? Thanks, > >>>>>>>>> ??????????????????????? Jc > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> ??????????????????? -- > >>>>>>>> ??????????????????? Thanks, > >>>>>>>> ??????????????????? Jc > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> ??????????????? -- > >>>>>>> ??????????????? Thanks, > >>>>>>> ??????????????? Jc > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> ??????????? -- > >>>>>> ??????????? Thanks, > >>>>>> ??????????? Jc > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ??????? -- > >>>>> ??????? Thanks, > >>>>> ??????? Jc > >>>> > >>> > >>> > >>> > >>> ??? -- > >>> ??? Thanks, > >>> ??? Jc > >>> > >>> > >>> > >>> -- > >>> > >>> Thanks, > >>> Jc > > > > > > > > -- > > Thanks, > Jc From rkennke at redhat.com Tue Dec 4 08:01:23 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 09:01:23 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi Magnus, >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? > > Build changes look good. Thanks, Magnus! Roman From rkennke at redhat.com Tue Dec 4 08:00:55 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 09:00:55 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <56886a20-3543-3006-8d11-164786a1e596@redhat.com> Hi Jini, > Thank you for making the changes. The SA portion looks good to me. Thank you! > One > nit though: > > In > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java, > in printGCAlgorithm(), does displaying the nbr of Parallel GC threads > not make sense for Shenandoah (like it is for G1, ZGC, etc)? I suppose it does. I will add it. Thanks, Roman > Thank you, > Jini. > > > On 12/4/2018 12:57 AM, Roman Kennke wrote: >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> ? - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> ? - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build????????? (kvn, erikj, ihse, pliden) >> [ ] shared-gc???????????? (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests????????? (lmesnik, pliden) >> [ ] shenandoah-gc >> [ ] shenandoah-tests >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. >>>> ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC >>>> guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>> filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the >>>>>> mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>> is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>> file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> >>>>>> which is what I intend to commit (and which should be equivalent >>>>>> to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ??*) shenandoah-gc >>>>>> >>>>>> >>>>>> ???? - Actual Shenandoah implementation, almost completely >>>>>> residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ??*) shared-gc >>>>>> >>>>>> >>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>> assert not >>>>>> fail (next to CMS and G1) >>>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ??*) shared-serviceability >>>>>> >>>>>> >>>>>> ???? - The usual code to support another GC >>>>>> >>>>>> ??*) shared-runtime >>>>>> >>>>>> >>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>> iterators to >>>>>> hook up with, >>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ???? - The usual macros in macro.hpp >>>>>> >>>>>> ??*) shared-build >>>>>> >>>>>> >>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>> source >>>>>> files, which is >>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>> significant >>>>>> regression if we >>>>>> ?????? don't) >>>>>> >>>>>> ??*) shared-tests >>>>>> >>>>>> >>>>>> ???? - Test infrastructure to support Shenandoah >>>>>> ???? - Shenandoah test groups >>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>> selected GC >>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>> sense to >>>>>> run with it >>>>>> >>>>>> ??*) shenandoah-tests >>>>>> >>>>>> >>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>> subdirectory >>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have >>>>>> some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in >>>>>> case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ?? - {Windows} x {x86_64}, >>>>>> ?? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ?? - the new Shenandoah tests >>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>> deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>> interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> From lutz.schmidt at sap.com Tue Dec 4 09:26:17 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 4 Dec 2018 09:26:17 +0000 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> Message-ID: <61D97750-0580-4A03-8B23-0888EA1A8FB5@sap.com> Hi Coleen, Vladimir, Tobias, Thomas, thanks for your reviews. With that overwhelming number of thumbs up, I'll go ahead and push. Regards, Lutz ?On 03.12.18, 20:05, "Thomas St?fe" wrote: +1 ..Thomas On Mon, Dec 3, 2018 at 5:49 PM Schmidt, Lutz wrote: > > Hi, > description in codeHeapState.cpp is adapted. In addition, I changed the name of the command line option to PrintCodeHeapAnalytics to better match the jcmd function name. > New webrev is here: http://cr.openjdk.java.net/~lucy/webrevs/8214526.01/ > Thanks, > Lutz > > On 03.12.18, 16:54, "Schmidt, Lutz" wrote: > > Oooohhh yes, good point. > Working on it... > Lutz > > On 03.12.18, 16:44, "Vladimir Kozlov" wrote: > > Please, fix comments which describe usage in codeHeapState.cpp > > Thanks, > Vladimir > > On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > > Hi Tobias, Thomas, > > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > > Thanks, > > Lutz > > > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > > > This makes much more sense than control via UL and mirrors what we do > > with e.g. NMT. There, statistic printout is triggered by jcmd and with > > -XX:+PrintNMTStatistics. > > > > Change itself looks good. I guess you need either a CSR or make the > > switch diagnostic (again, like PrintNMTStatistics). > > > > ..Thomas > > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > > > Hi all, > > > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > > > Thanks, > > > Lutz > > > > > > > > > > > > > > From per.liden at oracle.com Tue Dec 4 09:35:26 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 4 Dec 2018 10:35:26 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi Roman, On 12/3/18 8:27 PM, Roman Kennke wrote: > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) The above parts look good to me. Reviewed. Just one tiny nit (and I don't need to see a new webrev for this): In src/hotspot/share/gc/shared/gcCause.cpp you have this: + case _shenandoah_allocation_failure_evac: + return "Allocation Failure During Evac"; + + case _shenandoah_stop_vm: + return "Stopping VM"; + + case _shenandoah_concurrent_gc: + return "Shenandoah Concurrent GC"; + + case _shenandoah_traversal_gc: + return "Shenandoah Traversal GC"; + + case _shenandoah_upgrade_to_full_gc: + return "Shenandoah Upgrade To Full GC"; + And in src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java you have this: + _shenandoah_stop_vm ("Stop VM"), + _shenandoah_allocation_failure_evac ("Allocation Failure During Evacuation"), + _shenandoah_concurrent_gc ("Concurrent GC"), + _shenandoah_traversal_gc ("Traversal GC"), + _shenandoah_upgrade_to_full_gc ("Upgrade to Full GC"), It would be good to have the exact same strings in both places. There are currently small differences in all of them. "Evac" vs "Evacuation", "Stop" vs "Stopping", "Shenandoah" vs "", etc. May I also suggest that you skip "Shenandoah" in things like "Shenandoah Concurrent GC" as I kind of think it's implied by the context. But I also know that CMS/G1 isn't consistent on this point. An alternative would be to add "Shenandoah" to all of the strings to keep things consistent, but I'm not sure I like that better. You decide. > [ ] shenandoah-gc > [ ] shenandoah-tests I haven't looked very much on these parts, and I didn't plan to do so in detail right now. I think it's fine of the folks that have been working on the Shenandoah code reviewed this. cheers, Per > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ?*) shenandoah-gc >>>>> >>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ?*) shared-gc >>>>> >>>>> ??? - This is mostly boilerplate that is common to any GC >>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ?*) shared-serviceability >>>>> >>>>> ??? - The usual code to support another GC >>>>> >>>>> ?*) shared-runtime >>>>> >>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>> ??? - Warning and disabling JFR LeakProfiler >>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ??? - The usual macros in macro.hpp >>>>> >>>>> ?*) shared-build >>>>> >>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> ????? don't) >>>>> >>>>> ?*) shared-tests >>>>> >>>>> ??? - Test infrastructure to support Shenandoah >>>>> ??? - Shenandoah test groups >>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> ?*) shenandoah-tests >>>>> >>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> ??? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ? - {Windows} x {x86_64}, >>>>> ? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ? - the new Shenandoah tests >>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > From nils.eliasson at oracle.com Tue Dec 4 10:01:14 2018 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Tue, 4 Dec 2018 11:01:14 +0100 Subject: RFR(S): 8214773: Replace use of thread unsafe strtok Message-ID: <4b8cdd06-4a02-298d-c902-58ce0e79bf88@oracle.com> Hi, Hotspot has a few uses of the thread unsafe strtok function. Most of the them are probably benign since they happen during startup. The use in compilerDirectives however has been reported to cause errors in user applications that also use strtok though jni. (See bug report or mail to compiler list.) In this patch I replace all uses of strtok. The call need to go through the os specific files since windows uses a different function (strtok_s) than the rest of the platforms (strtok_r). Bug: https://bugs.openjdk.java.net/browse/JDK-8214773 Webrev: http://cr.openjdk.java.net/~neliasso/8214773/webrev.01/ Thanks to Andre Gillet at Nuance who reported the bug. Regards, Nils Eliasson From nils.eliasson at oracle.com Tue Dec 4 10:28:58 2018 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Tue, 4 Dec 2018 11:28:58 +0100 Subject: RFR(S): 8214773: Replace use of thread unsafe strtok In-Reply-To: <3b6c41ba-5f97-da92-be16-5553df045b85@oracle.com> References: <4b8cdd06-4a02-298d-c902-58ce0e79bf88@oracle.com> <3b6c41ba-5f97-da92-be16-5553df045b85@oracle.com> Message-ID: <17b37add-f53e-5bfd-d3f9-27a77e4400b1@oracle.com> Hi Tobias, I prefer that too. Thanks for noticing! // Nils On 2018-12-04 11:22, Tobias Hartmann wrote: > Hi Nils, > > this looks good to me but I prefer consistency of the pointer asterisk in declarations. I.e., either > use "char* var" (preferred) or "char *var". No new webrev required. > > Thanks, > Tobias > > On 04.12.18 11:01, Nils Eliasson wrote: >> Hi, >> >> Hotspot has a few uses of the thread unsafe strtok function. Most of the them are probably benign >> since they happen during startup. The use in compilerDirectives however has been reported to cause >> errors in user applications that also use strtok though jni. (See bug report or mail to compiler list.) >> >> In this patch I replace all uses of strtok. The call need to go through the os specific files since >> windows uses a different function (strtok_s) than the rest of the platforms (strtok_r). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214773 >> >> Webrev: http://cr.openjdk.java.net/~neliasso/8214773/webrev.01/ >> >> Thanks to Andre Gillet at Nuance who reported the bug. >> >> Regards, >> >> Nils Eliasson >> From rkennke at redhat.com Tue Dec 4 10:53:26 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 11:53:26 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <91b933b1-9b42-090e-5f0f-07454aeb456e@redhat.com> Hi Per, >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> ? - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> ? - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build????????? (kvn, erikj, ihse, pliden) >> [ ] shared-gc???????????? (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests????????? (lmesnik, pliden) > > The above parts look good to me. Reviewed. Great! Thanks! > Just one tiny nit (and I don't need to see a new webrev for this): > > In src/hotspot/share/gc/shared/gcCause.cpp you have this: > > +??? case _shenandoah_allocation_failure_evac: > +????? return "Allocation Failure During Evac"; > + > +??? case _shenandoah_stop_vm: > +????? return "Stopping VM"; > + > +??? case _shenandoah_concurrent_gc: > +????? return "Shenandoah Concurrent GC"; > + > +??? case _shenandoah_traversal_gc: > +????? return "Shenandoah Traversal GC"; > + > +??? case _shenandoah_upgrade_to_full_gc: > +????? return "Shenandoah Upgrade To Full GC"; > + > > And in > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java > you have this: > > +? _shenandoah_stop_vm ("Stop VM"), > +? _shenandoah_allocation_failure_evac ("Allocation Failure During > Evacuation"), > +? _shenandoah_concurrent_gc ("Concurrent GC"), > +? _shenandoah_traversal_gc ("Traversal GC"), > +? _shenandoah_upgrade_to_full_gc ("Upgrade to Full GC"), > > It would be good to have the exact same strings in both places. There > are currently small differences in all of them. "Evac" vs "Evacuation", > "Stop" vs "Stopping", "Shenandoah" vs "", etc. > > May I also suggest that you skip "Shenandoah" in things like "Shenandoah > Concurrent GC" as I kind of think it's implied by the context. But I > also know that CMS/G1 isn't consistent on this point. An alternative > would be to add "Shenandoah" to all of the strings to keep things > consistent, but I'm not sure I like that better. You decide. I'm addressing it here: http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-December/008572.html >> [ ] shenandoah-gc >> [ ] shenandoah-tests > > I haven't looked very much on these parts, and I didn't plan to do so in > detail right now. I think it's fine of the folks that have been working > on the Shenandoah code reviewed this. Yes, I think so too (except maybe stuff like shenandoah_globals.hpp, but that was already reviewed I think). Thanks for reviewing, Per! Roman > cheers, > Per > >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. >>>> ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC >>>> guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>> filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the >>>>>> mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>> is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>> file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> >>>>>> which is what I intend to commit (and which should be equivalent >>>>>> to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ??*) shenandoah-gc >>>>>> >>>>>> >>>>>> ???? - Actual Shenandoah implementation, almost completely >>>>>> residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ??*) shared-gc >>>>>> >>>>>> >>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>> assert not >>>>>> fail (next to CMS and G1) >>>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ??*) shared-serviceability >>>>>> >>>>>> >>>>>> ???? - The usual code to support another GC >>>>>> >>>>>> ??*) shared-runtime >>>>>> >>>>>> >>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>> iterators to >>>>>> hook up with, >>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ???? - The usual macros in macro.hpp >>>>>> >>>>>> ??*) shared-build >>>>>> >>>>>> >>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>> source >>>>>> files, which is >>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>> significant >>>>>> regression if we >>>>>> ?????? don't) >>>>>> >>>>>> ??*) shared-tests >>>>>> >>>>>> >>>>>> ???? - Test infrastructure to support Shenandoah >>>>>> ???? - Shenandoah test groups >>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>> selected GC >>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>> sense to >>>>>> run with it >>>>>> >>>>>> ??*) shenandoah-tests >>>>>> >>>>>> >>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>> subdirectory >>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have >>>>>> some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in >>>>>> case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ?? - {Windows} x {x86_64}, >>>>>> ?? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ?? - the new Shenandoah tests >>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>> deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>> interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> From david.holmes at oracle.com Tue Dec 4 12:47:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 22:47:49 +1000 Subject: RFR(S): 8214773: Replace use of thread unsafe strtok In-Reply-To: <4b8cdd06-4a02-298d-c902-58ce0e79bf88@oracle.com> References: <4b8cdd06-4a02-298d-c902-58ce0e79bf88@oracle.com> Message-ID: <2bab588c-8e0c-2b14-2314-e803cdcfadf3@oracle.com> Hi Nils, Can you just define one version in os_posix.cpp and one in os_windows.cpp? This per-os function really seems like overkill in this case. You could just add a Windows specific #define to convert strtok_r to strtok_s. Thanks, David On 4/12/2018 8:01 pm, Nils Eliasson wrote: > Hi, > > Hotspot has a few uses of the thread unsafe strtok function. Most of the > them are probably benign since they happen during startup. The use in > compilerDirectives however has been reported to cause errors in user > applications that also use strtok though jni. (See bug report or mail to > compiler list.) > > In this patch I replace all uses of strtok. The call need to go through > the os specific files since windows uses a different function (strtok_s) > than the rest of the platforms (strtok_r). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214773 > > Webrev: http://cr.openjdk.java.net/~neliasso/8214773/webrev.01/ > > Thanks to Andre Gillet at Nuance who reported the bug. > > Regards, > > Nils Eliasson > From amith.pawar at gmail.com Tue Dec 4 13:13:36 2018 From: amith.pawar at gmail.com (amith pawar) Date: Tue, 4 Dec 2018 18:43:36 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> Message-ID: Hi Derek, On Tue, Dec 4, 2018 at 3:26 AM White, Derek wrote: > Hi Amith, > > > Thanks for running the test cases for JDK-8205051 > . My theory was > incorrect - there's a reason I didn't try to fix that bug :-) > > > As far as clearing the UseNUMA flag goes, there's a few positions to take > when UseNUMA was specified, but the system has only 1 node, or the JVM is > bound to 1 node: > > 1. Pretend the user never set UseNUMA - turn it off. (Note that this > can easily occur using JVM configs in containers). > 2. Set up the JVM to use resources that it can't access - don't pay > attention to binding. > 3. Pay attention to binding. Single-node JVM goes through the UseNUMA > paths, but correctly acts as a non-NUMA system. > > > Currently the JVM does a combination of 1 & 2. And yes, 2 is a bug. > > Your patch seems to be implementing 3, which is fine, and it's great that > it actually works! But it's riskier in that it has to be correct for all > uses of UseNUMA (including various collectors), and for future uses of > UseNUMA. > > I had an expectation of what UseNUMA did in this case, but GC isn't my area > any more so I'll stay out of it now. > Thanks for giving your feedback. > > - Derek > > ------------------------------ > *From:* amith pawar > *Sent:* Monday, December 3, 2018 1:21 AM > *To:* White, Derek > *Cc:* thomas.schatzl at oracle.com; hotspot-dev at openjdk.java.net; > Prakash.Raghavendra at amd.com; roshanmanagal at gmail.com; > swatibits14 at gmail.com; roshanmangal at gmail.com > *Subject:* Re: [PATCH] JDK NUMA Interleaving issue > +Roshan Mangal > > On Mon, Dec 3, 2018 at 11:49 AM amith pawar wrote: > > Hi Derek > > On Sat, Dec 1, 2018 at 4:10 AM White, Derek > wrote: > > Hi Amith, > > > Thanks for tackling this! > > > I think that your patch may fix > https://bugs.openjdk.java.net/browse/JDK-8205051, although it's probably > fixing more than that. I was thinking that the problem was related to > calling " _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr);", > and you replaced that call with more specific node sets, which seems better > to me. > > > You can try the test cases listed in the bug to see if the patch fixes the > problem. > > Tested and didn't fix it. > > > The existing code disables UseNUMA if the process is bound to only one > node, but it looks like your patch removes this. Is that a correct read of > the patch, intentional, and/or a good idea? > > I removed that after testing it with SPECjbb for single node case. > With patch : Enabling/Disabling UseNUMA gives same score. > Without patch: UseNUMA scores 10-12% lesser than without UseNUMA > > > Thanks! > > - Derek > > > ------------------------------ > *From:* hotspot-dev on behalf of > amith pawar > *Sent:* Thursday, November 29, 2018 6:08 AM > *To:* thomas.schatzl at oracle.com > *Cc:* hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com > *Subject:* Re: [PATCH] JDK NUMA Interleaving issue > > External Email > > Hi Thomas, > > Sorry for the late reply and please check my inline comments below. > > On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl > wrote: > > > Hi Amith, > > > > welcome to the OpenJDK community! :) > > > Thanks > > > > > > On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: > > > Hi all, > > > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, > > > S1 and S2 region (if any other ) memory areas on requested Numa nodes > > > and it should not configure itself to access other Numa nodes. This > > > issue is observed only when Java is allowed to run on fewer NUMA > > > nodes than available on the system with numactl membind and > > > interleave options. Running on all the nodes does not have any > > > effect. This will cause some applications (objects residing in old > > > gen and survivor region) to run slower on system with large Numa > > > nodes. > > > > > [... long explanation...] > > > > Is it possible to summarize the problem and the changes with the > > following few lines: > > > > "NUMA interleaving of memory of old gen and survivor spaces (for > > parallel GC) tells the OS to interleave memory across all nodes of a > > NUMA system. However the VM process may be configured to be limited to > > run only on a few nodes, which means that large parts of the heap will > > be located on foreign nodes. This can incurs a large performance > > penalty. > > > > The proposed solution is to tell the OS to interleave memory only > > across available nodes when enabling NUMA." > > > OK. Since, I dont have write access to the defect link so can I request you > do the same. > > > > > We have had trouble understanding the problem statement and purpose of > > this patch when triaging (making sure the issue is understandable and > > can be worked on) as the text is rather technical. Having an easily > > understandable text also helps reviewers a lot. > > > > > Assuming my summary is appropriate, I have several other unrelated > > questions: > > > > - could you propose a better subject for this work? "JDK NUMA > > Interleaving issue" seems very generic. Something like "NUMA heap > > allocation does not respect VM membind/interleave settings" maybe? > > > This is also OK. > > > > > - there have been other NUMA related patches from AMD recently, in > > particular JDK-what is the relation to JDK-8205051? The text there > > reads awfully similar to this one, but I *think* JDK-8205051 is > > actually about making sure that the parallel gc eden is not put on > > inactive nodes. > > Can you confirm this (talk to your colleague) so that we can fix the > > description too? > > > JDK-8205051 is related to early GC. > JDK-8189922 specific to membind case where heap was allocated on non > requested NUMA nodes. > The proposed patch fixes two following issues > 1. Similar to JDK-8189922 but for interleave case. > 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer > NUMA nodes. > > > > - fyi, we are currently working on NUMA aware memory allocation support > > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to > > not step on each other's toes (see below). > > > We are not working anything related to G1. It may effect G1 if > numa_make_global function is called. > > > > > [... broken patch...] > > > > I tried to apply the patch to the jdk/jdk tree, which failed; I then > > started to manually apply it but stopped after not being able to find > > the context of some hunks. I do not think this change applies to the > > latest source tree. > > > > Please make sure that the patch applies to the latest jdk/jdk tree with > > errors. All changes generally must first go into the latest dev tree > > before you can apply for backporting. > > > > Could you please send the patch as attachment (not copy&pasted) to this > > list and cc me? Then I can create a webrev out of it. > > > > I did look a bit over the patch as much as I could (it's very hard > > trying to review a diff), some comments: > > > Sorry. Please check the attached patch. > > > > > - the _numa_interleave_memory_v2 function pointer is never assigned > > to in the patch in the CR, so it will not be used. Please make sure the > > patch is complete. > > Actually it is never defined anywhere, ie. the patch unlikely actually > > compiles even if I could apply it somewhere. > > > > Please avoid frustrating reviewers by sending incomplete patches. > > > Sorry again. Faced same issue when copied from the defect link but worked > for me from the mail. Now i have attached the patch. Can you please update > defect link with this patch ? > > > > > - I am not sure set_numa_interleave, i.e. the interleaving, should be > > done without UseNUMAInterleaving enabled. Some collectors may do their > > own interleaving in the future (JDK-8210473 / JEP-345) so this would > > massively interfere in how they work. (That issue may be because I am > > looking at a potentially incomplete diff, so forgive me if the patch > > already handles this). > > > Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect > when java externally invoked in interleave mode. It helps membind case. > > > > > - if some of the actions (interleaving/membind) fail, and it had been > > requested, it would be nice to print a log message. > > > Following two NUMA API's are used and they return nothing. Right now not > sure which cases to handle. Please suggest > void numa_tonode_memory(void *start, size_t size, int node); > void numa_interleave_memory(void *start, size_t size, struct bitmask > *nodemask); > > > > > Actually it would be nice to print information about e.g. the bitmask > > anyway in the log so that the user can easily verify that what he > > specified on the command line has been properly picked up by the VM - > > instead of looking through the proc filesystem. > > > As per the suggestion, patch is updated to print following information. > [0.001s][info][os] UseNUMA is enabled > [0.001s][info][os] Java is configured to run in interleave mode > [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, 2, > 3 > > > > > Thanks, > > Thomas > > > > > > > > > Thanks > Amit Pawar > > > Thanks, > Amit Pawar > > > > -- > With best regards, > amit pawar > -- With best regards, amit pawar From doko at ubuntu.com Tue Dec 4 13:24:00 2018 From: doko at ubuntu.com (Matthias Klose) Date: Tue, 4 Dec 2018 14:24:00 +0100 Subject: bootcycle builds x86_64-linux-gnu? Message-ID: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> with jdk-12+22, bootcycle builds fail at least on x86_64-linux-gnu. Is there some place where I can check if this kind of build succeeds for others? thanks, Matthias From shade at redhat.com Tue Dec 4 13:54:06 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 4 Dec 2018 14:54:06 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> Message-ID: On 12/4/18 2:24 PM, Matthias Klose wrote: > with jdk-12+22, bootcycle builds fail at least on x86_64-linux-gnu. Is there > some place where I can check if this kind of build succeeds for others? Just did this on jdk/jdk tip, and it passed: $ CONF=linux-x86_64-server-fastdebug make bootcycle-images ... Creating CDS archive for jdk image Stopping sjavac server Finished building target 'product-images' in configuration 'linux-x86_64-server-fastdebug' Finished building target 'bootcycle-images' in configuration 'linux-x86_64-server-fastdebug' -Aleksey From shade at redhat.com Tue Dec 4 14:24:13 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 4 Dec 2018 15:24:13 +0100 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" Message-ID: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8214787 This currently fails with aarch64-zero and ppc64el-zero. Fix: diff -r 981eb3c1b90d src/hotspot/cpu/zero/stack_zero.cpp --- a/src/hotspot/cpu/zero/stack_zero.cpp Tue Dec 04 07:06:38 2018 -0500 +++ b/src/hotspot/cpu/zero/stack_zero.cpp Tue Dec 04 15:18:29 2018 +0100 @@ -24,11 +24,11 @@ */ #include "precompiled.hpp" #include "interpreter/bytecodeInterpreter.hpp" #include "interpreter/interpreterRuntime.hpp" -#include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" #include "stack_zero.hpp" #include "stack_zero.inline.hpp" #include "runtime/frame.inline.hpp" #include "utilities/align.hpp" This is similar to related fix we did before: http://hg.openjdk.java.net/jdk/jdk/rev/78c21c351b90 Thanks, -Aleksey From thomas.stuefe at gmail.com Tue Dec 4 14:35:39 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 4 Dec 2018 15:35:39 +0100 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Message-ID: Looks fine and trivial. What is "T"? ..Thomas On Tue, Dec 4, 2018 at 3:24 PM Aleksey Shipilev wrote: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8214787 > > This currently fails with aarch64-zero and ppc64el-zero. > > Fix: > > diff -r 981eb3c1b90d src/hotspot/cpu/zero/stack_zero.cpp > --- a/src/hotspot/cpu/zero/stack_zero.cpp Tue Dec 04 07:06:38 2018 -0500 > +++ b/src/hotspot/cpu/zero/stack_zero.cpp Tue Dec 04 15:18:29 2018 +0100 > @@ -24,11 +24,11 @@ > */ > > #include "precompiled.hpp" > #include "interpreter/bytecodeInterpreter.hpp" > #include "interpreter/interpreterRuntime.hpp" > -#include "runtime/thread.hpp" > +#include "runtime/thread.inline.hpp" > #include "stack_zero.hpp" > #include "stack_zero.inline.hpp" > #include "runtime/frame.inline.hpp" > #include "utilities/align.hpp" > > This is similar to related fix we did before: > http://hg.openjdk.java.net/jdk/jdk/rev/78c21c351b90 > > Thanks, > -Aleksey > From shade at redhat.com Tue Dec 4 14:41:54 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 4 Dec 2018 15:41:54 +0100 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Message-ID: On 12/4/18 3:35 PM, Thomas St?fe wrote: > Looks fine and trivial. Thanks! > What is "T"? That is an emerging fancy nomenclature for the interesting corner case in patch difficulty designation pioneered by those who do not want to guess how many X-es to put in front of XXXXS and save everybody's time for reading and understanding the actual difficulty as well as implicitly requesting the confirmation of the aforementioned difficulty from the contributors who can make the call about patches size and time needed to be spent reading explanations about the patch itself, or any necessary metadata for the patch, difficulty included. In other words, "T" stands for "trivial". -Aleksey From zgu at redhat.com Tue Dec 4 14:45:44 2018 From: zgu at redhat.com (zgu at redhat.com) Date: Tue, 04 Dec 2018 09:45:44 -0500 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Message-ID: <1543934744.4592.0.camel@redhat.com> Looks good. -Zhengyu On Tue, 2018-12-04 at 15:24 +0100, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8214787 > > This currently fails with aarch64-zero and ppc64el-zero. > > Fix: > > diff -r 981eb3c1b90d src/hotspot/cpu/zero/stack_zero.cpp > --- a/src/hotspot/cpu/zero/stack_zero.cpp Tue Dec 04 07:06:38 2018 > -0500 > +++ b/src/hotspot/cpu/zero/stack_zero.cpp Tue Dec 04 15:18:29 2018 > +0100 > @@ -24,11 +24,11 @@ > */ > > #include "precompiled.hpp" > #include "interpreter/bytecodeInterpreter.hpp" > #include "interpreter/interpreterRuntime.hpp" > -#include "runtime/thread.hpp" > +#include "runtime/thread.inline.hpp" > #include "stack_zero.hpp" > #include "stack_zero.inline.hpp" > #include "runtime/frame.inline.hpp" > #include "utilities/align.hpp" > > This is similar to related fix we did before: > http://hg.openjdk.java.net/jdk/jdk/rev/78c21c351b90 > > Thanks, > -Aleksey > From thomas.stuefe at gmail.com Tue Dec 4 14:47:14 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 4 Dec 2018 15:47:14 +0100 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Message-ID: On Tue, Dec 4, 2018 at 3:42 PM Aleksey Shipilev wrote: > > On 12/4/18 3:35 PM, Thomas St?fe wrote: > > Looks fine and trivial. > > Thanks! > > > What is "T"? > > That is an emerging fancy nomenclature for the interesting corner case in patch difficulty > designation pioneered by those who do not want to guess how many X-es to put in front of XXXXS and > save everybody's time for reading and understanding the actual difficulty as well as implicitly > requesting the confirmation of the aforementioned difficulty from the contributors who can make the > call about patches size and time needed to be spent reading explanations about the patch itself, or > any necessary metadata for the patch, difficulty included. > > In other words, "T" stands for "trivial". Ah. So, the smallest possible patch size regardless of X. I was still thinking in T-Shirt sizes and got confused... ..Thomas > > -Aleksey > From daniel.daugherty at oracle.com Tue Dec 4 14:57:09 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 4 Dec 2018 09:57:09 -0500 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Message-ID: On 12/4/18 9:41 AM, Aleksey Shipilev wrote: > On 12/4/18 3:35 PM, Thomas St?fe wrote: >> Looks fine and trivial. > Thanks! > >> What is "T"? > That is an emerging fancy nomenclature for the interesting corner case in patch difficulty > designation pioneered by those who do not want to guess how many X-es to put in front of XXXXS and > save everybody's time for reading and understanding the actual difficulty as well as implicitly > requesting the confirmation of the aforementioned difficulty from the contributors who can make the > call about patches size and time needed to be spent reading explanations about the patch itself, or > any necessary metadata for the patch, difficulty included. > > In other words, "T" stands for "trivial". > > -Aleksey Wow... must now clean the coffee off my screen... :-) Dan From jesper.wilhelmsson at oracle.com Tue Dec 4 15:29:44 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Tue, 4 Dec 2018 16:29:44 +0100 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Message-ID: <49FC5C5C-E62C-44F4-8144-DA775902DAF9@oracle.com> The (T) in the subject line is easy to miss so I would prefer is it is also stated in the email body text if a patch is considered to be trivial. We have seen examples where the author of a patch has considered the patch to be trivial but the reviewers had other opinions. In a case like that, if a reviewer is OK with the patch, but expects a second review because he didn't see the (T), we have a problem. The XS, L, XL etc are actually orthogonal to wether a patch is trivial or not. Things like large cleanups/renamings/code removal etc can also be trivial even though they would take some time to review, and obviously there are really small patches that are very easy to understand but are not trivial at all. /Jesper > On 4 Dec 2018, at 15:41, Aleksey Shipilev wrote: > > On 12/4/18 3:35 PM, Thomas St?fe wrote: >> Looks fine and trivial. > > Thanks! > >> What is "T"? > > That is an emerging fancy nomenclature for the interesting corner case in patch difficulty > designation pioneered by those who do not want to guess how many X-es to put in front of XXXXS and > save everybody's time for reading and understanding the actual difficulty as well as implicitly > requesting the confirmation of the aforementioned difficulty from the contributors who can make the > call about patches size and time needed to be spent reading explanations about the patch itself, or > any necessary metadata for the patch, difficulty included. > > In other words, "T" stands for "trivial". > > -Aleksey > From matthias.baesken at sap.com Tue Dec 4 15:44:05 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 4 Dec 2018 15:44:05 +0000 Subject: RFR(S): 8214773: Replace use of thread unsafe strtok Message-ID: Hello Nils, the fix looks fine to me . Btw, , speaking about " thread unsafe functions" - we also still have a number of remaining strerror usages . Examples : hotspot : zErrno.cpp 49 return strerror(_error); logFileOutput.cpp 251 _file_name, strerror(errno)); jdk : jdk/src/java.base/unix/native/libjli/java_md_common.c 195 /* 196 * TODO: its safer to use strerror_r but is not available on 197 * Solaris 8. Until then.... 198 */ 199 emsg = strerror(errno); For hotspot it would probably make sense to go for os::strerror instead ; not so sure for jdk . Best regards, Matthias > Message: 1 > Date: Tue, 4 Dec 2018 22:47:49 +1000 > From: David Holmes > To: Nils Eliasson , > hotspot-dev at openjdk.java.net, hotspot-compiler- > dev at openjdk.java.net > Subject: Re: RFR(S): 8214773: Replace use of thread unsafe strtok > Message-ID: <2bab588c-8e0c-2b14-2314-e803cdcfadf3 at oracle.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > > > Hi Nils, > > Can you just define one version in os_posix.cpp and one in os_windows.cpp? > > This per-os function really seems like overkill in this case. You could > just add a Windows specific #define to convert strtok_r to strtok_s. > > Thanks, > David > > > On 4/12/2018 8:01 pm, Nils Eliasson wrote: > > Hi, > > > > Hotspot has a few uses of the thread unsafe strtok function. Most of the > > them are probably benign since they happen during startup. The use in > > compilerDirectives however has been reported to cause errors in user > > applications that also use strtok though jni. (See bug report or mail to > > compiler list.) > > > > In this patch I replace all uses of strtok. The call need to go through > > the os specific files since windows uses a different function (strtok_s) > > than the rest of the platforms (strtok_r). > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214773 > > > > Webrev: http://cr.openjdk.java.net/~neliasso/8214773/webrev.01/ > > > > Thanks to Andre Gillet at Nuance who reported the bug. > > > > Regards, > > > > Nils Eliasson > > From tobias.hartmann at oracle.com Tue Dec 4 16:34:24 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 4 Dec 2018 17:34:24 +0100 Subject: [12] RFR(S): 8208277: Code cache heap (-XX:ReservedCodeCacheSize) doesn't work with 1GB LargePages Message-ID: Hi, please review the following patch: https://bugs.openjdk.java.net/browse/JDK-8208277 http://cr.openjdk.java.net/~thartmann/8208277/webrev.00/ The code cache does not use 1GB pages even if the InitialCodeCacheSize is set to 1GB. The reason for this is the logic in CodeCache::page_size which sets the page size such that the code cache contains at least 8 pages to allow for an incremental commit of memory (otherwise, with a 1GB code cache and 1GB pages, all the memory would be committed right from the beginning). I've fixed this by setting the minimum number of pages to 1 if ReservedCodeCacheSize is equal to InitialCodeCacheSize. I've also disabled code cache segmentation if the code cache does not at least contain 8 pages because segmentation would otherwise disable the advantage of huge pages. Without my fix, only one 1GB page is used for the Java heap: -sh-4.1# base/jdk-12/fastdebug/bin/java -Xmx1g -Xms1g -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+UseLargePages -XX:ReservedCodeCacheSize=1G -XX:InitialCodeCacheSize=1G Test & [1] 1961 -sh-4.1# grep huge /proc/*/numa_maps /proc/1961/numa_maps:c0000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 -sh-4.1# grep Huge /proc/meminfo HugePages_Total: 20 HugePages_Free: 19 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 1048576 kB With my fix, two 1GB pages are used (one for the heap and one for the code cache): -sh-4.1# fix/jdk-12/fastdebug/bin/java -Xmx1g -Xms1g -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+UseLargePages -XX:ReservedCodeCacheSize=1G -XX:InitialCodeCacheSize=1G Test &[1] 13813 -sh-4.1# grep huge /proc/*/numa_maps /proc/13813/numa_maps:c0000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 /proc/13813/numa_maps:7f3400000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 -sh-4.1# grep Huge /proc/meminfo HugePages_Total: 20 HugePages_Free: 18 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 1048576 kB If InitialCodeCacheSize is set to the default, the code cache will not use 1GB pages to allow incremental committing of memory. Thanks, Tobias From vladimir.kozlov at oracle.com Tue Dec 4 16:49:55 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 4 Dec 2018 08:49:55 -0800 Subject: [12] RFR(S): 8208277: Code cache heap (-XX:ReservedCodeCacheSize) doesn't work with 1GB LargePages In-Reply-To: References: Message-ID: Looks reasonable. Did you tested Segmented CodeCache and huge pages if you use 8Gb size? Thanks, Vladimir On 12/4/18 8:34 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8208277 > http://cr.openjdk.java.net/~thartmann/8208277/webrev.00/ > > The code cache does not use 1GB pages even if the InitialCodeCacheSize is set to 1GB. The reason for > this is the logic in CodeCache::page_size which sets the page size such that the code cache contains > at least 8 pages to allow for an incremental commit of memory (otherwise, with a 1GB code cache and > 1GB pages, all the memory would be committed right from the beginning). > > I've fixed this by setting the minimum number of pages to 1 if ReservedCodeCacheSize is equal to > InitialCodeCacheSize. I've also disabled code cache segmentation if the code cache does not at least > contain 8 pages because segmentation would otherwise disable the advantage of huge pages. > > Without my fix, only one 1GB page is used for the Java heap: > > -sh-4.1# base/jdk-12/fastdebug/bin/java -Xmx1g -Xms1g -XX:+AlwaysPreTouch -XX:+UseG1GC > -XX:+UseLargePages -XX:ReservedCodeCacheSize=1G -XX:InitialCodeCacheSize=1G Test & > [1] 1961 > -sh-4.1# grep huge /proc/*/numa_maps > /proc/1961/numa_maps:c0000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 > -sh-4.1# grep Huge /proc/meminfo > HugePages_Total: 20 > HugePages_Free: 19 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 1048576 kB > > With my fix, two 1GB pages are used (one for the heap and one for the code cache): > > -sh-4.1# fix/jdk-12/fastdebug/bin/java -Xmx1g -Xms1g -XX:+AlwaysPreTouch -XX:+UseG1GC > -XX:+UseLargePages -XX:ReservedCodeCacheSize=1G -XX:InitialCodeCacheSize=1G Test &[1] 13813 > -sh-4.1# grep huge /proc/*/numa_maps > /proc/13813/numa_maps:c0000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 > /proc/13813/numa_maps:7f3400000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1 > -sh-4.1# grep Huge /proc/meminfo > HugePages_Total: 20 > HugePages_Free: 18 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 1048576 kB > > If InitialCodeCacheSize is set to the default, the code cache will not use 1GB pages to allow > incremental committing of memory. > > Thanks, > Tobias > From tobias.hartmann at oracle.com Tue Dec 4 17:03:21 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 4 Dec 2018 18:03:21 +0100 Subject: [12] RFR(S): 8208277: Code cache heap (-XX:ReservedCodeCacheSize) doesn't work with 1GB LargePages In-Reply-To: References: Message-ID: <19a850a4-9752-9268-9cc5-6bf21c6e8a38@oracle.com> Hi Vladimir, thanks for the review! On 04.12.18 17:49, Vladimir Kozlov wrote: > Did you tested Segmented CodeCache and huge pages if you use 8Gb size? The maximum ReservedCodeCacheSize is 2G ("Invalid ReservedCodeCacheSize=8192M. Must be at most 2048M") but I've tested with <= 2G and SegmentedCodeCache enabled via the command line (which is still possible). Best regards, Tobias From vladimir.kozlov at oracle.com Tue Dec 4 17:06:35 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 4 Dec 2018 09:06:35 -0800 Subject: [12] RFR(S): 8208277: Code cache heap (-XX:ReservedCodeCacheSize) doesn't work with 1GB LargePages In-Reply-To: <19a850a4-9752-9268-9cc5-6bf21c6e8a38@oracle.com> References: <19a850a4-9752-9268-9cc5-6bf21c6e8a38@oracle.com> Message-ID: <53b782c1-d033-9222-647f-876caf0f5b96@oracle.com> Good. Thanks, Vladimir On 12/4/18 9:03 AM, Tobias Hartmann wrote: > Hi Vladimir, > > thanks for the review! > > On 04.12.18 17:49, Vladimir Kozlov wrote: >> Did you tested Segmented CodeCache and huge pages if you use 8Gb size? > > The maximum ReservedCodeCacheSize is 2G ("Invalid ReservedCodeCacheSize=8192M. Must be at most > 2048M") but I've tested with <= 2G and SegmentedCodeCache enabled via the command line (which is > still possible). > > Best regards, > Tobias > From tobias.hartmann at oracle.com Tue Dec 4 17:08:29 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 4 Dec 2018 18:08:29 +0100 Subject: [12] RFR(S): 8208277: Code cache heap (-XX:ReservedCodeCacheSize) doesn't work with 1GB LargePages In-Reply-To: <53b782c1-d033-9222-647f-876caf0f5b96@oracle.com> References: <19a850a4-9752-9268-9cc5-6bf21c6e8a38@oracle.com> <53b782c1-d033-9222-647f-876caf0f5b96@oracle.com> Message-ID: Thanks Vladimir. Best regards, Tobias On 04.12.18 18:06, Vladimir Kozlov wrote: > Good. > > Thanks, > Vladimir > > On 12/4/18 9:03 AM, Tobias Hartmann wrote: >> Hi Vladimir, >> >> thanks for the review! >> >> On 04.12.18 17:49, Vladimir Kozlov wrote: >>> Did you tested Segmented CodeCache and huge pages if you use 8Gb size? >> >> The maximum ReservedCodeCacheSize is 2G ("Invalid ReservedCodeCacheSize=8192M. Must be at most >> 2048M") but I've tested with <= 2G and SegmentedCodeCache enabled via the command line (which is >> still possible). >> >> Best regards, >> Tobias >> From aph at redhat.com Tue Dec 4 17:16:09 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 4 Dec 2018 17:16:09 +0000 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> Message-ID: <06b204e9-7c1d-2f36-f9ee-7cac443335e9@redhat.com> On 12/4/18 2:41 PM, Aleksey Shipilev wrote: > That is an emerging fancy nomenclature for the interesting corner case in patch difficulty > designation pioneered by those who do not want to guess how many X-es to put in front of XXXXS and > save everybody's time for reading and understanding the actual difficulty as well as implicitly > requesting the confirmation of the aforementioned difficulty from the contributors who can make the > call about patches size and time needed to be spent reading explanations about the patch itself, or > any necessary metadata for the patch, difficulty included. As a penance for writing that I'm going to invite you at the next meeting to read it out without taking a breath. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Tue Dec 4 17:43:15 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 4 Dec 2018 18:43:15 +0100 Subject: RFR (T) 8214787: Zero builds fail with "undefined JavaThread::thread_state()" In-Reply-To: <06b204e9-7c1d-2f36-f9ee-7cac443335e9@redhat.com> References: <7f13804b-f900-e509-953f-535a1bcc574a@redhat.com> <06b204e9-7c1d-2f36-f9ee-7cac443335e9@redhat.com> Message-ID: On Tue, Dec 4, 2018 at 6:16 PM Andrew Haley wrote: > > On 12/4/18 2:41 PM, Aleksey Shipilev wrote: > > That is an emerging fancy nomenclature for the interesting corner case in patch difficulty > > designation pioneered by those who do not want to guess how many X-es to put in front of XXXXS and > > save everybody's time for reading and understanding the actual difficulty as well as implicitly > > requesting the confirmation of the aforementioned difficulty from the contributors who can make the > > call about patches size and time needed to be spent reading explanations about the patch itself, or > > any necessary metadata for the patch, difficulty included. > > As a penance for writing that I'm going to invite you at the next > meeting to read it out without taking a breath. > Please make a recording available. ...Thomas > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From nils.eliasson at oracle.com Tue Dec 4 18:15:52 2018 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Tue, 4 Dec 2018 19:15:52 +0100 Subject: RFR(S): 8214773: Replace use of thread unsafe strtok In-Reply-To: <2bab588c-8e0c-2b14-2314-e803cdcfadf3@oracle.com> References: <4b8cdd06-4a02-298d-c902-58ce0e79bf88@oracle.com> <2bab588c-8e0c-2b14-2314-e803cdcfadf3@oracle.com> Message-ID: <1bf89e75-8aa8-aca5-280c-099f30dd1b61@oracle.com> Hi David, Thanks for your suggestion, I got much better this way: http://cr.openjdk.java.net/~neliasso/8214773/webrev.02/ Regards, Nils On 2018-12-04 13:47, David Holmes wrote: > > > Hi Nils, > > Can you just define one version in os_posix.cpp and one in > os_windows.cpp? > > This per-os function really seems like overkill in this case. You > could just add a Windows specific #define to convert strtok_r to > strtok_s. > > Thanks, > David > > > On 4/12/2018 8:01 pm, Nils Eliasson wrote: >> Hi, >> >> Hotspot has a few uses of the thread unsafe strtok function. Most of >> the them are probably benign since they happen during startup. The >> use in compilerDirectives however has been reported to cause errors >> in user applications that also use strtok though jni. (See bug report >> or mail to compiler list.) >> >> In this patch I replace all uses of strtok. The call need to go >> through the os specific files since windows uses a different function >> (strtok_s) than the rest of the platforms (strtok_r). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214773 >> >> Webrev: http://cr.openjdk.java.net/~neliasso/8214773/webrev.01/ >> >> Thanks to Andre Gillet at Nuance who reported the bug. >> >> Regards, >> >> Nils Eliasson >> From leonid.mesnik at oracle.com Tue Dec 4 20:03:14 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 4 Dec 2018 12:03:14 -0800 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> Hi The shared tests changes looks good for me. Thank you for fixing and testing different combinations. Leonid > On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: > > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> *) shenandoah-gc >>>>> >>>>> - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> *) shared-gc >>>>> >>>>> - This is mostly boilerplate that is common to any GC >>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> *) shared-serviceability >>>>> >>>>> - The usual code to support another GC >>>>> >>>>> *) shared-runtime >>>>> >>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> e.g. ClassLoaderData, CodeCache, etc >>>>> - Warning and disabling JFR LeakProfiler >>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> - Locks initialization in mutexLocker.cpp as usual >>>>> - VM operations defines for Shenandoah's VM ops >>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> - The usual macros in macro.hpp >>>>> >>>>> *) shared-build >>>>> >>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> don't) >>>>> >>>>> *) shared-tests >>>>> >>>>> - Test infrastructure to support Shenandoah >>>>> - Shenandoah test groups >>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> *) shenandoah-tests >>>>> >>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> - {Windows} x {x86_64}, >>>>> - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> - the new Shenandoah tests >>>>> - jcstress with/without aggressive Shenandoah verification >>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > From rkennke at redhat.com Tue Dec 4 20:37:49 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 21:37:49 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> Message-ID: Thanks, Leonid, for reviewing! Roman > Hi > > The shared tests changes looks good for me. Thank you for fixing and testing different combinations. > > Leonid > >> On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: >> >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build (kvn, erikj, ihse, pliden) >> [ ] shared-gc (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests (lmesnik, pliden) >> [ ] shenandoah-gc >> [ ] shenandoah-tests >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> *) shenandoah-gc >>>>>> >>>>>> - Actual Shenandoah implementation, almost completely residing in >>>>>> gc/shenandoah >>>>>> >>>>>> *) shared-gc >>>>>> >>>>>> - This is mostly boilerplate that is common to any GC >>>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>>> fail (next to CMS and G1) >>>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> *) shared-serviceability >>>>>> >>>>>> - The usual code to support another GC >>>>>> >>>>>> *) shared-runtime >>>>>> >>>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>>> hook up with, >>>>>> e.g. ClassLoaderData, CodeCache, etc >>>>>> - Warning and disabling JFR LeakProfiler >>>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> - Locks initialization in mutexLocker.cpp as usual >>>>>> - VM operations defines for Shenandoah's VM ops >>>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> - The usual macros in macro.hpp >>>>>> >>>>>> *) shared-build >>>>>> >>>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>> files, which is >>>>>> useful to get the whole marking loop inlined (observed significant >>>>>> regression if we >>>>>> don't) >>>>>> >>>>>> *) shared-tests >>>>>> >>>>>> - Test infrastructure to support Shenandoah >>>>>> - Shenandoah test groups >>>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>>> run with it >>>>>> >>>>>> *) shenandoah-tests >>>>>> >>>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>> - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> - {Windows} x {x86_64}, >>>>>> - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> - the new Shenandoah tests >>>>>> - jcstress with/without aggressive Shenandoah verification >>>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> > From hohensee at amazon.com Tue Dec 4 21:17:50 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Tue, 4 Dec 2018 21:17:50 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <8c5da4d8-bbec-ee55-3c02-94e261fa8977@redhat.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> <8c5da4d8-bbec-ee55-3c02-94e261fa8977@redhat.com> Message-ID: <378B453C-AA6D-456C-8CA5-E6098A37FFD4@amazon.com> I don't think it's all that difficult to generate oopmaps in asm code. See, e.g., generate_uncommon_trap_blob() in sharedRuntime_x86_64.cpp. But, I agree that we should do as much as we can in Java, so if we're ok with coarse-grained strip-mining, put the strip-mine loop in Java. Paul ?On 11/30/18, 8:39 AM, "Andrew Haley" wrote: On 11/30/18 2:28 PM, Hohensee, Paul wrote: > My bad on the definition of "intrinsic". > > I was thinking of "intrinsic" as a method for which C2 generates specialized IR which would then be subject to optimization (as in library_call.cpp), not hand-coded asm (as in stubGenerator_aarch64.cpp). It's very much the latter. > I don't know how feasible it is to switch from the latter to the former, but imo it would be good to do if we could. Pretty near impossible for the crypto stuff, which relies on magic hand-unrolled software pipelining for maximum performance. > On the maximum time an intrinsic should block, Roland pointed at https://bugs.openjdk.java.net/browse/JDK-8186027. Maybe use the same approach for hand-coded asm, i.e, insert safepoint polls into the hand-coded asm based on LoopStripMiningIter. It'd be hard. We'd need to generate oop maps in the middle of the intrinsics. > Or, are the intrinsics completely separate native methods? If so, your approach sounds good to me. They just call out to the stubs. As a matter of maintainability I believe we should do as much as possible in Java rather than add even more hand-coded stuff. Apart from anything else it'd mean that we wouldn't have to fix this in N different back ends. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From david.holmes at oracle.com Tue Dec 4 21:57:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Dec 2018 07:57:01 +1000 Subject: RFR(S): 8214773: Replace use of thread unsafe strtok In-Reply-To: <1bf89e75-8aa8-aca5-280c-099f30dd1b61@oracle.com> References: <4b8cdd06-4a02-298d-c902-58ce0e79bf88@oracle.com> <2bab588c-8e0c-2b14-2314-e803cdcfadf3@oracle.com> <1bf89e75-8aa8-aca5-280c-099f30dd1b61@oracle.com> Message-ID: Hi Nils, On 5/12/2018 4:15 am, Nils Eliasson wrote: > Hi David, > > Thanks for your suggestion, I got much better this way: > > http://cr.openjdk.java.net/~neliasso/8214773/webrev.02/ That looks good to me! Can you just add a comment in os_windows.hpp before the #define: // strtok_s is the Windows thread-safe equivalent of POSIX strtok_r No need for updated webrev. Thanks, David > Regards, > > Nils > > > On 2018-12-04 13:47, David Holmes wrote: >> >> >> Hi Nils, >> >> Can you just define one version in os_posix.cpp and one in >> os_windows.cpp? >> >> This per-os function really seems like overkill in this case. You >> could just add a Windows specific #define to convert strtok_r to >> strtok_s. >> >> Thanks, >> David >> >> >> On 4/12/2018 8:01 pm, Nils Eliasson wrote: >>> Hi, >>> >>> Hotspot has a few uses of the thread unsafe strtok function. Most of >>> the them are probably benign since they happen during startup. The >>> use in compilerDirectives however has been reported to cause errors >>> in user applications that also use strtok though jni. (See bug report >>> or mail to compiler list.) >>> >>> In this patch I replace all uses of strtok. The call need to go >>> through the os specific files since windows uses a different function >>> (strtok_s) than the rest of the platforms (strtok_r). >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214773 >>> >>> Webrev: http://cr.openjdk.java.net/~neliasso/8214773/webrev.01/ >>> >>> Thanks to Andre Gillet at Nuance who reported the bug. >>> >>> Regards, >>> >>> Nils Eliasson >>> From david.holmes at oracle.com Wed Dec 5 00:31:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Dec 2018 10:31:35 +1000 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> Message-ID: <31cf8b15-4fc4-6f37-aa93-563961c5b427@oracle.com> On 4/12/2018 11:24 pm, Matthias Klose wrote: > with jdk-12+22, bootcycle builds fail at least on x86_64-linux-gnu. Is there > some place where I can check if this kind of build succeeds for others? They don't fail for us, we run them regularly. Cheers, David > thanks, Matthias > From vladimir.x.ivanov at oracle.com Wed Dec 5 01:12:47 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 4 Dec 2018 17:12:47 -0800 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot In-Reply-To: References: <896662e4-fd36-a466-0e4d-2c69c59e8889@oracle.com> Message-ID: <9dc1bddc-ec4d-376b-db65-4c0ee685add0@oracle.com> >>> I just added a query to the bug report. I can't tell what other flags >>> get tested in same run that uses DeoptimizeALot. Is it better to >>> exclude the test from all such runs, or to force DeoptimizeALot off >>> in the @run lines? >> >> The flag is used in hs-comp testing (tier7-8) and it turns on >> heavyweight stress mode. The actual flag combinations being tested are: >> >> ?? -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation >> [-Xcomp] -XX:+DeoptimizeALot >> >> I'm in favor of excluding the test, since I don't see much value in >> running the test with the flag explicitly disabled: it duplicates >> testing in other configurations. > > Which other tier config would this duplicate? As long as we are stress > testing the reachability fence then I'm okay with this, but I want to be > sure we are stress testing it. Ensuring reachability fences are kept in > place and maintain object liveness is exactly what we need to check > under heavy compilation etc. It's hard to precisely enumerate all the configurations where it is executed, but from what I'm seeing the configs: * -XX:+DeoptimizeALot is only part of compiler testing * jdk jtreg tests (part of jtregNonCompiler group) are also executed in tier4 & tier6; From what I'm seeing -Xcomp mode is covered: -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation -Xcomp I don't immediately see whether it's tested in non-Xcomp with low compilation threshold: -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation The test is executed in mixed mode (with default threshold) in non-compiler tiers + promotion testing. My point was more about -XX:+DeoptimizeALot being a dedicated stress mode which is used _in addition_ to other testing modes. Considering the test doesn't work with it, I'm in favor of excluding it (as was proposed) than running it with -XX:-DeoptimizeALot. Best regards, Vladimir Ivanov From david.holmes at oracle.com Wed Dec 5 01:15:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Dec 2018 11:15:23 +1000 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot In-Reply-To: <9dc1bddc-ec4d-376b-db65-4c0ee685add0@oracle.com> References: <896662e4-fd36-a466-0e4d-2c69c59e8889@oracle.com> <9dc1bddc-ec4d-376b-db65-4c0ee685add0@oracle.com> Message-ID: On 5/12/2018 11:12 am, Vladimir Ivanov wrote: > >>>> I just added a query to the bug report. I can't tell what other >>>> flags get tested in same run that uses DeoptimizeALot. Is it better >>>> to exclude the test from all such runs, or to force DeoptimizeALot >>>> off in the @run lines? >>> >>> The flag is used in hs-comp testing (tier7-8) and it turns on >>> heavyweight stress mode. The actual flag combinations being tested are: >>> >>> ?? -ea -esa -XX:CompileThreshold=100 -server >>> -XX:[+-]TieredCompilation [-Xcomp] -XX:+DeoptimizeALot >>> >>> I'm in favor of excluding the test, since I don't see much value in >>> running the test with the flag explicitly disabled: it duplicates >>> testing in other configurations. >> >> Which other tier config would this duplicate? As long as we are stress >> testing the reachability fence then I'm okay with this, but I want to >> be sure we are stress testing it. Ensuring reachability fences are >> kept in place and maintain object liveness is exactly what we need to >> check under heavy compilation etc. > > It's hard to precisely enumerate all the configurations where it is > executed, but from what I'm seeing the configs: > > ? * -XX:+DeoptimizeALot is only part of compiler testing > > ? * jdk jtreg tests (part of jtregNonCompiler group) are also executed > in tier4 & tier6; > > From what I'm seeing -Xcomp mode is covered: > > ?? -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation > -Xcomp > > > I don't immediately see whether it's tested in non-Xcomp with low > compilation threshold: > > ? -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation > > The test is executed in mixed mode (with default threshold) in > non-compiler tiers + promotion testing. > > > My point was more about -XX:+DeoptimizeALot being a dedicated stress > mode which is used _in addition_ to other testing modes. Considering the > test doesn't work with it, I'm in favor of excluding it (as was > proposed) than running it with -XX:-DeoptimizeALot. Thanks for all the info. I just wanted to be sure that excluding the run that sets +DeoptimizeALot didn't also mean we were excluding all/most of the stress testing for this test. Looks good to go. Thanks, David > Best regards, > Vladimir Ivanov From vladimir.x.ivanov at oracle.com Wed Dec 5 01:16:44 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 4 Dec 2018 17:16:44 -0800 Subject: [12] RFR (T): 8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot In-Reply-To: References: <896662e4-fd36-a466-0e4d-2c69c59e8889@oracle.com> <9dc1bddc-ec4d-376b-db65-4c0ee685add0@oracle.com> Message-ID: <4aa3d8cf-9400-bb66-011d-e4eafa2cbdcc@oracle.com> Thanks for review, David & Igor! Best regards, Vladimir Ivanov On 04/12/2018 17:15, David Holmes wrote: > On 5/12/2018 11:12 am, Vladimir Ivanov wrote: >> >>>>> I just added a query to the bug report. I can't tell what other >>>>> flags get tested in same run that uses DeoptimizeALot. Is it better >>>>> to exclude the test from all such runs, or to force DeoptimizeALot >>>>> off in the @run lines? >>>> >>>> The flag is used in hs-comp testing (tier7-8) and it turns on >>>> heavyweight stress mode. The actual flag combinations being tested are: >>>> >>>> ?? -ea -esa -XX:CompileThreshold=100 -server >>>> -XX:[+-]TieredCompilation [-Xcomp] -XX:+DeoptimizeALot >>>> >>>> I'm in favor of excluding the test, since I don't see much value in >>>> running the test with the flag explicitly disabled: it duplicates >>>> testing in other configurations. >>> >>> Which other tier config would this duplicate? As long as we are >>> stress testing the reachability fence then I'm okay with this, but I >>> want to be sure we are stress testing it. Ensuring reachability >>> fences are kept in place and maintain object liveness is exactly what >>> we need to check under heavy compilation etc. >> >> It's hard to precisely enumerate all the configurations where it is >> executed, but from what I'm seeing the configs: >> >> ?? * -XX:+DeoptimizeALot is only part of compiler testing >> >> ?? * jdk jtreg tests (part of jtregNonCompiler group) are also >> executed in tier4 & tier6; >> >> ?From what I'm seeing -Xcomp mode is covered: >> >> ??? -ea -esa -XX:CompileThreshold=100 -server >> -XX:[+-]TieredCompilation -Xcomp >> >> >> I don't immediately see whether it's tested in non-Xcomp with low >> compilation threshold: >> >> ?? -ea -esa -XX:CompileThreshold=100 -server -XX:[+-]TieredCompilation >> >> The test is executed in mixed mode (with default threshold) in >> non-compiler tiers + promotion testing. >> >> >> My point was more about -XX:+DeoptimizeALot being a dedicated stress >> mode which is used _in addition_ to other testing modes. Considering >> the test doesn't work with it, I'm in favor of excluding it (as was >> proposed) than running it with -XX:-DeoptimizeALot. > > Thanks for all the info. I just wanted to be sure that excluding the run > that sets +DeoptimizeALot didn't also mean we were excluding all/most of > the stress testing for this test. > > Looks good to go. > > Thanks, > David > > >> Best regards, >> Vladimir Ivanov From erik.osterlund at oracle.com Wed Dec 5 09:55:30 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 5 Dec 2018 10:55:30 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <7d75efb2-337b-9722-6636-97d1152c656c@oracle.com> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> <7d75efb2-337b-9722-6636-97d1152c656c@oracle.com> Message-ID: <94b7e91d-b99c-6e16-090f-66a50c2fa8e8@oracle.com> Hi, I talked to Rickard offline. He had some suggestions for improvement: 1) Exit early in CompiledMethod::clean_ic_if_metadata_is_dead if the ic is clean. 2) Make a template function for cleaning the reloc entries that do not expect IC cleaning to fail, rather than repeating the same guarantee message. 3) Split out the bodies of the two for loops that restart if we ran out of IC stubs, to separate functions so it becomes easier to read. 4) Add some extra debugging code. I added a counter that is incremented when we fail a transition due to IC stub depletion, and clear it when refilling IC stubs, and assert in the safepoint cleanup code that the counter is cleared. This catches mistakes where a transition failed, but nobody bothered to refill and try again. It actually caught a spot where an IC cache was cleaned without checking if successful in SharedRuntime::reresolve_call_site. So this was a great idea. I also made it explicit when returning from set_to_megamorphic if it failed due to running out of IC stubs or vtable stubs. Only in the case of running out of IC stubs do we need to refill IC stubs. Full webrev: http://cr.openjdk.java.net/~eosterlund/8214338/webrev.02/ Incremental: http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01_02/ Thanks, /Erik On 2018-12-03 16:39, Erik ?sterlund wrote: > Any more takers? I need one more review. I have passed hs-tier1-6 twice > with this now. > > /Erik > > On 2018-11-30 12:19, Erik ?sterlund wrote: >> Hi Dean, >> >> Thanks for reviewing this! >> >> On 2018-11-27 23:55, dean.long at oracle.com wrote: >>> There's a typo in the new guarantee message: "tranisition". >> >> Fixed in-place. >> >>> Regarding the infinite loops, can we detect if progress isn't being >>> made and assert/panic, rather than hanging? >> >> I'm afraid not. It's a similar analogy to the problem of progress >> guarantees of a lock-free algorithm. They have a global progress >> guarantee for the system, but no local guarantee of progress for each >> operation, that can in theory be starved indefinitely due to other >> operations winning every race. The situation is similar here, at least >> in theory, that the thread refilling the IC stubs didn't get a single >> stub back, because another thread started depleted the IC stubs again. >> Naturally, if this was ever to happen, we would get time outs in our >> tests though, to indicate that there is a problem. >> >> Thanks, >> /Erik >> >>> dl >>> >>> On 11/27/18 1:29 PM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>> Thank you for reviewing this. >>>> >>>> Full webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >>>> >>>> Incremental webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >>>> >>>> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>>>> Hi Erik.? Can you explain why you removed the pre-allocated "next >>>>> stub"?? I guess it was no longer necessary?? If so, then you should >>>>> update the "always has sentinel" comment in is_empty. >>>> >>>> Updated. Indeed, the sentinel no longer serves a purpose so I >>>> removed it. Found some more traces of the sentinel that I removed in >>>> the last webrev. >>>> >>>>> In your guarantee messages, some use "should not fail" and some use >>>>> "should never fail".? It's not a big deal but maybe they should be >>>>> the same. >>>> >>>> Sure. Updated. >>>> >>>>> You introduced a couple of infinite loops.? Are we guaranteed to >>>>> exit these loops eventually?? Is there an upper bound on how many >>>>> iterations are needed? >>>> >>>> In my latest webrev I removed some unnecessary set_to_clean() where >>>> IC caches are already is_clean(). With that in place, there should >>>> be global progress guarantees and a single IC stub in the buffer >>>> should be all you really "need". Although, you might want more. In >>>> fact, I'm questioning if the 10K size buffer is big enough today, >>>> but I'll leave that for another day. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> The rest looks good. >>>>> >>>>> dl >>>>> >>>>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> IC stub refilling currently requires a safepoint operation. When >>>>>> done right at the point where an CompiledIC is about to get >>>>>> patched to a transitional state using an IC stub, locks may >>>>>> already be held, causing a bunch of locking issues - especially >>>>>> for concurrent class unloading. Therefore, the IC stub refilling >>>>>> ought to be moved out so that IC cache cleaning and transitioning >>>>>> may be done without any safepoints, and the locks in the path >>>>>> ought to not perform safepoint checking. >>>>>> >>>>>> This is implemented by allowing IC transitions to fail when they >>>>>> require IC stubs, and we run out of them. This propages back to a >>>>>> higher level where IC stubs are refilled after having released the >>>>>> IC locker. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>> >>> From aph at redhat.com Wed Dec 5 10:35:50 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 5 Dec 2018 10:35:50 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <378B453C-AA6D-456C-8CA5-E6098A37FFD4@amazon.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> <8c5da4d8-bbec-ee55-3c02-94e261fa8977@redhat.com> <378B453C-AA6D-456C-8CA5-E6098A37FFD4@amazon.com> Message-ID: <1fc32364-3f7d-d700-228b-b1dde0bb5888@redhat.com> On 12/4/18 9:17 PM, Hohensee, Paul wrote: > I don't think it's all that difficult to generate oopmaps in asm code. See, e.g., generate_uncommon_trap_blob() in sharedRuntime_x86_64.cpp. By "difficult" I mean do it many times, on different ports, co-ordinating the port maintainers. > But, I agree that we >should do as much as we can in Java, so if we're ok with coarse-grained strip-mining, put the strip-mine loop in Java. Right. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rickard.backman at oracle.com Wed Dec 5 10:37:27 2018 From: rickard.backman at oracle.com (Rickard =?utf-8?Q?B=C3=A4ckman?=) Date: Wed, 5 Dec 2018 11:37:27 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <94b7e91d-b99c-6e16-090f-66a50c2fa8e8@oracle.com> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> <7d75efb2-337b-9722-6636-97d1152c656c@oracle.com> <94b7e91d-b99c-6e16-090f-66a50c2fa8e8@oracle.com> Message-ID: <20181205103726.vy5pq4cyfhnasb5v@rbackman> Glad to hear that debugging code actually paid off! Looks good! /R On 12/05, Erik ?sterlund wrote: > Hi, > > I talked to Rickard offline. He had some suggestions for improvement: > > 1) Exit early in CompiledMethod::clean_ic_if_metadata_is_dead if the ic is > clean. > 2) Make a template function for cleaning the reloc entries that do not > expect IC cleaning to fail, rather than repeating the same guarantee > message. > 3) Split out the bodies of the two for loops that restart if we ran out of > IC stubs, to separate functions so it becomes easier to read. > 4) Add some extra debugging code. I added a counter that is incremented when > we fail a transition due to IC stub depletion, and clear it when refilling > IC stubs, and assert in the safepoint cleanup code that the counter is > cleared. This catches mistakes where a transition failed, but nobody > bothered to refill and try again. It actually caught a spot where an IC > cache was cleaned without checking if successful in > SharedRuntime::reresolve_call_site. So this was a great idea. > > I also made it explicit when returning from set_to_megamorphic if it failed > due to running out of IC stubs or vtable stubs. Only in the case of running > out of IC stubs do we need to refill IC stubs. > > Full webrev: > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.02/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01_02/ > > Thanks, > /Erik > > On 2018-12-03 16:39, Erik ?sterlund wrote: > > Any more takers? I need one more review. I have passed hs-tier1-6 twice > > with this now. > > > > /Erik > > > > On 2018-11-30 12:19, Erik ?sterlund wrote: > > > Hi Dean, > > > > > > Thanks for reviewing this! > > > > > > On 2018-11-27 23:55, dean.long at oracle.com wrote: > > > > There's a typo in the new guarantee message: "tranisition". > > > > > > Fixed in-place. > > > > > > > Regarding the infinite loops, can we detect if progress isn't > > > > being made and assert/panic, rather than hanging? > > > > > > I'm afraid not. It's a similar analogy to the problem of progress > > > guarantees of a lock-free algorithm. They have a global progress > > > guarantee for the system, but no local guarantee of progress for > > > each operation, that can in theory be starved indefinitely due to > > > other operations winning every race. The situation is similar here, > > > at least in theory, that the thread refilling the IC stubs didn't > > > get a single stub back, because another thread started depleted the > > > IC stubs again. Naturally, if this was ever to happen, we would get > > > time outs in our tests though, to indicate that there is a problem. > > > > > > Thanks, > > > /Erik > > > > > > > dl > > > > > > > > On 11/27/18 1:29 PM, Erik ?sterlund wrote: > > > > > Hi Dean, > > > > > > > > > > Thank you for reviewing this. > > > > > > > > > > Full webrev: > > > > > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ > > > > > > > > > > Incremental webrev: > > > > > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ > > > > > > > > > > On 2018-11-27 19:07, dean.long at oracle.com wrote: > > > > > > Hi Erik.? Can you explain why you removed the > > > > > > pre-allocated "next stub"?? I guess it was no longer > > > > > > necessary?? If so, then you should update the "always > > > > > > has sentinel" comment in is_empty. > > > > > > > > > > Updated. Indeed, the sentinel no longer serves a purpose so > > > > > I removed it. Found some more traces of the sentinel that I > > > > > removed in the last webrev. > > > > > > > > > > > In your guarantee messages, some use "should not fail" > > > > > > and some use "should never fail".? It's not a big deal > > > > > > but maybe they should be the same. > > > > > > > > > > Sure. Updated. > > > > > > > > > > > You introduced a couple of infinite loops.? Are we > > > > > > guaranteed to exit these loops eventually?? Is there an > > > > > > upper bound on how many iterations are needed? > > > > > > > > > > In my latest webrev I removed some unnecessary > > > > > set_to_clean() where IC caches are already is_clean(). With > > > > > that in place, there should be global progress guarantees > > > > > and a single IC stub in the buffer should be all you really > > > > > "need". Although, you might want more. In fact, I'm > > > > > questioning if the 10K size buffer is big enough today, but > > > > > I'll leave that for another day. > > > > > > > > > > Thanks, > > > > > /Erik > > > > > > > > > > > The rest looks good. > > > > > > > > > > > > dl > > > > > > > > > > > > On 11/27/18 5:00 AM, Erik ?sterlund wrote: > > > > > > > Hi, > > > > > > > > > > > > > > IC stub refilling currently requires a safepoint > > > > > > > operation. When done right at the point where an > > > > > > > CompiledIC is about to get patched to a transitional > > > > > > > state using an IC stub, locks may already be held, > > > > > > > causing a bunch of locking issues - especially for > > > > > > > concurrent class unloading. Therefore, the IC stub > > > > > > > refilling ought to be moved out so that IC cache > > > > > > > cleaning and transitioning may be done without any > > > > > > > safepoints, and the locks in the path ought to not > > > > > > > perform safepoint checking. > > > > > > > > > > > > > > This is implemented by allowing IC transitions to > > > > > > > fail when they require IC stubs, and we run out of > > > > > > > them. This propages back to a higher level where IC > > > > > > > stubs are refilled after having released the IC > > > > > > > locker. > > > > > > > > > > > > > > Webrev: > > > > > > > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ > > > > > > > > > > > > > > Bug: > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8214338 > > > > > > > > > > > > > > Thanks, > > > > > > > /Erik > > > > > > > > > > /R From erik.osterlund at oracle.com Wed Dec 5 10:38:50 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 5 Dec 2018 11:38:50 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <20181205103726.vy5pq4cyfhnasb5v@rbackman> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> <7d75efb2-337b-9722-6636-97d1152c656c@oracle.com> <94b7e91d-b99c-6e16-090f-66a50c2fa8e8@oracle.com> <20181205103726.vy5pq4cyfhnasb5v@rbackman> Message-ID: Hi Rickard, Thanks for the review! /Erik On 2018-12-05 11:37, Rickard B?ckman wrote: > Glad to hear that debugging code actually paid off! > Looks good! > > /R > > On 12/05, Erik ?sterlund wrote: >> Hi, >> >> I talked to Rickard offline. He had some suggestions for improvement: >> >> 1) Exit early in CompiledMethod::clean_ic_if_metadata_is_dead if the ic is >> clean. >> 2) Make a template function for cleaning the reloc entries that do not >> expect IC cleaning to fail, rather than repeating the same guarantee >> message. >> 3) Split out the bodies of the two for loops that restart if we ran out of >> IC stubs, to separate functions so it becomes easier to read. >> 4) Add some extra debugging code. I added a counter that is incremented when >> we fail a transition due to IC stub depletion, and clear it when refilling >> IC stubs, and assert in the safepoint cleanup code that the counter is >> cleared. This catches mistakes where a transition failed, but nobody >> bothered to refill and try again. It actually caught a spot where an IC >> cache was cleaned without checking if successful in >> SharedRuntime::reresolve_call_site. So this was a great idea. >> >> I also made it explicit when returning from set_to_megamorphic if it failed >> due to running out of IC stubs or vtable stubs. Only in the case of running >> out of IC stubs do we need to refill IC stubs. >> >> Full webrev: >> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.02/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01_02/ >> >> Thanks, >> /Erik >> >> On 2018-12-03 16:39, Erik ?sterlund wrote: >>> Any more takers? I need one more review. I have passed hs-tier1-6 twice >>> with this now. >>> >>> /Erik >>> >>> On 2018-11-30 12:19, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>> Thanks for reviewing this! >>>> >>>> On 2018-11-27 23:55, dean.long at oracle.com wrote: >>>>> There's a typo in the new guarantee message: "tranisition". >>>> >>>> Fixed in-place. >>>> >>>>> Regarding the infinite loops, can we detect if progress isn't >>>>> being made and assert/panic, rather than hanging? >>>> >>>> I'm afraid not. It's a similar analogy to the problem of progress >>>> guarantees of a lock-free algorithm. They have a global progress >>>> guarantee for the system, but no local guarantee of progress for >>>> each operation, that can in theory be starved indefinitely due to >>>> other operations winning every race. The situation is similar here, >>>> at least in theory, that the thread refilling the IC stubs didn't >>>> get a single stub back, because another thread started depleted the >>>> IC stubs again. Naturally, if this was ever to happen, we would get >>>> time outs in our tests though, to indicate that there is a problem. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> dl >>>>> >>>>> On 11/27/18 1:29 PM, Erik ?sterlund wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Thank you for reviewing this. >>>>>> >>>>>> Full webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >>>>>> >>>>>> Incremental webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >>>>>> >>>>>> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>>>>>> Hi Erik.? Can you explain why you removed the >>>>>>> pre-allocated "next stub"?? I guess it was no longer >>>>>>> necessary?? If so, then you should update the "always >>>>>>> has sentinel" comment in is_empty. >>>>>> >>>>>> Updated. Indeed, the sentinel no longer serves a purpose so >>>>>> I removed it. Found some more traces of the sentinel that I >>>>>> removed in the last webrev. >>>>>> >>>>>>> In your guarantee messages, some use "should not fail" >>>>>>> and some use "should never fail".? It's not a big deal >>>>>>> but maybe they should be the same. >>>>>> >>>>>> Sure. Updated. >>>>>> >>>>>>> You introduced a couple of infinite loops.? Are we >>>>>>> guaranteed to exit these loops eventually?? Is there an >>>>>>> upper bound on how many iterations are needed? >>>>>> >>>>>> In my latest webrev I removed some unnecessary >>>>>> set_to_clean() where IC caches are already is_clean(). With >>>>>> that in place, there should be global progress guarantees >>>>>> and a single IC stub in the buffer should be all you really >>>>>> "need". Although, you might want more. In fact, I'm >>>>>> questioning if the 10K size buffer is big enough today, but >>>>>> I'll leave that for another day. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> The rest looks good. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> IC stub refilling currently requires a safepoint >>>>>>>> operation. When done right at the point where an >>>>>>>> CompiledIC is about to get patched to a transitional >>>>>>>> state using an IC stub, locks may already be held, >>>>>>>> causing a bunch of locking issues - especially for >>>>>>>> concurrent class unloading. Therefore, the IC stub >>>>>>>> refilling ought to be moved out so that IC cache >>>>>>>> cleaning and transitioning may be done without any >>>>>>>> safepoints, and the locks in the path ought to not >>>>>>>> perform safepoint checking. >>>>>>>> >>>>>>>> This is implemented by allowing IC transitions to >>>>>>>> fail when they require IC stubs, and we run out of >>>>>>>> them. This propages back to a higher level where IC >>>>>>>> stubs are refilled after having released the IC >>>>>>>> locker. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>>>>>> >>>>>>>> Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>>> >>>>> > /R > From thomas.schatzl at oracle.com Wed Dec 5 10:47:54 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Dec 2018 11:47:54 +0100 Subject: RFR (S): 8214850: Rename vm_operations* to vmOperations* Message-ID: <6d0c700f01192c1ad56e8c6ae2dc5f80733e72cd.camel@oracle.com> Hi all, can I have reviews for this change that renames the vm_operations files to vmOperations? This has been requested in the JDK-8214791 review but I did not want to merge these. CR: https://bugs.openjdk.java.net/browse/JDK-8214850 Webrev: http://cr.openjdk.java.net/~tschatzl/8214850/webrev/ Testing: build on all (Oracle) platforms Thanks, Thomas From david.holmes at oracle.com Wed Dec 5 11:08:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Dec 2018 21:08:03 +1000 Subject: RFR (S): 8214850: Rename vm_operations* to vmOperations* In-Reply-To: <6d0c700f01192c1ad56e8c6ae2dc5f80733e72cd.camel@oracle.com> References: <6d0c700f01192c1ad56e8c6ae2dc5f80733e72cd.camel@oracle.com> Message-ID: Hi Thomas, On 5/12/2018 8:47 pm, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that renames the vm_operations > files to vmOperations? > > This has been requested in the JDK-8214791 review but I did not want to > merge these. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8214850 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8214850/webrev/ > Testing: > build on all (Oracle) platforms I'm not a fan of gratuitous renames as it makes it harder to query the history of a file. But this seems to be an accurate renaming so in that sense this is Reviewed. But if we're going to impose a particular naming scheme perhaps we should bite the bullet and bring everything into line? arguments_ext, os_ext, os_perf, vm_version ... Cheers, David > Thanks, > Thomas > > From tobias.hartmann at oracle.com Wed Dec 5 12:47:06 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 5 Dec 2018 13:47:06 +0100 Subject: RFR(S): 8214773: Replace use of thread unsafe strtok In-Reply-To: <1bf89e75-8aa8-aca5-280c-099f30dd1b61@oracle.com> References: <4b8cdd06-4a02-298d-c902-58ce0e79bf88@oracle.com> <2bab588c-8e0c-2b14-2314-e803cdcfadf3@oracle.com> <1bf89e75-8aa8-aca5-280c-099f30dd1b61@oracle.com> Message-ID: <55adb4fc-f40b-f114-e4b5-5413eeaffed9@oracle.com> Hi Nils, looks good to me too. Best regards, Tobias On 04.12.18 19:15, Nils Eliasson wrote: > Hi David, > > Thanks for your suggestion, I got much better this way: > > http://cr.openjdk.java.net/~neliasso/8214773/webrev.02/ > > Regards, > > Nils > > > On 2018-12-04 13:47, David Holmes wrote: >> >> >> Hi Nils, >> >> Can you just define one version in os_posix.cpp and one in os_windows.cpp? >> >> This per-os function really seems like overkill in this case. You could just add a Windows >> specific #define to convert strtok_r to strtok_s. >> >> Thanks, >> David >> >> >> On 4/12/2018 8:01 pm, Nils Eliasson wrote: >>> Hi, >>> >>> Hotspot has a few uses of the thread unsafe strtok function. Most of the them are probably benign >>> since they happen during startup. The use in compilerDirectives however has been reported to >>> cause errors in user applications that also use strtok though jni. (See bug report or mail to >>> compiler list.) >>> >>> In this patch I replace all uses of strtok. The call need to go through the os specific files >>> since windows uses a different function (strtok_s) than the rest of the platforms (strtok_r). >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214773 >>> >>> Webrev: http://cr.openjdk.java.net/~neliasso/8214773/webrev.01/ >>> >>> Thanks to Andre Gillet at Nuance who reported the bug. >>> >>> Regards, >>> >>> Nils Eliasson >>> From coleen.phillimore at oracle.com Wed Dec 5 14:27:22 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 5 Dec 2018 09:27:22 -0500 Subject: RFR (S): 8214850: Rename vm_operations* to vmOperations* In-Reply-To: References: <6d0c700f01192c1ad56e8c6ae2dc5f80733e72cd.camel@oracle.com> Message-ID: <0abe8a03-35c3-1472-4ca5-c3571446b0c4@oracle.com> This change looks good to me.? I type this name wrong every time. On 12/5/18 6:08 AM, David Holmes wrote: > Hi Thomas, > > On 5/12/2018 8:47 pm, Thomas Schatzl wrote: >> Hi all, >> >> ?? can I have reviews for this change that renames the vm_operations >> files to vmOperations? >> >> This has been requested in the JDK-8214791 review but I did not want to >> merge these. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8214850 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8214850/webrev/ >> Testing: >> build on all (Oracle) platforms > > I'm not a fan of gratuitous renames as it makes it harder to query the > history of a file. > hg mv will keep the history of the file moved. > But this seems to be an accurate renaming so in that sense this is > Reviewed. > > But if we're going to impose a particular naming scheme perhaps we > should bite the bullet and bring everything into line? arguments_ext, > os_ext, os_perf, vm_version ... > Yes, eventually, depending on distracting the misnaming is, and how difficult to find the file.? vm_version seems like a candidate. thanks, Coleen > Cheers, > David > >> Thanks, >> ?? Thomas >> >> From jcbeyler at google.com Wed Dec 5 19:36:38 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 5 Dec 2018 11:36:38 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> Message-ID: Hi all, My apologies to having to come back for another review for this change: I ran into a snag when trying to pull the latest changes compared to the base I was working on. I basically forgot that there was an issue with snprintf and that I had solved it via JDK-8213622. Could I have a new review of this webrev: Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 Incremental from the port of webrev.03 that got LGTMs: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ A few comments on this because it took me a while to get things in a state I thought was good: - I had to implement an itoa method, do we have something like that in the test base (remember that JDK-8213622 could not use sprintf due to being in the test code)? - The differences here compared to the one you all reviewed are: - I found that adding to the strlen/memcpy error prone and thought that I would try to make it less so. If you want to compare, I extended the strlen/memcpy with the new format to show you if you prefer [1] - Note that the diff between the "old extended way from [1]" to the webrev.04 can be found in [2] - I added a test to test the exception wrapper in tests :); I'm not sure it is deemed useful or not but helped me assure myself that I was not doing things wrong; you can find the base test file here [3]; should we have this or not? (I know that normally we don't add tests to vmTestbase but thought this might be an exception) Thanks for your help and my apologies for the snag, Jc [1]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html [2]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 [3] http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html On Mon, Dec 3, 2018 at 11:29 PM David Holmes wrote: > Looks fine to me. > > Thanks, > David > > On 4/12/2018 4:04 pm, JC Beyler wrote: > > Hi both, > > > > Thanks for the reviews! Since Serguei did not insist on get_basename, I > > went for get_dirname since the method is a local static method and won't > > have its name start spreading, I think it's ok too. > > > > For the naming of the local variable, the idea initially was to use the > > same name as the local variable for JNIEnv already used to reduce the > > code change. Since I'm now adding the line macro at the end anyway, this > > does not matter anymore so I converged all local variables to "jni". > > > > So, without further ado, here is the new version: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > > > This passes the various tests changed by the webrev on my dev machine. > > > > Let me know what you think, > > Jc > > > > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com > > > > wrote: > > > > On 12/3/18 20:15, Chris Plummer wrote: > > > Hi JC, > > > > > > Overall it looks good. A few naming nits thought: > > > > > > In bi01t001.cpp, why have you declared the > > ExceptionCheckingJniEnvPtr > > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename the > > > method argument passed in from jni to jni_env. > > > > > > Related to this, I also noticed in some files that already are > using > > > ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, > you > > > delcared it as env(jni_env). So that means there are 3 different > > names > > > you have used for the ExceptionCheckingJniEnvPtr local variable. > > They > > > should be consistent. > > > > > > Also, can you rename get_basename() to get_dirname()? I know > Serguei > > > suggested get_basename() a while back, but unless "basename" is > > > commonly used for this purpose, I think "dirname" is more self > > > explanatory. > > > > In general, I'm Okay with get_dirname(). > > Just to mention dirname can be both short or full, so it is a little > > confusing as well. > > It is the reason why the get_basename() was suggested. > > However, I do not insist on get_basename() nor get_full_dirname(). :) > > > > Thanks, > > Serguei > > > > > > > thanks, > > > > > > Chris > > > > > > On 12/2/18 10:29 PM, David Holmes wrote: > > >> Hi Jc, > > >> > > >> I've been lurking on this one and have had a look through. I'm > okay > > >> with the FatalError approach for the tests - we don't expect > > anything > > >> to go wrong in a well written test in a correctly functioning VM. > > >> > > >> Thanks, > > >> David > > >> > > >> > > >> > > >> On 3/12/2018 3:24 pm, JC Beyler wrote: > > >>> Hi all, > > >>> > > >>> Would someone on the GC or runtime team be motivated to give > > this a > > >>> review? :) > > >>> > > >>> It would be much appreciated! > > >>> > > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>> > > >>> Thanks for your help, > > >>> Jc > > >>> > > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > > > >>> >> > wrote: > > >>> > > >>> Hi Chris, > > >>> > > >>> Yes I was waiting for another review since you had > explicitly > > >>> asked :) > > >>> > > >>> And sounds good that when someone from GC or runtime gives a > > >>> review, > > >>> I'll wait for your full review on the webrev.02! > > >>> > > >>> Thanks again for your help, > > >>> Jc > > >>> > > >>> > > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer > > >>> > > >> > > wrote: > > >>> > > >>> Hi JC, > > >>> > > >>> I think it would be good to get a review from the gc or > > runtime > > >>> teams, since this also affects their tests. > > >>> > > >>> Also, once we are settled on this FatalError approach, > > I still > > >>> need to give your webrev-02 a full review. I only > > skimmed over > > >>> parts of it (I did look at all the changes in > webrevo-01). > > >>> > > >>> thanks, > > >>> > > >>> Chris > > >>> > > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com > > > > >>> > > wrote: > > >>>> Hi Jc, > > >>>> > > >>>> I've already reviewed this too. > > >>>> > > >>>> Thanks, > > >>>> Serguei > > >>>> > > >>>> > > >>>> On 11/27/18 06:56, JC Beyler wrote: > > >>>>> Thanks Chris, > > >>>>> > > >>>>> Anybody else motivated to look at this and review it? > :) > > >>>>> Jc > > >>>>> > > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer > > >>>>> > > >> > > >>>>> wrote: > > >>>>> > > >>>>> Hi JC, > > >>>>> > > >>>>> I'm ok with the FatalError approach, but would > > like to > > >>>>> hear opinions from others also. > > >>>>> > > >>>>> thanks, > > >>>>> > > >>>>> Chris > > >>>>> > > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: > > >>>>>> Hi Chris, > > >>>>>> > > >>>>>> Thanks for taking the time to look at it and yes > you > > >>>>>> have raised exactly why the webrev is between two > > >>>>>> worlds: in cases where a fatal error on failure > is > > >>>>>> wanted, should we simplify the code to remove > > the return > > >>>>>> tests since we do them internally? Now that I've > > looked > > >>>>>> around for non-fatal cases, I think the answer > > is yes, > > >>>>>> it simplifies the code while maintaining the > checks. > > >>>>>> > > >>>>>> I looked a bit and it seems that I can't find > > easily a > > >>>>>> case where the test accepts a JNI failure to > > then move > > >>>>>> on. Therefore, perhaps, for now, the fail with a > > Fatal > > >>>>>> is enough and we can work on the tests to clean > > them up? > > >>>>>> > > >>>>>> That means that this is the new webrev with only > > Fatal > > >>>>>> and cleans up the tests so that it is no longer > in > > >>>>>> between two worlds: > > >>>>>> > > >>>>>> Webrev: > > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > > >>>>>> > > >>>>>> Bug: > > https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>>>> > > >>>>>> (This passes testing on my dev machine for all > the > > >>>>>> modified tests) > > >>>>>> > > >>>>>> with the example you provided, it now looks like: > > >>>>>> > > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > >>>>>> > > >>>>>> > > < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > > > >>>>>> > > >>>>>> > > >>>>>> Where it does, to me at least, seem cleaner and > less > > >>>>>> "noisy". > > >>>>>> > > >>>>>> Let me know what you think, > > >>>>>> Jc > > >>>>>> > > >>>>>> > > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer > > >>>>>> > > > >>>>>> > >> wrote: > > >>>>>> > > >>>>>> Hi JC, > > >>>>>> > > >>>>>> Sorry about the delay. I had to go back an > > look at > > >>>>>> the initial 8210842 webrev and RFR thread to > see > > >>>>>> what this was initially all about. > > >>>>>> > > >>>>>> In general the changes look good. > > >>>>>> > > >>>>>> I don't have a good answer to your > > >>>>>> FatalError/NonFatalError question. It makes > > the code > > >>>>>> a lot cleaner to use FatalError, but then it > > is a > > >>>>>> behavior change, and you also need to deal > with > > >>>>>> tests that intentionally induce errors (do > > you have > > >>>>>> an example of that). > > >>>>>> > > >>>>>> In any case, right now your webrev seems to > be > > >>>>>> between two worlds. You are producing > > FatalError, > > >>>>>> but still checking results. Here's a good > > example: > > >>>>>> > > >>>>>> > > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > >>>>>> > > >>>>>> > > < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > > > >>>>>> > > >>>>>> > > >>>>>> I'm not sure if this is just a temporary > > state until > > >>>>>> it was decided which approach to take. > > >>>>>> > > >>>>>> thanks, > > >>>>>> > > >>>>>> Chris > > >>>>>> > > >>>>>> > > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: > > >>>>>>> Hi all, > > >>>>>>> > > >>>>>>> Chris thought it made sense to have more > > eyes on > > >>>>>>> this change than just serviceability as it > will > > >>>>>>> modify to tests that are not only > > serviceability > > >>>>>>> tests so I've moved this to conversation > > here :) > > >>>>>>> > > >>>>>>> For convenience, I've copy-pasted the > > initial RFR: > > >>>>>>> > > >>>>>>> Could I have a review for the extension and > > usage > > >>>>>>> of the ExceptionJniWrapper. This adds lines > and > > >>>>>>> filenames to the end of the wrapper JNI > > methods, > > >>>>>>> adds tracing, and throws an error if need > > be. I've > > >>>>>>> ported the gc/lock files to use the new > > >>>>>>> TRACE_JNI_CALL add-on and I've ported a few > > of the > > >>>>>>> tests that were already changed for the > > assignment > > >>>>>>> webrev for JDK-8212884. > > >>>>>>> > > >>>>>>> Webrev: > > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 > > >>>>>>> > > >>>>>>> Bug: > > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>>>>> > > >>>>>>> For illustration, if I force an error to the > > >>>>>>> AP04/ap04t03 test and set the verbosity on, > > I get > > >>>>>>> something like: > > >>>>>>> > > >>>>>>> >> Calling JNI method FindClass from > > >>>>>>> ap04t003.cpp:343 > > >>>>>>> >> Calling with these parameter(s): > > >>>>>>> java/lang/Threadd > > >>>>>>> Wait for thread to finish > > >>>>>>> << Called JNI method FindClass from > > >>>>>>> ap04t003.cpp:343 > > >>>>>>> Exception in thread "Thread-0" > > >>>>>>> java.lang.NoClassDefFoundError: > > java/lang/Threadd > > >>>>>>> at > > >>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>>>>> > > >>>>>>> Method) > > >>>>>>> at > > >>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > > > >>>>>>> > > >>>>>>> at > > >>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > > > >>>>>>> > > >>>>>>> Caused by: java.lang.ClassNotFoundException: > > >>>>>>> java.lang.Threadd > > >>>>>>> at > > >>>>>>> > > > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > > > >>>>>>> > > >>>>>>> at > > >>>>>>> > > > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > > > >>>>>>> > > >>>>>>> at > > >>>>>>> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > > >>>>>>> ... 3 more > > >>>>>>> FATAL ERROR in native method: JNI method > > FindClass > > >>>>>>> : internal error from ap04t003.cpp:343 > > >>>>>>> at > > >>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>>>>> > > >>>>>>> Method) > > >>>>>>> at > > >>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > > > >>>>>>> > > >>>>>>> at > > >>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > > > >>>>>>> > > >>>>>>> > > >>>>>>> Questions/comments I have about this are: > > >>>>>>> - Do we want to force fatal errors when a > JNI > > >>>>>>> call fails in general? Most of these tests > > do the > > >>>>>>> right thing and test the return of the JNI > > calls, > > >>>>>>> for example: > > >>>>>>> thrClass = > > jni->FindClass("java/lang/Threadd", > > >>>>>>> TRACE_JNI_CALL); > > >>>>>>> if (thrClass == NULL) { > > >>>>>>> > > >>>>>>> but now the wrapper actually would do a > > fatal if > > >>>>>>> the FindClass call would return a nullptr, > > so we > > >>>>>>> could remove that test altogether. What do > you > > >>>>>>> think? > > >>>>>>> - I prefer to leave them as the tests > then > > >>>>>>> become closer to what real users would have > in > > >>>>>>> their code and is the "recommended" way of > > doing it > > >>>>>>> > > >>>>>>> - The alternative is to use the > > NonFatalError I > > >>>>>>> added which then just prints out that > something > > >>>>>>> went wrong, letting the test continue. > Question > > >>>>>>> will be what should be the default? The > > fatal or > > >>>>>>> the non-fatal error handling? > > >>>>>>> > > >>>>>>> On a different subject: > > >>>>>>> - On the new tests, I've removed the > > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper > > handles the > > >>>>>>> tracing and the verify in almost the same > > way; only > > >>>>>>> difference I can really tell is that the > > complain > > >>>>>>> method from NSK has a max complain before > > stopping > > >>>>>>> to "complain"; I have not added that part > > of the > > >>>>>>> code in this webrev > > >>>>>>> > > >>>>>>> Once we decide on these, I can continue on > the > > >>>>>>> files from JDK-8212884 and then do both the > > >>>>>>> assignment in an if extraction followed-by > this > > >>>>>>> type of webrev in an easier fashion. > > Depending on > > >>>>>>> decisions here, NSK*VERIFY can be > deprecated as > > >>>>>>> well as we go forward. > > >>>>>>> > > >>>>>>> Thanks! > > >>>>>>> Jc > > >>>>>>> > > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris > Plummer > > >>>>>>> > > > >>>>>>> > >> wrote: > > >>>>>>> > > >>>>>>> On 11/19/18 10:07 AM, JC Beyler wrote: > > >>>>>>>> Hi all, > > >>>>>>>> > > >>>>>>>> @David/Chris: should I then push this > > RFR to > > >>>>>>>> the hotspot mailing or the runtime > > one? For > > >>>>>>>> what it's worth, a lot of the tests > > under the > > >>>>>>>> vmTestbase are jvmti so the review also > > >>>>>>>> affects serviceability; it just turns > > out I > > >>>>>>>> started with the GC originally and > > then hit > > >>>>>>>> some other tests I had touched via the > > >>>>>>>> assignment extraction. > > >>>>>>> I think hotspot would be best. > > >>>>>>> > > >>>>>>> Chris > > >>>>>>>> > > >>>>>>>> @Serguei: Done for the method > > renaming, for > > >>>>>>>> the indent, are you talking about > > going from > > >>>>>>>> the 8-indent to 4-indent? If so, would > > it not > > >>>>>>>> just be better to do a new JBS bug and > > do the > > >>>>>>>> whole files in one go? I ask because > > >>>>>>>> otherwise, it will look a bit weird to > > have > > >>>>>>>> parts of the file as 8-indent and > others > > >>>>>>>> 4-indent? > > >>>>>>>> > > >>>>>>>> Thanks for looking at it! > > >>>>>>>> Jc > > >>>>>>>> > > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM > > >>>>>>>> serguei.spitsyn at oracle.com serguei.spitsyn at oracle.com> > > >>>>>>>> > > > > >>>>>>>> > > > >>>>>>>> > >> wrote: > > >>>>>>>> > > >>>>>>>> Hi Jc, > > >>>>>>>> > > >>>>>>>> We have to start this review > > anyway. :) > > >>>>>>>> It looks good to me in general. > > >>>>>>>> Thank you for your consistency in > this > > >>>>>>>> refactoring! > > >>>>>>>> > > >>>>>>>> Some minor comments. > > >>>>>>>> > > >>>>>>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > > > > >>>>>>>> > > >>>>>>>> +static const char* > > remove_folders(const > > >>>>>>>> char* fullname) { I'd suggest to > > rename > > >>>>>>>> the function name to something > > traditional > > >>>>>>>> like get_basename. Otherwise, it > > sounds > > >>>>>>>> like this function has to really > > remove > > >>>>>>>> folders. :) Also, all *Locker.cpp > have > > >>>>>>>> wrong indent in the bodies of if > > and while > > >>>>>>>> statements. Could this be fixed > > with the > > >>>>>>>> refactoring? I did not look on how > > this > > >>>>>>>> impacts the tests other than > > >>>>>>>> serviceability. Thanks, Serguei > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On 11/16/18 19:43, JC Beyler wrote: > > >>>>>>>>> Hi all, > > >>>>>>>>> > > >>>>>>>>> Anybody motivated to review this? > :) > > >>>>>>>>> Jc > > >>>>>>>>> > > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 PM JC > > Beyler > > >>>>>>>>> > > > >>>>>>>>> > >> wrote: > > >>>>>>>>> > > >>>>>>>>> Hi all, > > >>>>>>>>> > > >>>>>>>>> Could I have a review for the > > >>>>>>>>> extension and usage of the > > >>>>>>>>> ExceptionJniWrapper. This > > adds lines > > >>>>>>>>> and filenames to the end of > the > > >>>>>>>>> wrapper JNI methods, adds > > tracing, > > >>>>>>>>> and throws an error if need > > be. I've > > >>>>>>>>> ported the gc/lock files to > > use the > > >>>>>>>>> new TRACE_JNI_CALL add-on and > > I've > > >>>>>>>>> ported a few of the tests > > that were > > >>>>>>>>> already changed for the > > assignment > > >>>>>>>>> webrev for JDK-8212884. > > >>>>>>>>> > > >>>>>>>>> Webrev: > > >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ > > >>>>>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/> > > >>>>>>>>> Bug: > > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>>>>>>> > > >>>>>>>>> For illustration, if I force > > an error > > >>>>>>>>> to the AP04/ap04t03 test and > > set the > > >>>>>>>>> verbosity on, I get something > > like: > > >>>>>>>>> > > >>>>>>>>> >> Calling JNI method > > FindClass from > > >>>>>>>>> ap04t003.cpp:343 > > >>>>>>>>> >> Calling with these > > parameter(s): > > >>>>>>>>> java/lang/Threadd > > >>>>>>>>> Wait for thread to finish > > >>>>>>>>> << Called JNI method > > FindClass from > > >>>>>>>>> ap04t003.cpp:343 > > >>>>>>>>> Exception in thread "Thread-0" > > >>>>>>>>> java.lang.NoClassDefFoundError: > > >>>>>>>>> java/lang/Threadd > > >>>>>>>>> at > > >>>>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>>>>>>> > > >>>>>>>>> Method) > > >>>>>>>>> at > > >>>>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > > > >>>>>>>>> > > >>>>>>>>> at > > >>>>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > > > >>>>>>>>> > > >>>>>>>>> Caused by: > > >>>>>>>>> java.lang.ClassNotFoundException: > > >>>>>>>>> java.lang.Threadd > > >>>>>>>>> at > > >>>>>>>>> > > > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > > > >>>>>>>>> > > >>>>>>>>> at > > >>>>>>>>> > > > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > > > >>>>>>>>> > > >>>>>>>>> at > > >>>>>>>>> > > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > > >>>>>>>>> ... 3 more > > >>>>>>>>> FATAL ERROR in native method: > JNI > > >>>>>>>>> method FindClass : internal > error > > >>>>>>>>> from ap04t003.cpp:343 > > >>>>>>>>> at > > >>>>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>>>>>>> > > >>>>>>>>> Method) > > >>>>>>>>> at > > >>>>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > > > >>>>>>>>> > > >>>>>>>>> at > > >>>>>>>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> Questions/comments I have > about > > >>>>>>>>> this are: > > >>>>>>>>> - Do we want to force fatal > > errors > > >>>>>>>>> when a JNI call fails in > general? > > >>>>>>>>> Most of these tests do the > right > > >>>>>>>>> thing and test the return of > > the JNI > > >>>>>>>>> calls, for example: > > >>>>>>>>> thrClass = > > >>>>>>>>> jni->FindClass("java/lang/Threadd", > > >>>>>>>>> TRACE_JNI_CALL); > > >>>>>>>>> if (thrClass == NULL) { > > >>>>>>>>> > > >>>>>>>>> but now the wrapper actually > > would do > > >>>>>>>>> a fatal if the FindClass call > > would > > >>>>>>>>> return a nullptr, so we could > > remove > > >>>>>>>>> that test altogether. What do > > you > > >>>>>>>>> think? > > >>>>>>>>> - I prefer to leave them > > as the > > >>>>>>>>> tests then become closer to > > what real > > >>>>>>>>> users would have in their > > code and is > > >>>>>>>>> the "recommended" way of > doing it > > >>>>>>>>> > > >>>>>>>>> - The alternative is to > > use the > > >>>>>>>>> NonFatalError I added which > > then just > > >>>>>>>>> prints out that something > > went wrong, > > >>>>>>>>> letting the test continue. > > Question > > >>>>>>>>> will be what should be the > > default? > > >>>>>>>>> The fatal or the non-fatal > error > > >>>>>>>>> handling? > > >>>>>>>>> > > >>>>>>>>> On a different subject: > > >>>>>>>>> - On the new tests, I've > > removed > > >>>>>>>>> the NSK_JNI_VERIFY since the > JNI > > >>>>>>>>> wrapper handles the tracing > > and the > > >>>>>>>>> verify in almost the same > > way; only > > >>>>>>>>> difference I can really tell > > is that > > >>>>>>>>> the complain method from NSK > > has a > > >>>>>>>>> max complain before stopping > to > > >>>>>>>>> "complain"; I have not added > that > > >>>>>>>>> part of the code in this > webrev > > >>>>>>>>> > > >>>>>>>>> Once we decide on these, I can > > >>>>>>>>> continue on the files from > > >>>>>>>>> JDK-8212884 and then do both > the > > >>>>>>>>> assignment in an if extraction > > >>>>>>>>> followed-by this type of > > webrev in an > > >>>>>>>>> easier fashion. Depending on > > >>>>>>>>> decisions here, NSK*VERIFY > can be > > >>>>>>>>> deprecated as well as we go > > forward. > > >>>>>>>>> > > >>>>>>>>> Thank you for the > > reviews/comments :) > > >>>>>>>>> Jc > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> -- > > >>>>>>>>> Thanks, > > >>>>>>>>> Jc > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> -- > > >>>>>>>> Thanks, > > >>>>>>>> Jc > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> Thanks, > > >>>>>>> Jc > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> Thanks, > > >>>>>> Jc > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> Thanks, > > >>>>> Jc > > >>>> > > >>> > > >>> > > >>> > > >>> -- > > >>> Thanks, > > >>> Jc > > >>> > > >>> > > >>> > > >>> -- > > >>> > > >>> Thanks, > > >>> Jc > > > > > > > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc From david.holmes at oracle.com Wed Dec 5 21:51:24 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Dec 2018 07:51:24 +1000 Subject: RFR (S): 8214850: Rename vm_operations* to vmOperations* In-Reply-To: <0abe8a03-35c3-1472-4ca5-c3571446b0c4@oracle.com> References: <6d0c700f01192c1ad56e8c6ae2dc5f80733e72cd.camel@oracle.com> <0abe8a03-35c3-1472-4ca5-c3571446b0c4@oracle.com> Message-ID: On 6/12/2018 12:27 am, coleen.phillimore at oracle.com wrote: > > This change looks good to me.? I type this name wrong every time. > On 12/5/18 6:08 AM, David Holmes wrote: >> Hi Thomas, >> >> On 5/12/2018 8:47 pm, Thomas Schatzl wrote: >>> Hi all, >>> >>> ?? can I have reviews for this change that renames the vm_operations >>> files to vmOperations? >>> >>> This has been requested in the JDK-8214791 review but I did not want to >>> merge these. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8214850 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8214850/webrev/ >>> Testing: >>> build on all (Oracle) platforms >> >> I'm not a fan of gratuitous renames as it makes it harder to query the >> history of a file. >> > > hg mv will keep the history of the file moved. Yes but you have to tell hg to follow across the rename; and graphical tools like TortoiseHg don't seem to follow; and if you only know the old name of the file (eg from before the module system) I have yet to determine how to get hg or Tortoise to show it to me. Cheers, David > >> But this seems to be an accurate renaming so in that sense this is >> Reviewed. >> >> But if we're going to impose a particular naming scheme perhaps we >> should bite the bullet and bring everything into line? arguments_ext, >> os_ext, os_perf, vm_version ... >> > > Yes, eventually, depending on distracting the misnaming is, and how > difficult to find the file.? vm_version seems like a candidate. > > thanks, > Coleen >> Cheers, >> David >> >>> Thanks, >>> ?? Thomas >>> >>> > From gnu.andrew at redhat.com Thu Dec 6 04:29:55 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Thu, 6 Dec 2018 04:29:55 +0000 Subject: [aarch64-port-dev ] RFR: AArch64: 8209415: Fix JVMTI test failure HS202 In-Reply-To: References: <5f2fd3f3-de58-a352-14f7-0d33f92371f5@redhat.com> Message-ID: On Tue, 13 Nov 2018 at 17:23, Andrew Dinn wrote: > > On 13/11/2018 16:40, Andrew Haley wrote: > > In a moment of madness one of us (no names, no pack drill!) managed > > to translate from x86 > > > > __ cmpb(Address(rbcp, 0), Bytecodes::_invokestatic); > > __ jcc(Assembler::notEqual, L_done); > > > > to > > > > __ ldrb(rscratch1, Address(rbcp, 0)); > > __ cmpw(r1, Bytecodes::_invokestatic); > > __ br(Assembler::EQ, L_done); > > > > Two bugs in thee lines... fixed thusly. > > > > http://cr.openjdk.java.net/~aph/8209415/ > > > > OK for trunk and jdk8u ? > Oh my goodness, that's a doozy. Amazing that it lay there for so long > (the author seems to have checked it in in 2015 :-P ) > > Patch is good! > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander Yep, it's there in 7 too: https://icedtea.classpath.org/hg/release/icedtea7-forest-2.6/hotspot/file/tip/src/cpu/aarch64/vm/templateInterpreter_aarch64.cpp#l1885 I'll get it fixed there as well: https://icedtea.classpath.org/bugzilla/show_bug.cgi?id=3669. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From merkel05 at gmail.com Thu Dec 6 07:37:20 2018 From: merkel05 at gmail.com (Sergey) Date: Thu, 6 Dec 2018 08:37:20 +0100 Subject: Fwd: [PATCH][TESTBUG] JDK-8214052: Wrong shell used In-Reply-To: References: Message-ID: Hi, I've posted the original mail to the wrong mailing list and thanks to David for guidance. Problem goes as follows: as stated in the bug description [1] there are few places in the tests where extended bash syntax is being used. First one is (as per ticket in the headline): - vmTestbase/vm/compiler/CodeCacheInfoOnCompilation I've made a quick fix for that one and patch could be found inlined below. Moreover what I've noticed is that grep gives me another test script: - GeneratePropertyPassword.sh [2] that seems to be linked with another ticket [3] which is closed, though it still uses the bash "[[" extension. Perhaps this [4] fix needs to be reapplied. In regards to the patch below, please let me know if I've missed something. Changes were successfully tested with `dash v0.5.10.2-1`. [1] https://bugs.openjdk.java.net/browse/JDK-8214052 [2] http://hg.openjdk.java.net/jdk/sandbox/file/413c28945e0f/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh#l37 [3] https://bugs.openjdk.java.net/browse/JDK-8025886 [4] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/a2551e12a9ea0.5.10.2-1 Regards, su - diff --git a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh --- a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh +++ b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh @@ -27,7 +27,7 @@ ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) isComp=$( echo ${ver} | grep -c "compiled mode") -if [[ $isComp != 1 ]]; then +if [ "$isComp" -ne 1 ]; then echo "skipped. This test works only with -Xcomp" exit fi @@ -40,7 +40,7 @@ res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -XX:+PrintCodeCacheOnCompilation -XX:-Inline vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce "${pattern}") echo "res: " ${res} -if (( "${res}" != "0" )); then +if [ "$res" != "0" ]; then echo "passed" true else From matthias.baesken at sap.com Thu Dec 6 10:36:13 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 6 Dec 2018 10:36:13 +0000 Subject: RFR: 8214944: replace strerror by os::strerror Message-ID: Hello , please review this small fix . Hotspot still has a few uses of the thread unsafe strerror function. Most places where strerror has been used previously, were already replaced by os::strerror (using a errno_to_string error text array). But a few places remained and should also use better use os:strerror. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8214944 http://cr.openjdk.java.net/~mbaesken/webrevs/8214944.0/ Thanks, Matthias From nils.eliasson at oracle.com Thu Dec 6 11:20:12 2018 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Thu, 6 Dec 2018 12:20:12 +0100 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: References: Message-ID: <80a39b43-50ce-24f4-592a-52a90c2b28bd@oracle.com> Looks great! Thanks for fixing. - reviewed. Regards, Nils On 2018-12-06 11:36, Baesken, Matthias wrote: > Hello , please review this small fix . > > Hotspot still has a few uses of the thread unsafe strerror function. > Most places where strerror has been used previously, were already replaced by os::strerror (using a errno_to_string error text array). > > But a few places remained and should also use better use os:strerror. > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8214944 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8214944.0/ > > > Thanks, Matthias From thomas.schatzl at oracle.com Thu Dec 6 12:56:31 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 06 Dec 2018 13:56:31 +0100 Subject: RFR (S): 8214850: Rename vm_operations* to vmOperations* In-Reply-To: <0abe8a03-35c3-1472-4ca5-c3571446b0c4@oracle.com> References: <6d0c700f01192c1ad56e8c6ae2dc5f80733e72cd.camel@oracle.com> <0abe8a03-35c3-1472-4ca5-c3571446b0c4@oracle.com> Message-ID: <0bfb0c79d4228b846591ba7dac69139e30c84f4d.camel@oracle.com> Hi Coleen, David, On Wed, 2018-12-05 at 09:27 -0500, coleen.phillimore at oracle.com wrote: > This change looks good to me. I type this name wrong every time. > > On 12/5/18 6:08 AM, David Holmes wrote: > > Hi Thomas, > > > > On 5/12/2018 8:47 pm, Thomas Schatzl wrote: > > > Hi all, > > > > > > can I have reviews for this change that renames the > > > vm_operations files to vmOperations? > > > > > > This has been requested in the JDK-8214791 review but I did not > > > want to merge these. > > > > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8214850 > > > Webrev: > > > http://cr.openjdk.java.net/~tschatzl/8214850/webrev/ > > > Testing: > > > build on all (Oracle) platforms > > > > I'm not a fan of gratuitous renames as it makes it harder to query > > the history of a file. > > > > hg mv will keep the history of the file moved. > > > > But this seems to be an accurate renaming so in that sense this is > > Reviewed. Thanks for your reviews. Thomas From david.holmes at oracle.com Thu Dec 6 12:56:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Dec 2018 22:56:51 +1000 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: References: Message-ID: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> Hi Matthias, On 6/12/2018 8:36 pm, Baesken, Matthias wrote: > Hello , please review this small fix . > > Hotspot still has a few uses of the thread unsafe strerror function. > Most places where strerror has been used previously, were already replaced by os::strerror (using a errno_to_string error text array). > > But a few places remained and should also use better use os:strerror. It's not that they "remain" but were added later. Previous strerror replacement was done under JDK-8148425. The new uses in OS-specific get_mtime functions were added in a few weeks later by JDK-8146879. The ZGC use is obviously much more recent. Pity there is no way to detect when we use something like that which we shouldn't. Fix looks good. Thanks, David ----- > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8214944 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8214944.0/ > > > Thanks, Matthias > From matthias.baesken at sap.com Thu Dec 6 13:19:52 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 6 Dec 2018 13:19:52 +0000 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> References: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> Message-ID: Hi David and Nils, thanks for the reviews . > > Pity there is no way to detect when we use something like that which we > shouldn't. > Seems VS2017 has compiler warnings for this , at least https://github.com/fmtlib/fmt/issues/703 suggests this . See the example output : format.cc:172:17: warning: 'strerror' is deprecated: This function or variable may be unsafe. Consider using strerror_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [-Wdeprecated-declarations] buffer_ = strerror(error_code_); ^ C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\string.h:180:16: note: 'strerror' has been explicitly marked deprecated here _Check_return_ _CRT_INSECURE_DEPRECATE(strerror_s) ^ Might be we do not have these warnings enabled for some reason . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 6. Dezember 2018 13:57 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: RFR: 8214944: replace strerror by os::strerror > > Hi Matthias, > > On 6/12/2018 8:36 pm, Baesken, Matthias wrote: > > Hello , please review this small fix . > > > > Hotspot still has a few uses of the thread unsafe strerror function. > > Most places where strerror has been used previously, were already > replaced by os::strerror (using a errno_to_string error text array). > > > > But a few places remained and should also use better use os:strerror. > > It's not that they "remain" but were added later. Previous strerror > replacement was done under JDK-8148425. The new uses in OS-specific > get_mtime functions were added in a few weeks later by JDK-8146879. The > ZGC use is obviously much more recent. > > Pity there is no way to detect when we use something like that which we > shouldn't. > > Fix looks good. > > Thanks, > David > ----- > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8214944 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8214944.0/ > > > > > > Thanks, Matthias > > From coleen.phillimore at oracle.com Thu Dec 6 13:47:53 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Dec 2018 08:47:53 -0500 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> Message-ID: <496b3ac5-defc-92b6-7339-64c88c2e2dfa@oracle.com> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/shenandoah-gc/src/hotspot/share/gc/shenandoah/vm_operations_shenandoah.cpp.html Can you rename these to shenandoahVMOperations.hpp/cpp to match the newly agreed upon naming convention for this? See 8214791: Consistently name gc files containing VM operations [Was: Re: RFR (S): 8214791: Rename vm_operations_g1* files to g1VMOperations*] thanks, Coleen On 12/4/18 3:37 PM, Roman Kennke wrote: > Thanks, Leonid, for reviewing! > > Roman > > >> Hi >> >> The shared tests changes looks good for me. Thank you for fixing and testing different combinations. >> >> Leonid >> >>> On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: >>> >>> Round 5 of Shenandoah review includes: >>> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >>> correct now. We believe the CMS @requires was also not quite right and >>> fixed it the same. >>> >>> It reads now: Don't run this test if: >>> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >>> true, as set by harness >>> - Actual GC set by harness is Shenandoah *and* >>> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >>> default in Shenandoah, so this needs to be double-inverteed). >>> >>> The @requires for CMS was wrong before (we think), because it would also >>> filter defaultGC + ExplicitGCInvokesConcurrent. >>> >>> - Sorting of macros was fixed, as was pointed out by Per >>> - Some stuff was added to SA, as suggested by Jini >>> - Rebased on most current jdk/jdk code >>> >>> Webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >>> >>> I also need reviews from GC reviewers for the CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8214349 >>> >>> I already got reviews for: >>> [x] shared-runtime (coleenp) >>> [x] shared-compiler (kvn) >>> >>> I got reviews for shared-build, but an earlier version, so maybe makes >>> sense to look over this again. Erik J, Magnus? >>> >>> I still need approvals for: >>> [ ] shared-build (kvn, erikj, ihse, pliden) >>> [ ] shared-gc (pliden, kbarrett) >>> [ ] shared-serviceability (jgeorge, pliden) >>> [ ] shared-tests (lmesnik, pliden) >>> [ ] shenandoah-gc >>> [ ] shenandoah-tests >>> >>> >>> Thanks for your patience and ongoing support! >>> >>> Cheers, >>> Roman >>> >>> >>>> Hi all, >>>> >>>> here comes round 4 of Shenandoah upstreaming review: >>>> >>>> This includes fixes for the issues that Per brought up: >>>> - Verify and gracefully reject dangerous flags combinations that >>>> disables required barriers >>>> - Revisited @requires filters in tests >>>> - Trim unused code from Shenandoah's SA impl >>>> - Move ShenandoahGCTracer to gc/shenandoah >>>> - Fix ordering of GC names in various files >>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>> >>>> Thanks everybody for taking time to review this! >>>> Roman >>>> >>>>> Hello all, >>>>> >>>>> Thanks so far for all the reviews and support! >>>>> >>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>>> >>>>> Things we've changed today: >>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>> requires an addition in build machinery though, see >>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>> - Improved zero-disabling and build-code-simplification as suggested by >>>>> Magnus and Per >>>>> - Cleaned up some leftovers in C2 >>>>> - Improved C2 loop opts code by introducing another APIs in >>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>>> that. >>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>>> - Rebased on jdk-12+22 >>>>> >>>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>>> Alright, we fixed: >>>>>> - The minor issues that Kim reported in shared-gc >>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>> >>>>>> Some notes: >>>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>> >>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>> >>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>> those with ZGC? >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>> >>>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>>> next round). >>>>>> >>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>> the code and see what is ok, and especially what is not ok, so that we >>>>>> can focus our efforts on the relevant parts? >>>>>> >>>>>> Updated set of webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>> mainline. >>>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>> changes and >>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>>> eventually >>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>> >>>>>>> JEP: >>>>>>> https://openjdk.java.net/jeps/189 >>>>>>> Bug entry: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>> >>>>>>> Webrevs: >>>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>> >>>>>>> For those who want to see the full change, have a look at the >>>>>>> shenandoah-complete >>>>>>> >>>>>>> directory, >>>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>>> shenandoah-master.patch >>>>>>> , >>>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>>> 'shenandoah-complete' webrev). >>>>>>> >>>>>>> Sections to review (at this point) are the following: >>>>>>> *) shenandoah-gc >>>>>>> >>>>>>> - Actual Shenandoah implementation, almost completely residing in >>>>>>> gc/shenandoah >>>>>>> >>>>>>> *) shared-gc >>>>>>> >>>>>>> - This is mostly boilerplate that is common to any GC >>>>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>>>> fail (next to CMS and G1) >>>>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>>>> >>>>>>> *) shared-serviceability >>>>>>> >>>>>>> - The usual code to support another GC >>>>>>> >>>>>>> *) shared-runtime >>>>>>> >>>>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>>>> hook up with, >>>>>>> e.g. ClassLoaderData, CodeCache, etc >>>>>>> - Warning and disabling JFR LeakProfiler >>>>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>> Shenandoah C2 optimizations >>>>>>> - Locks initialization in mutexLocker.cpp as usual >>>>>>> - VM operations defines for Shenandoah's VM ops >>>>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>> Shenandoah's logging >>>>>>> - The usual macros in macro.hpp >>>>>>> >>>>>>> *) shared-build >>>>>>> >>>>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>>>> Vladimir K. beforehand >>>>>>> - Some flags for shenandoah-enabled compilation to get >>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>> Shenandoah's barriers >>>>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>>> files, which is >>>>>>> useful to get the whole marking loop inlined (observed significant >>>>>>> regression if we >>>>>>> don't) >>>>>>> >>>>>>> *) shared-tests >>>>>>> >>>>>>> - Test infrastructure to support Shenandoah >>>>>>> - Shenandoah test groups >>>>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>>>> run with it >>>>>>> >>>>>>> *) shenandoah-tests >>>>>>> >>>>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>>> - A couple of tests configurations have been added, e.g. >>>>>>> TestGCBasherWithShenandoah.java >>>>>>> >>>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>>> work left to do >>>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>>> want to take >>>>>>> a peek at it. >>>>>>> >>>>>>> We have regular builds on: >>>>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>> - {Windows} x {x86_64}, >>>>>>> - {MacOS X} x {x86_64} >>>>>>> >>>>>>> This also routinely passes: >>>>>>> - the new Shenandoah tests >>>>>>> - jcstress with/without aggressive Shenandoah verification >>>>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>>>> >>>>>>> >>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>>> related changes, and of course the many early adopters for reporting >>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>> without any of you! >>>>>>> >>>>>>> Best regards, >>>>>>> Roman >>>>>>> From rahul.v.raghavan at oracle.com Thu Dec 6 14:03:43 2018 From: rahul.v.raghavan at oracle.com (Rahul Raghavan) Date: Thu, 6 Dec 2018 19:33:43 +0530 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: Message-ID: <3380cc2d-c606-653c-576d-a7b8682c37e5@oracle.com> [Just forwarding the below review request email to hotspot-compiler-dev, aarch32-port-dev mailing lists also] -------- Forwarded Message -------- Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 Date: Mon, 3 Dec 2018 06:06:47 +0000 From: Nick Gasson To: hotspot-dev at openjdk.java.net , Boris Ulasevich CC: nd Hi, Could someone please help me review this fix for a test failure on ARM32: Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ This fixes two assertion failures related to biased locking when C2's bias inlining feature is disabled (-XX:-OptoBiasInlining): MacroAssembler::atomic_cas_bool takes three register arguments plus a temporary register for use in the CAS loop. If the caller passes `noreg' as this temporary register (which happens in the !OptoBiasInlining call path from MacroAssembler::fast_lock) then MacroAssembler::atomic_cas_bool will default to using LR as a temporary. But it's possible with C2 that LR is one of the other three arguments which then triggers an assert_different_registers assertion failure. Fix this by supplying an additional scratch register to MacroAssembler::fast_lock if !OptoBiasInlining. In the !OptoBiasInlining case MacroAssembler::fast_lock calls MacroAssembler::biased_locking_enter to handle the case where the mark word contains the biased lock pattern which fast_lock wouldn't otherwise see if OptoBiasInlining was true. However in the case that we fail to acquire the biased lock this falls through to label `failed' and runs the normal fast_lock code that implicitly assumes the mark word does not have the bias pattern. This means we can end up with a BasicLock where _displaced_header contains the biased lock pattern which is an illegal state and subsequently triggers an assertion failure in ObjectSynchronizer::fast_exit. The right thing to do here is branch to `done' whenever the bias lock pattern is present and let the runtime handle the failure case. Also edited the comment describing MacroAssembler::biased_locking_enter to more accurately describe what it does (the comment about `slow_case' being optional is wrong). Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this patch. Thanks, Nick From rkennke at redhat.com Thu Dec 6 14:36:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 6 Dec 2018 15:36:24 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <496b3ac5-defc-92b6-7339-64c88c2e2dfa@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> <496b3ac5-defc-92b6-7339-64c88c2e2dfa@oracle.com> Message-ID: <73e720e4-251c-aa06-418b-3fc8409e7cf6@redhat.com> Hi Coleen, > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/shenandoah-gc/src/hotspot/share/gc/shenandoah/vm_operations_shenandoah.cpp.html > > > Can you rename these to shenandoahVMOperations.hpp/cpp to match the > newly agreed upon naming convention for this? > > See 8214791: Consistently name gc files containing VM operations [Was: > Re: RFR (S): 8214791: Rename vm_operations_g1* files to g1VMOperations*] Doing so: http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-December/008595.html I will integrate this in next round of webrevs. I expect the next will be the final round of webrevs. I got positive reviews for all shared-* parts, I'm waiting for shenandoah-* reviews from Shenandoah devs, plus Zhengyu+Thomas' TaskQueue stuff to arrive in upstream jdk. The CSR for Shenandoah flags has been approved, and the JEP should be moved to targeted JDK12 ~tomorrow. I intend/expect to push Shenandoah during the next couple of days, unless somebody speaks up until then :-) Thanks, Roman > > thanks, > Coleen > > On 12/4/18 3:37 PM, Roman Kennke wrote: >> Thanks, Leonid, for reviewing! >> >> Roman >> >> >>> Hi >>> >>> The shared tests changes looks good for me. Thank you for fixing and >>> testing different combinations. >>> >>> Leonid >>> >>>> On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: >>>> >>>> Round 5 of Shenandoah review includes: >>>> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >>>> correct now. We believe the CMS @requires was also not quite right and >>>> fixed it the same. >>>> >>>> It reads now: Don't run this test if: >>>> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >>>> true, as set by harness >>>> - Actual GC set by harness is Shenandoah *and* >>>> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >>>> default in Shenandoah, so this needs to be double-inverteed). >>>> >>>> The @requires for CMS was wrong before (we think), because it would >>>> also >>>> filter defaultGC + ExplicitGCInvokesConcurrent. >>>> >>>> - Sorting of macros was fixed, as was pointed out by Per >>>> - Some stuff was added to SA, as suggested by Jini >>>> - Rebased on most current jdk/jdk code >>>> >>>> Webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >>>> >>>> I also need reviews from GC reviewers for the CSR: >>>> https://bugs.openjdk.java.net/browse/JDK-8214349 >>>> >>>> I already got reviews for: >>>> [x] shared-runtime (coleenp) >>>> [x] shared-compiler (kvn) >>>> >>>> I got reviews for shared-build, but an earlier version, so maybe makes >>>> sense to look over this again. Erik J, Magnus? >>>> >>>> I still need approvals for: >>>> [ ] shared-build????????? (kvn, erikj, ihse, pliden) >>>> [ ] shared-gc???????????? (pliden, kbarrett) >>>> [ ] shared-serviceability (jgeorge, pliden) >>>> [ ] shared-tests????????? (lmesnik, pliden) >>>> [ ] shenandoah-gc >>>> [ ] shenandoah-tests >>>> >>>> >>>> Thanks for your patience and ongoing support! >>>> >>>> Cheers, >>>> Roman >>>> >>>> >>>>> Hi all, >>>>> >>>>> here comes round 4 of Shenandoah upstreaming review: >>>>> >>>>> This includes fixes for the issues that Per brought up: >>>>> - Verify and gracefully reject dangerous flags combinations that >>>>> disables required barriers >>>>> - Revisited @requires filters in tests >>>>> - Trim unused code from Shenandoah's SA impl >>>>> - Move ShenandoahGCTracer to gc/shenandoah >>>>> - Fix ordering of GC names in various files >>>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>>> >>>>> Thanks everybody for taking time to review this! >>>>> Roman >>>>> >>>>>> Hello all, >>>>>> >>>>>> Thanks so far for all the reviews and support! >>>>>> >>>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>>> Also, I fixed the numbering of my webrevs to match the >>>>>> review-round. ;-) >>>>>> >>>>>> Things we've changed today: >>>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>>> requires an addition in build machinery though, see >>>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>>> - Improved zero-disabling and build-code-simplification as >>>>>> suggested by >>>>>> Magnus and Per >>>>>> - Cleaned up some leftovers in C2 >>>>>> - Improved C2 loop opts code by introducing another APIs in >>>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC >>>>>> guards now. >>>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>>> noted earlier. This stuff is Shenandoah-specific, so let's just >>>>>> call it >>>>>> that. >>>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, >>>>>> etc) >>>>>> - Rebased on jdk-12+22 >>>>>> >>>>>> - Question: let us know if you need separate RFE for the new BSC2 >>>>>> APIs? >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>>> Alright, we fixed: >>>>>>> - The minor issues that Kim reported in shared-gc >>>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>>> >>>>>>> Some notes: >>>>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>>> (expectedly) failing. It can run CMS, default GC and any other GC >>>>>>> just >>>>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>>>> filters >>>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>>> >>>>>>> >>>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>>> - >>>>>>> test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>>> >>>>>>> we filter them for Shenandoah now. I'm wondering: how do you get >>>>>>> past >>>>>>> those with ZGC? >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>>> >>>>>>> >>>>>>> (Note to Leonid and tests reviewers: I'll add those related >>>>>>> filters in >>>>>>> next round). >>>>>>> >>>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code >>>>>>> look >>>>>>> better. I can tell that we're not done with C2 yet. Can you look >>>>>>> over >>>>>>> the code and see what is ok, and especially what is not ok, so >>>>>>> that we >>>>>>> can focus our efforts on the relevant parts? >>>>>>> >>>>>>> Updated set of webrevs: >>>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Roman >>>>>>> >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>>> mainline. >>>>>>>> I divided the review into parts that roughly correspond to the >>>>>>>> mailing lists >>>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>>> changes and >>>>>>>> 'shenandoah' code changes (actually, mostly additions). The >>>>>>>> intend is to >>>>>>>> eventually >>>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>>> >>>>>>>> JEP: >>>>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>>>> Bug entry: >>>>>>>> >>>>>>>> ? https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>>> >>>>>>>> Webrevs: >>>>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>>> >>>>>>>> For those who want to see the full change, have a look at the >>>>>>>> shenandoah-complete >>>>>>>> >>>>>>>> >>>>>>>> directory, >>>>>>>> it contains the full combined webrev. Alternatively, there is >>>>>>>> the file >>>>>>>> shenandoah-master.patch >>>>>>>> , >>>>>>>> >>>>>>>> which is what I intend to commit (and which should be equivalent >>>>>>>> to the >>>>>>>> 'shenandoah-complete' webrev). >>>>>>>> >>>>>>>> Sections to review (at this point) are the following: >>>>>>>> ? *) shenandoah-gc >>>>>>>> >>>>>>>> >>>>>>>> ???? - Actual Shenandoah implementation, almost completely >>>>>>>> residing in >>>>>>>> gc/shenandoah >>>>>>>> >>>>>>>> ? *) shared-gc >>>>>>>> >>>>>>>> >>>>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>>>> assert not >>>>>>>> fail (next to CMS and G1) >>>>>>>> ???? - taskqueue.hpp has some small adjustments to enable >>>>>>>> subclassing >>>>>>>> >>>>>>>> ? *) shared-serviceability >>>>>>>> >>>>>>>> >>>>>>>> ???? - The usual code to support another GC >>>>>>>> >>>>>>>> ? *) shared-runtime >>>>>>>> >>>>>>>> >>>>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>>>> iterators to >>>>>>>> hook up with, >>>>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>>> Shenandoah C2 optimizations >>>>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>>> Shenandoah's logging >>>>>>>> ???? - The usual macros in macro.hpp >>>>>>>> >>>>>>>> ? *) shared-build >>>>>>>> >>>>>>>> >>>>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>>>> Vladimir K. beforehand >>>>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>>> Shenandoah's barriers >>>>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>>>> source >>>>>>>> files, which is >>>>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>>>> significant >>>>>>>> regression if we >>>>>>>> ?????? don't) >>>>>>>> >>>>>>>> ? *) shared-tests >>>>>>>> >>>>>>>> >>>>>>>> ???? - Test infrastructure to support Shenandoah >>>>>>>> ???? - Shenandoah test groups >>>>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>>>> selected GC >>>>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>>>> sense to >>>>>>>> run with it >>>>>>>> >>>>>>>> ? *) shenandoah-tests >>>>>>>> >>>>>>>> >>>>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>>>> subdirectory >>>>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>>>> TestGCBasherWithShenandoah.java >>>>>>>> >>>>>>>> I intentionally left out shared-compiler for now, because we >>>>>>>> have some >>>>>>>> work left to do >>>>>>>> there, but if you click around you'll find the patch anyway, in >>>>>>>> case you >>>>>>>> want to take >>>>>>>> a peek at it. >>>>>>>> >>>>>>>> We have regular builds on: >>>>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>>> ?? - {Windows} x {x86_64}, >>>>>>>> ?? - {MacOS X} x {x86_64} >>>>>>>> >>>>>>>> This also routinely passes: >>>>>>>> ?? - the new Shenandoah tests >>>>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>>> >>>>>>>> >>>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>>>> deserves >>>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>>> contributions, everybody else in Red Hat's OpenJDK team for >>>>>>>> testing, >>>>>>>> advice and support, my collegues in Oracle's GC, runtime and >>>>>>>> compiler >>>>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>>>> interface and >>>>>>>> related changes, and of course the many early adopters for >>>>>>>> reporting >>>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>>> without any of you! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Roman >>>>>>>> > From matthias.baesken at sap.com Thu Dec 6 15:17:05 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 6 Dec 2018 15:17:05 +0000 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: References: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> Message-ID: Hi David, here is a little example program showing the strerror-related warning of VS2017 (guess it works also with lower VS versions) ; compile with -W3 : $ cl strwarn.cpp -W3 -o strwarn Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24234.1 for x64 Copyright (C) Microsoft Corporation. All rights reserved. ...... strwarn.cpp(12): warning C4996: 'strerror': This function or variable may be unsafe. Consider using strerror_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. .... ------------------- /* see : jdk/src/hotspot/share/utilities/globalDefinitions_visCPP.hpp #pragma warning( disable : 4996 ) // unsafe string functions. Same as define _CRT_SECURE_NO_WARNINGS/_CRT_SECURE_NO_DEPRICATE */ #include #include #include int main() { FILE* f = fopen ("nofile","r"); if (f == NULL) { printf ("Error opening file: %s \n",strerror(errno)); } return 0; } ------------------------- However we disable warning C4996 in jdk/src/hotspot/share/utilities/globalDefinitions_visCPP.hpp , so no wonder we do not get it. Probably removing the disabling of warning C4996 in jdk/src/hotspot/share/utilities/globalDefinitions_visCPP.hpp might cause much more warnings than just the strerror-warning "family" , so I am not sure if it is really an option at the moment . Best regards, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Donnerstag, 6. Dezember 2018 14:20 > To: 'David Holmes' ; 'hotspot- > dev at openjdk.java.net' > Cc: 'magnus.ihse.bursie at oracle.com' > Subject: RE: RFR: 8214944: replace strerror by os::strerror > > Hi David and Nils, thanks for the reviews . > > > > > Pity there is no way to detect when we use something like that which we > > shouldn't. > > > > Seems VS2017 has compiler warnings for this , at least > > https://github.com/fmtlib/fmt/issues/703 > > suggests this . > See the example output : > > > format.cc:172:17: warning: 'strerror' is deprecated: This function or variable > may be unsafe. Consider using strerror_s instead. To disable deprecation, > use _CRT_SECURE_NO_WARNINGS. See online help for details. [- > Wdeprecated-declarations] > buffer_ = strerror(error_code_); > ^ > C:\Program Files (x86)\Windows > Kits\10\Include\10.0.16299.0\ucrt\string.h:180:16: note: 'strerror' has been > explicitly marked deprecated here > _Check_return_ _CRT_INSECURE_DEPRECATE(strerror_s) > ^ > > Might be we do not have these warnings enabled for some reason . > > > Best regards, Matthias > > > > -----Original Message----- > > From: David Holmes > > Sent: Donnerstag, 6. Dezember 2018 13:57 > > To: Baesken, Matthias ; 'hotspot- > > dev at openjdk.java.net' > > Subject: Re: RFR: 8214944: replace strerror by os::strerror > > > > Hi Matthias, > > > > On 6/12/2018 8:36 pm, Baesken, Matthias wrote: > > > Hello , please review this small fix . > > > > > > Hotspot still has a few uses of the thread unsafe strerror function. > > > Most places where strerror has been used previously, were already > > replaced by os::strerror (using a errno_to_string error text array). > > > > > > But a few places remained and should also use better use os:strerror. > > > > It's not that they "remain" but were added later. Previous strerror > > replacement was done under JDK-8148425. The new uses in OS-specific > > get_mtime functions were added in a few weeks later by JDK-8146879. The > > ZGC use is obviously much more recent. > > > > Pity there is no way to detect when we use something like that which we > > shouldn't. > > > > Fix looks good. > > > > Thanks, > > David > > ----- > > > > > > > > Bug/webrev : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8214944 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8214944.0/ > > > > > > > > > Thanks, Matthias > > > From doko at ubuntu.com Thu Dec 6 15:34:01 2018 From: doko at ubuntu.com (Matthias Klose) Date: Thu, 6 Dec 2018 16:34:01 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> Message-ID: On 04.12.18 14:54, Aleksey Shipilev wrote: > On 12/4/18 2:24 PM, Matthias Klose wrote: >> with jdk-12+22, bootcycle builds fail at least on x86_64-linux-gnu. Is there >> some place where I can check if this kind of build succeeds for others? > > Just did this on jdk/jdk tip, and it passed: > > $ CONF=linux-x86_64-server-fastdebug make bootcycle-images > ... > Creating CDS archive for jdk image > Stopping sjavac server > Finished building target 'product-images' in configuration 'linux-x86_64-server-fastdebug' > Finished building target 'bootcycle-images' in configuration 'linux-x86_64-server-fastdebug' my bad, that happens in the test-image target: In the build log I see: FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' make/Main.gmk:1087: *** target pattern contains no '%'. Stop. make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' make[5]: warning: -j1 forced in submake: resetting jobserver mode. Main.gmk: # Let "run-test" be an alias for "test" $(foreach t, $(ALL_NAMED_TESTS), $(eval run-test-$t: test-$t)) $ make --version GNU Make 4.2.1 Built for x86_64-pc-linux-gnu From shade at redhat.com Thu Dec 6 16:41:26 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 6 Dec 2018 17:41:26 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> Message-ID: <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> On 12/6/18 4:34 PM, Matthias Klose wrote: > my bad, that happens in the test-image target: > > In the build log I see: > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp > > > make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' > make/Main.gmk:1087: *** target pattern contains no '%'. Stop. > make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' > make[5]: warning: -j1 forced in submake: resetting jobserver mode. I wonder if "~" in your path screws things up. "test-image" also passes for me on current jdk/jdk: ~/trunks/jdk-jdk $ CONF=linux-x86_64-server-fastdebug make test-image ... Finished building target 'test-image' in configuration 'linux-x86_64-server-fastdebug' -Aleksey From doko at ubuntu.com Thu Dec 6 18:31:17 2018 From: doko at ubuntu.com (Matthias Klose) Date: Thu, 6 Dec 2018 19:31:17 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> Message-ID: On 06.12.18 17:41, Aleksey Shipilev wrote: > On 12/6/18 4:34 PM, Matthias Klose wrote: >> my bad, that happens in the test-image target: >> >> In the build log I see: >> >> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >> >> >> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >> make/Main.gmk:1087: *** target pattern contains no '%'. Stop. >> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >> make[5]: warning: -j1 forced in submake: resetting jobserver mode. > > I wonder if "~" in your path screws things up. that worked at least for the openjdk-11 development cycle, and replacing the tilde with a dash makes no difference. From erik.joelsson at oracle.com Thu Dec 6 19:04:18 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 6 Dec 2018 11:04:18 -0800 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> Message-ID: Could you insert this before line 1087 in make/Main.gmk and post the output? $(call PrintVar, ALL_NAMED_TESTS) /Erik On 2018-12-06 10:31, Matthias Klose wrote: > On 06.12.18 17:41, Aleksey Shipilev wrote: >> On 12/6/18 4:34 PM, Matthias Klose wrote: >>> my bad, that happens in the test-image target: >>> >>> In the build log I see: >>> >>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>> >>> >>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>> make/Main.gmk:1087: *** target pattern contains no '%'. Stop. >>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >> I wonder if "~" in your path screws things up. > that worked at least for the openjdk-11 development cycle, and replacing the > tilde with a dash makes no difference. From doko at ubuntu.com Thu Dec 6 19:23:29 2018 From: doko at ubuntu.com (Matthias Klose) Date: Thu, 6 Dec 2018 20:23:29 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> Message-ID: On 06.12.18 20:04, Erik Joelsson wrote: > Could you insert this before line 1087 in make/Main.gmk and post the output? > > $(call PrintVar, ALL_NAMED_TESTS) > > /Erik > > On 2018-12-06 10:31, Matthias Klose wrote: >> On 06.12.18 17:41, Aleksey Shipilev wrote: >>> On 12/6/18 4:34 PM, Matthias Klose wrote: >>>> my bad, that happens in the test-image target: >>>> >>>> In the build log I see: >>>> >>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>>> >>>> >>>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>> make/Main.gmk:1087: *** target pattern contains no '%'.? Stop. >>>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >>> I wonder if "~" in your path screws things up. >> that worked at least for the openjdk-11 development cycle, and replacing the >> tilde with a dash makes no difference. ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc hotspot_handshake hotspot_misc hotspot_native_sanity ho tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime hotspot_runtime hotspot_runtime_minimalvm ho tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt jdk_beans jdk_client_sanity jdk_collections jd k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core jdk_imageio jdk_instrument jdk_instrument_sanity jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity jdk_lang jdk_launcher jdk_management jdk_mana gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common tier1_compiler tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro make-make-base make-java-compilation make-copy-files make-idea make-compile-commands failure-handler make< From coleen.phillimore at oracle.com Thu Dec 6 20:47:53 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Dec 2018 15:47:53 -0500 Subject: RFR (S) 8214972: Uses of klass_holder() except GC need to apply GC barriers Message-ID: Summary: Fix klass_holder() and make all callers use it, remove holder_phantom().* *This removes the confusion about which one to use, except for GC, who has to know. open webrev at http://cr.openjdk.java.net/~coleenp/8214972.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8214972 Ran tier1-3 tests. Thanks, Coleen ** From kim.barrett at oracle.com Thu Dec 6 20:55:55 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Dec 2018 15:55:55 -0500 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> References: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> Message-ID: <2B532229-00AC-4076-883D-37CFBE926D37@oracle.com> > On Dec 6, 2018, at 7:56 AM, David Holmes wrote: > Pity there is no way to detect when we use something like that which we shouldn?t. You asked for it: https://bugs.openjdk.java.net/browse/JDK-8214976 :) From erik.osterlund at oracle.com Thu Dec 6 21:00:19 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 6 Dec 2018 22:00:19 +0100 Subject: RFR (S) 8214972: Uses of klass_holder() except GC need to apply GC barriers In-Reply-To: References: Message-ID: Hi Coleen, Looks good. Thanks, /Erik > On 6 Dec 2018, at 21:47, coleen.phillimore at oracle.com wrote: > > Summary: Fix klass_holder() and make all callers use it, remove holder_phantom().* > > *This removes the confusion about which one to use, except for GC, who has to know. > > open webrev at http://cr.openjdk.java.net/~coleenp/8214972.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8214972 > > Ran tier1-3 tests. > > Thanks, > Coleen > ** From erik.joelsson at oracle.com Thu Dec 6 21:03:09 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 6 Dec 2018 13:03:09 -0800 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> Message-ID: Nothing strange in there. Is it only printed once? I wouldn't be surprised if this got printed more than one time during a normal make execution (due to reloads caused by -include). If it is, then perhaps there is something different in a later print? /Erik On 2018-12-06 11:23, Matthias Klose wrote: > On 06.12.18 20:04, Erik Joelsson wrote: >> Could you insert this before line 1087 in make/Main.gmk and post the output? >> >> $(call PrintVar, ALL_NAMED_TESTS) >> >> /Erik >> >> On 2018-12-06 10:31, Matthias Klose wrote: >>> On 06.12.18 17:41, Aleksey Shipilev wrote: >>>> On 12/6/18 4:34 PM, Matthias Klose wrote: >>>>> my bad, that happens in the test-image target: >>>>> >>>>> In the build log I see: >>>>> >>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>>>> >>>>> >>>>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>> make/Main.gmk:1087: *** target pattern contains no '%'.? Stop. >>>>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >>>> I wonder if "~" in your path screws things up. >>> that worked at least for the openjdk-11 development cycle, and replacing the >>> tilde with a dash makes no difference. > ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all > hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co > mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc > hotspot_handshake hotspot_misc hotspot_native_sanity ho > tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime > hotspot_runtime hotspot_runtime_minimalvm ho > tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime > hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru > ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt > jdk_beans jdk_client_sanity jdk_collections jd > k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core > jdk_imageio jdk_instrument jdk_instrument_sanity > jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity > jdk_lang jdk_launcher jdk_management jdk_mana > gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi > jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 > jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity > jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other > jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common tier1_compiler > tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms > tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher > tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime > tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 > tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages > vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi > vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick > vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring > vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict > vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth > vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact > vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle > vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick > vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro > make-make-base make-java-compilation make-copy-files make-idea > make-compile-commands failure-handler make< From kim.barrett at oracle.com Thu Dec 6 21:01:16 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Dec 2018 16:01:16 -0500 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: References: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> Message-ID: <24DE86FA-6C28-4F2C-8504-3DE5F0705E21@oracle.com> > On Dec 6, 2018, at 10:17 AM, Baesken, Matthias wrote: > However we disable warning C4996 in jdk/src/hotspot/share/utilities/globalDefinitions_visCPP.hpp , > so no wonder we do not get it. > Probably removing the disabling of warning C4996 in jdk/src/hotspot/share/utilities/globalDefinitions_visCPP.hpp might cause much more warnings than just the strerror-warning "family" , > so I am not sure if it is really an option at the moment . The _CRT_SECURE_NO_WARNINGS &etc stuff is *not* something we want to change. This misfeature declares a bunch of standard functions to be "deprecated" in favor of corresponding non-standard functions. From coleen.phillimore at oracle.com Thu Dec 6 21:05:44 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Dec 2018 16:05:44 -0500 Subject: RFR (S) 8214972: Uses of klass_holder() except GC need to apply GC barriers In-Reply-To: References: Message-ID: <7efda1b5-77a8-3cee-c8e9-98a00cb09b86@oracle.com> Thanks Erik! Coleen On 12/6/18 4:00 PM, Erik Osterlund wrote: > Hi Coleen, > > Looks good. > > Thanks, > /Erik > >> On 6 Dec 2018, at 21:47, coleen.phillimore at oracle.com wrote: >> >> Summary: Fix klass_holder() and make all callers use it, remove holder_phantom().* >> >> *This removes the confusion about which one to use, except for GC, who has to know. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8214972.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8214972 >> >> Ran tier1-3 tests. >> >> Thanks, >> Coleen >> ** From kim.barrett at oracle.com Thu Dec 6 22:04:54 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Dec 2018 17:04:54 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <4f10af7b-a18a-ac95-6371-7b2c5c6f3a89@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> <396B4688-DF16-47D5-9F53-31FC2524C4F3@oracle.com> <4f10af7b-a18a-ac95-6371-7b2c5c6f3a89@oracle.com> Message-ID: <4CEA4338-5D3D-4537-A80B-F7AC5420AFC7@oracle.com> > On Dec 3, 2018, at 2:33 AM, David Holmes wrote: > > Hi Kim, > > On 3/12/2018 5:03 pm, Kim Barrett wrote: >>> On Nov 27, 2018, at 4:34 PM, David Holmes wrote: >>> >>> On 28/11/2018 4:12 am, Kim Barrett wrote: >>>> The "natural result" is that the derived class copy-assign code does >>>> what it does. There are things that can go badly wrong here regarding >>>> lifetimes of embedded object references. But the asserted restriction >>>> on the allocated location of the object being copy-assigned to does >>>> nothing that I can see to prevent those potential problems. >>>> Can you provide a use-case where the assertion actually does something >>>> useful? I removed it because I don?t think such a thing exists. >>> >>> AFAICS the only time it makes any sense to use assignment is with stack/embedded ResourceObj. The assert verifies that is the case. >>> >>> This seems a very clear cut use of an assert to ensure an API is not being misused. Does it prevent all possible misuse - no. But that's not a reason to remove what it does do. >>> >>> Do we ever need the assignment semantics? Maybe we should just prohibit it altogether? >>> >>> David >> I don't understand why you might think that, and don't agree. Maybe >> there are additional unstated and unenforced restrictions? >> But part of my rejection of such a position is that a commonly >> understood idiomatic operation like assignment should either just >> work, or it shouldn't be allowed at all. Having a strange restriction >> like this makes code much harder to understand. Make a new operation >> for this restricted assignment. > > How do you make a special assignment operator? AFAIK there is only one op= You spell it differently and forbid op=, e.g. something like ResourceObj& assign_living_dangerously(const ResourceObj&); ResourceObj& operator=(const ResourceObj&) = delete; // C++11 > ResourceObj is not a nice clean, well-defined class I can certainly agree with that! > - it is one class that embodies the possibility of being one of three different kinds of "resource obj" (embedded/stack, arena-allocated, C-heap allocated) and the operations of copying and assignment don't make sense for all of these - hence the restriction. A ResourceObj cannot have ownership of any contained allocated objects (ownership in the sense of having responsibility for deallocation), because the resource and arena mechanisms by design do not call the destructor of relevant objects. (Such destructor avoidance is allowed by C++ 3.8/4, so long as the destructor doesn't have any side effects the application depends on.) So copy and copy-assign for a class derived from ResourceObj (and so possibly resource allocated) can't really do anything more than a shallow copy unless it can somehow know that it was not resource allocated. The relevant API provided by ResourceObj is DEBUG_ONLY, so can't be used for conditionalization, only assertion. Some classes (GrowableArray, for example), have their own mechanism for determining allocation location and ResourceMark nesting when relevant. GrowableArray is actually a pretty good example of how things can go awry right now. It has the default copy constructor and copy assignment operator, which are both wrong and better not be used in some cases. Both copy and copy-assign when the copy source is on_C_heap will result in double-free. (The ResourceObj copy-assign assert doesn't help here.) Copy-assign to an on_C_heap object will leak the old data array. (The ResourceObj copy-assign assert doesn't help very much here either. The array can be stack/member allocated but configured to allocate its data from the C heap; that's an entirely reasonable configuration.) Both of those could be addressed by providing the appropriate definition (similar to what, for example, std::vector does). Of course, resource/arena allocating a growable array configured to use the C heap for its data is a mistake, since the destructor won't be called to free the data, so leaking it. There are other cases that could be okay or not, and may or may not be consistent with the ResourceObj copy-assign assertion. I think the debug-only nesting checking done by GrowableArray is likely more useful and correct than that ResourceObj assertion. I looked at a few actual uses. None were fatally wrong, but none were good either. Some looked accidental. >> Unfortunately, there are already many uses of ResourceObj assignment. >> I spent a bit of time exploring the idea of removing or replacing it, >> but that seems way too hard for the intended scope of this cleanup. >> (Note that for some of the existing uses it is not obvious whether the >> restriction applies. Presumably it doesn't, since we're not hitting >> the assert.) While I like the idea of prohibiting assignment (and >> possibly copy too, which has many of the same issues), I'm not going >> to pursue that beyond perhaps an RFE for future work. >> BTW, as part of that exploration I found such gems as this, from >> opto/cfgnode.cpp:962: >> Node_Array node_map = new Node_Array(a); > > I can certainly understand people being confused about C++ allocation, construction and assignment. So I think the above acts as: > > Node_Array node_map; // Stack object using default constructor > node_map.operator=(new Node_Array(a)); > > Which means the whole "new Node_Array(a)" is completely useless (does it cause a leak?) as operator= ignores it. Nope. It is equivalent to Node_Array node_map(new Node_Array(a)); No operator= involved at all. Rather, node_map is a stack-allocated Node_Array constructed via the conversion (not copy!) constructor Node_Array(Node_Array*). That constructor default constructs the ResourceObj base class object and initializes the Node_Array members from the corresponding members of the argument object. The result of "new Node_Array(a)" is then dropped. Since it's a ResourceObj, it was allocated in the resource area and will be reclaimed when the innermost enclosing ResourceMark is exited. A better way to write this would be Node_Array node_map(a); >> Anyway, even though I think the assert prevents some things that would >> work (but should perhaps be avoided anyway), and doesn't prevent some >> things that won't work, I'm willing to reinstate it in order to make >> progress. >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8213481/open.01/ >> incr: http://cr.openjdk.java.net/~kbarrett/8213481/open.01.inc/ > > Thanks for doing that. Anyway, this has gone somewhat astray from this particular change. I'm not sure where I am, review-wise. I *think* both David and Vladimir are okay with the latest version? From kim.barrett at oracle.com Fri Dec 7 00:13:28 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Dec 2018 19:13:28 -0500 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: References: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> <2B532229-00AC-4076-883D-37CFBE926D37@oracle.com> Message-ID: <3F625340-D194-4435-980E-A7BCEA77230B@oracle.com> > On Dec 6, 2018, at 4:07 PM, Thomas St?fe wrote: > > On Thu, Dec 6, 2018 at 9:56 PM Kim Barrett wrote: >> >>> On Dec 6, 2018, at 7:56 AM, David Holmes wrote: >>> Pity there is no way to detect when we use something like that which we shouldn?t. >> >> You asked for it: >> https://bugs.openjdk.java.net/browse/JDK-8214976 >> :) >> > > That is neat! Thanks. > Note that I would like to have a simple, compiler-independent way to > locally disable those warnings though. I?d left that as an exercise for the reader, but have since added a comment to the RFE describing how to do that. From david.holmes at oracle.com Fri Dec 7 00:46:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Dec 2018 10:46:36 +1000 Subject: RFR: 8214944: replace strerror by os::strerror In-Reply-To: <2B532229-00AC-4076-883D-37CFBE926D37@oracle.com> References: <4a99ac5c-efb2-0534-5266-d40fa6fedbd7@oracle.com> <2B532229-00AC-4076-883D-37CFBE926D37@oracle.com> Message-ID: On 7/12/2018 6:55 am, Kim Barrett wrote: >> On Dec 6, 2018, at 7:56 AM, David Holmes wrote: >> Pity there is no way to detect when we use something like that which we shouldn?t. > > You asked for it: > https://bugs.openjdk.java.net/browse/JDK-8214976 > :) I like it! Thanks, David From joe.darcy at oracle.com Fri Dec 7 02:20:15 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 06 Dec 2018 18:20:15 -0800 Subject: JDK 13 RFR of hotspot portions of JDK-8205626: Start of release updates for JDK 13 Message-ID: <5C09D8DF.1030503@oracle.com> Hello, With the start of JDK 13 around the corner, please review the hotspot portions of: JDK-8205626: Start of release updates for JDK 13 http://cr.openjdk.java.net/~darcy/jdk13-fork.2 Those updates include (but are not limited to) N -> (N+1) updates in: src/hotspot/share/classfile/classFileParser.cpp src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/classfile/Classfile.java Thanks, -Joe From david.holmes at oracle.com Fri Dec 7 03:05:44 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Dec 2018 13:05:44 +1000 Subject: JDK 13 RFR of hotspot portions of JDK-8205626: Start of release updates for JDK 13 In-Reply-To: <5C09D8DF.1030503@oracle.com> References: <5C09D8DF.1030503@oracle.com> Message-ID: Hi Joe, On 7/12/2018 12:20 pm, Joseph D. Darcy wrote: > Hello, > > With the start of JDK 13 around the corner, please review the hotspot > portions of: > > ??? JDK-8205626: Start of release updates for JDK 13 > ??? http://cr.openjdk.java.net/~darcy/jdk13-fork.2 > > Those updates include (but are not limited to) N -> (N+1) updates in: > > ??? src/hotspot/share/classfile/classFileParser.cpp > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/classfile/Classfile.java I couldn't see any other hotspot changes besides those two. ?? Changes seem fine. Pity the classfile version is not available through a Java level API. Thanks, David > > Thanks, > > -Joe From dean.long at oracle.com Fri Dec 7 03:29:03 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 6 Dec 2018 19:29:03 -0800 Subject: RFR (S) 8214972: Uses of klass_holder() except GC need to apply GC barriers In-Reply-To: References: Message-ID: <0d7642c1-c6a1-4b2a-bad0-7135c33a1cfd@oracle.com> It looks like we no longer use java_mirror() for is_unsafe_anonymous() classes? dl On 12/6/18 12:47 PM, coleen.phillimore at oracle.com wrote: > Summary: Fix klass_holder() and make all callers use it, remove > holder_phantom().* > > *This removes the confusion about which one to use, except for GC, who > has to know. > > open webrev at http://cr.openjdk.java.net/~coleenp/8214972.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8214972 > > Ran tier1-3 tests. > > Thanks, > Coleen > ** From kim.barrett at oracle.com Fri Dec 7 06:31:52 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Dec 2018 01:31:52 -0500 Subject: RFR: 8214315: G1: fatal error: acquiring lock SATB_Q_FL_lock/1 out of order with lock tty_lock/0 Message-ID: Please review this change to the lock rank for the tty_lock. We introduce a new lock rank 'tty', which is placed below the 'special' rank but above the 'access' ranks. This new rank is used only by the tty_lock, which formerly had 'event' rank. This eliminates a rare lock rank inversion in nmethod::log_new_method, where the tty_lock is locked around a resolve of a jweak. That resolve could require allocating a SATB buffer or enqueuing a full one, either of which will attempt to lock an 'access' mutex. With the old ranks, that 'access' mutex has higher rank than the already owned tty_lock, asserting in a debug build, and potentially deadlocking in a product build. (The deadlock scenario seems unlikely; see JDK-8214997 for a related scenario.) CR: https://bugs.openjdk.java.net/browse/JDK-8214997 Webrev: http://cr.openjdk.java.net/~kbarrett/8214315/open.00/ Testing: mach5 tier1-5 From erik.osterlund at oracle.com Fri Dec 7 07:38:49 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 7 Dec 2018 08:38:49 +0100 Subject: RFR: 8214315: G1: fatal error: acquiring lock SATB_Q_FL_lock/1 out of order with lock tty_lock/0 In-Reply-To: References: Message-ID: Hi Kim, Looks good. Thanks, /Erik On 2018-12-07 07:31, Kim Barrett wrote: > Please review this change to the lock rank for the tty_lock. We > introduce a new lock rank 'tty', which is placed below the 'special' > rank but above the 'access' ranks. This new rank is used only by the > tty_lock, which formerly had 'event' rank. > > This eliminates a rare lock rank inversion in nmethod::log_new_method, > where the tty_lock is locked around a resolve of a jweak. That > resolve could require allocating a SATB buffer or enqueuing a full > one, either of which will attempt to lock an 'access' mutex. With the > old ranks, that 'access' mutex has higher rank than the already owned > tty_lock, asserting in a debug build, and potentially deadlocking in a > product build. (The deadlock scenario seems unlikely; see JDK-8214997 > for a related scenario.) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8214997 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8214315/open.00/ > > Testing: > mach5 tier1-5 > From shade at redhat.com Fri Dec 7 10:26:26 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 7 Dec 2018 11:26:26 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <9fca6745-8c43-a15b-a967-2ce1be84e0be@redhat.com> On 12/4/18 8:10 AM, Roman Kennke wrote: > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > [ ] shenandoah-gc > [ ] shenandoah-tests These two parts look good. -Aleksey From david.holmes at oracle.com Fri Dec 7 10:35:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Dec 2018 20:35:17 +1000 Subject: hg: jdk/jdk: 8214944: replace strerror by os::strerror In-Reply-To: <201812070806.wB7869pn017767@aojmv0008.oracle.com> References: <201812070806.wB7869pn017767@aojmv0008.oracle.com> Message-ID: Hi Matthias, Unfortunately this has caused a tier1 build failure with PCH disabled: open/src/hotspot/share/gc/z/zErrno.cpp:49:10: error: 'os' has not been declared return os::strerror(_error); ^~ Can you please file a bug and fix ASAP. Thanks, David On 7/12/2018 6:06 pm, matthias.baesken at sap.com wrote: > Changeset: 442d322bb6d8 > Author: mbaesken > Date: 2018-12-06 09:48 +0100 > URL: http://hg.openjdk.java.net/jdk/jdk/rev/442d322bb6d8 > > 8214944: replace strerror by os::strerror > Reviewed-by: dholmes, neliasso > > ! src/hotspot/os/aix/os_aix.cpp > ! src/hotspot/os/bsd/os_bsd.cpp > ! src/hotspot/os/linux/os_linux.cpp > ! src/hotspot/os/solaris/os_solaris.cpp > ! src/hotspot/os/windows/os_windows.cpp > ! src/hotspot/share/gc/z/zErrno.cpp > ! src/hotspot/share/logging/logFileOutput.cpp > From david.holmes at oracle.com Fri Dec 7 10:37:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Dec 2018 20:37:27 +1000 Subject: hg: jdk/jdk: 8214944: replace strerror by os::strerror In-Reply-To: References: <201812070806.wB7869pn017767@aojmv0008.oracle.com> Message-ID: Never mind it's already been fixed. David On 7/12/2018 8:35 pm, David Holmes wrote: > Hi Matthias, > > Unfortunately this has caused a tier1 build failure with PCH disabled: > > open/src/hotspot/share/gc/z/zErrno.cpp:49:10: error: 'os' has not been > declared > ??? return os::strerror(_error); > ?????????? ^~ > > Can you please file a bug and fix ASAP. > > Thanks, > David > > On 7/12/2018 6:06 pm, matthias.baesken at sap.com wrote: >> Changeset: 442d322bb6d8 >> Author:??? mbaesken >> Date:????? 2018-12-06 09:48 +0100 >> URL:?????? http://hg.openjdk.java.net/jdk/jdk/rev/442d322bb6d8 >> >> 8214944: replace strerror by os::strerror >> Reviewed-by: dholmes, neliasso >> >> ! src/hotspot/os/aix/os_aix.cpp >> ! src/hotspot/os/bsd/os_bsd.cpp >> ! src/hotspot/os/linux/os_linux.cpp >> ! src/hotspot/os/solaris/os_solaris.cpp >> ! src/hotspot/os/windows/os_windows.cpp >> ! src/hotspot/share/gc/z/zErrno.cpp >> ! src/hotspot/share/logging/logFileOutput.cpp >> From rwestrel at redhat.com Fri Dec 7 10:40:06 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 07 Dec 2018 11:40:06 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <871s6tiprt.fsf@redhat.com> > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ C2 changes look good to me. Roland. From thomas.schatzl at oracle.com Fri Dec 7 10:54:01 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 07 Dec 2018 11:54:01 +0100 Subject: RFR: 8214315: G1: fatal error: acquiring lock SATB_Q_FL_lock/1 out of order with lock tty_lock/0 In-Reply-To: References: Message-ID: Hi, On Fri, 2018-12-07 at 01:31 -0500, Kim Barrett wrote: > Please review this change to the lock rank for the tty_lock. We > introduce a new lock rank 'tty', which is placed below the 'special' > rank but above the 'access' ranks. This new rank is used only by the > tty_lock, which formerly had 'event' rank. > > This eliminates a rare lock rank inversion in > nmethod::log_new_method, where the tty_lock is locked around a > resolve of a jweak. That resolve could require allocating a SATB > buffer or enqueuing a full one, either of which will attempt to lock > an 'access' mutex. With the old ranks, that 'access' mutex has > higher rank than the already owned tty_lock, asserting in a debug > build, and potentially deadlocking in a product build. (The deadlock > scenario seems unlikely; see JDK-8214997 for a related scenario.) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8214997 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8214315/open.00/ > > Testing: > mach5 tier1-5 > looks good. Thomas From matthias.baesken at sap.com Fri Dec 7 12:00:40 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 7 Dec 2018 12:00:40 +0000 Subject: 8214976: Warn about uses of functions replaced for portability - was : RE: RFR: 8214944: replace strerror by os::strerror Message-ID: Hi Kim, this looks great, thanks ! Maybe strtok could be considered too for the list of __attribute__ declarations to cause warnings , because this got pushed recently : 8214773: Replace use of thread unsafe strtok https://hg.openjdk.java.net/jdk/jdk/rev/8df9cf767f79 Best regards, Matthias > -----Original Message----- > From: Kim Barrett > Sent: Freitag, 7. Dezember 2018 01:13 > To: Thomas St?fe > Cc: David Holmes ; HotSpot Open Source > Developers ; Baesken, Matthias > > Subject: Re: RFR: 8214944: replace strerror by os::strerror > > > On Dec 6, 2018, at 4:07 PM, Thomas St?fe > wrote: > > > > On Thu, Dec 6, 2018 at 9:56 PM Kim Barrett > wrote: > >> > >>> On Dec 6, 2018, at 7:56 AM, David Holmes > wrote: > >>> Pity there is no way to detect when we use something like that which > we shouldn?t. > >> > >> You asked for it: > >> https://bugs.openjdk.java.net/browse/JDK-8214976 > >> :) > >> > > > > That is neat! > > Thanks. > > > Note that I would like to have a simple, compiler-independent way to > > locally disable those warnings though. > > I?d left that as an exercise for the reader, but have since added a comment to > the RFE > describing how to do that. From thomas.schatzl at oracle.com Fri Dec 7 12:14:59 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 07 Dec 2018 13:14:59 +0100 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: Message-ID: Hi Man, first, sorry for the wait, this RFR fell through the cracks somehow here... On Fri, 2018-11-16 at 19:10 -0800, Man Cao wrote: > Hi all, > > Could I have reviews for this refactoring change? > Webrev: https://cr.openjdk.java.net/~manc/8212206/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8212206 > > It basically moves AdaptiveSizePolicy::check_gc_overhead_limit() to > its own class so it can be shared with collectors that do not support > AdaptiveSizePolicy. > Each collector can provide its own functions to check if GC time or > space overhead has exceeded threshold. It also removes one unused > parameter "young_live". > I think this is sufficient to share the common code for implementing > UseGCOverheadLimit for G1 (JDK-8212084). > > Initially I tried moving all code for collecting GC time overhead in > AdaptiveSizePolicy to a separate class, e.g. > _collection_(). > However, G1 already has its own ways to compute GC cost in > G1Analytics. For example, G1HeapSizingPolicy::expansion_amount() > already uses G1Analytics::_recent_avg_pause_time_ratio for GC cost. > It is undesirable to have two ways to compute GC cost in one > collector. > In general, different collectors have different ways to compute GC > time and space overhead. It could be too restrictive to make an > interface to define how to collect GC time and space overhead. > > Thus, this patch only moves the code to check GC overhead in a common > place. It could be reused to implement UseGCOverheadLimit for > collectors such as ZGC in the future. - I would prefer if "OverheadChecker" would be prefixed by a "GC", i.e. GCOverheadChecker to make sure it is about GC - not sure about the usefulness of the OverheadTester interface. At least in the current implementation its implementation could be easily replaced by static functions returning a bool. That would save a lot of boiler plate code. Is there some reason to have explicit classes for that? At least please add a "GC" prefix to the class here too. - style: in new classes, please align the visibility modifiers with the "class" keyword without indentation. - extra newline at adaptiveSizePolicy.cpp:489 looks good otherwise. > > There could be further refactoring to do on top of this patch. I'd > like to get some opinions on whether they are desirable to do, and if > they could be done as part of this RFE: > (1) The develop flag AdaptiveSizePolicyGCTimeLimitThreshold could be > renamed to GCOverheadLimitThreshold to be more accurate. I'm not sure > about the process of renaming a develop flag, though. Just rename it. Develop flags can be changed without any additional process. > (2) The instance of the new class OverheadChecker could be a member > of CollectedHeap, instead of a member of AdaptiveSizePolicy. Note > that CollectedHeap::mem_allocate() already has a parameter > "bool* gc_overhead_limit_was_exceeded". Assuming that all collectors want to implement this, and actually need to I am leaning towards doing so. However the ZGC/Shenandoah people might object to this. > (3) The hsperf counter sun.gc.policy.gcTimeLimitExceeded is currently > only available to ParallelGC. It can be moved to GCPolicyCounters so > it is available to all collectors. Also gcTimeLimitExceeded is a > misnomer, gcOverheadLimitExceeded is more accurate. Okay. However I do not know how to rename them and what are the requirements here. Probably a CSR is needed, although I have seen translation tables used (in src/jdk.internal.jvmstat/share/classes/sun/jvmstat/perfdata/resources/a liasmap). I cc'ed hotspot-dev in the assumption somebody else there knows more about this. Thanks, Thomas From rkennke at redhat.com Fri Dec 7 13:00:51 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 7 Dec 2018 14:00:51 +0100 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: Here comes round 6, possibly/hopefully the last round of webrevs for upstreaming Shenandoah: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ It incorporates: - renames vm_operations_shenandoah* to shenandoahVMOperations*, as suggested by Coleen - reshuffles gcCause in shared-gc SA as suggested by Per - print number of threads in SA, as suggested by Jini - fixed a problem in webrev generation that did not track move of CriticalNativeStress.java and CriticalNativeArgs.java The CSR has been approved, the JEP moved to target jdk12, and I got positive reviews for all parts. I intend to push this early next week, unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes in by then, I'll incorporate it. Thanks everybody for reviewing and reviewing and reviewing again ;-) Cheers, Roman > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ?*) shenandoah-gc >>>>> >>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ?*) shared-gc >>>>> >>>>> ??? - This is mostly boilerplate that is common to any GC >>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ?*) shared-serviceability >>>>> >>>>> ??? - The usual code to support another GC >>>>> >>>>> ?*) shared-runtime >>>>> >>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>> ??? - Warning and disabling JFR LeakProfiler >>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ??? - The usual macros in macro.hpp >>>>> >>>>> ?*) shared-build >>>>> >>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> ????? don't) >>>>> >>>>> ?*) shared-tests >>>>> >>>>> ??? - Test infrastructure to support Shenandoah >>>>> ??? - Shenandoah test groups >>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> ?*) shenandoah-tests >>>>> >>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> ??? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ? - {Windows} x {x86_64}, >>>>> ? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ? - the new Shenandoah tests >>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Dec 7 13:09:08 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 7 Dec 2018 08:09:08 -0500 Subject: RFR (S) 8214972: Uses of klass_holder() except GC need to apply GC barriers In-Reply-To: <0d7642c1-c6a1-4b2a-bad0-7135c33a1cfd@oracle.com> References: <0d7642c1-c6a1-4b2a-bad0-7135c33a1cfd@oracle.com> Message-ID: <2145ba81-17aa-078b-b21b-fb4d1f596247@oracle.com> On 12/6/18 10:29 PM, dean.long at oracle.com wrote: > It looks like we no longer use java_mirror() for is_unsafe_anonymous() > classes? Hi Dean,? I'm glad you looked at this.? The ClassLoaderData::_holder is the java mirror now, so we only have to look at that, and mark it for the concurrent collectors. Coleen > > dl > > > On 12/6/18 12:47 PM, coleen.phillimore at oracle.com wrote: >> Summary: Fix klass_holder() and make all callers use it, remove >> holder_phantom().* >> >> *This removes the confusion about which one to use, except for GC, >> who has to know. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8214972.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8214972 >> >> Ran tier1-3 tests. >> >> Thanks, >> Coleen >> ** > From manc at google.com Fri Dec 7 14:42:31 2018 From: manc at google.com (Man Cao) Date: Fri, 7 Dec 2018 22:42:31 +0800 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: Message-ID: Hi Thomas, Thanks for the review. No worries about the delay. Responding to some comments below, will address other comments soon. - not sure about the usefulness of the OverheadTester interface. At > least in the current implementation its implementation could be easily > replaced by static functions returning a bool. That would save a lot of > boiler plate code. Is there some reason to have explicit classes for > that? The OverheadTester interface provides a way to pass functions as parameters to OverheadChecker::check_gc_overhead_limit(), and hide details about what parameters are required to compute is_exceeded(). We could change OverheadChecker::check_gc_overhead_limit() to take "bool time_overhead_exceeded, bool space_overhead_exceeded" parameters instead of two pointers to OverheadTester. Then we can have two static functions instead of the OverheadTester interface. However, it would introduce a small behavior change for AdaptiveSizePolicy::check_gc_overhead_limit(). Specifically, it would compute whether time and space overhead limits are exceeded every time the function is called. Currently it only needs to compute these under the following condition: !GCCause::is_user_requested_gc(gc_cause) && !GCCause::is_serviceability_requested_gc(gc_cause) && is_full_gc If this behavior change is OK, I can replace the OverheadTester interface. Assuming that all collectors want to implement this, and actually need > to I am leaning towards doing so. However the ZGC/Shenandoah people > might object to this. I think it is reasonable for all collectors to support this, but the low-pause collectors may have a very different definition for GC overhead. I also received internal user complaints that the current UseGCOverheadLimit for ParallelGC and CMS is too difficult to get triggered, because of the requirement for 5 consecutive full GCs. It might also be a problem for G1, as it avoids full GCs eagerly. I think we might need to change the 5-full-GC requirement in the future to make it more useful. Okay. However I do not know how to rename them and what are the > requirements here. Probably a CSR is needed, although I have seen > translation tables used (in > src/jdk.internal.jvmstat/share/classes/sun/jvmstat/perfdata/resources/a > liasmap). I cc'ed hotspot-dev in the assumption somebody else there > knows more about this. If the process is too complicated, I can live with the current imperfect name. It is not causing much ambiguity. I can just move it's definition to GCPolicyCounters. Keeping its name unchanged also reduces users' work when they upgrade to new JDK, if they are already monitoring this counter using their own tools. In fact, the more useful counter for monitoring purpose is the raw value for gc_cost(). We have a local patch to export it for ParallelGC and CMS, but it doesn't work for G1 yet. Perhaps we can upstream it in the future when it works for G1. -Man From dean.long at oracle.com Fri Dec 7 19:58:03 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 7 Dec 2018 11:58:03 -0800 Subject: RFR (S) 8214972: Uses of klass_holder() except GC need to apply GC barriers In-Reply-To: <2145ba81-17aa-078b-b21b-fb4d1f596247@oracle.com> References: <0d7642c1-c6a1-4b2a-bad0-7135c33a1cfd@oracle.com> <2145ba81-17aa-078b-b21b-fb4d1f596247@oracle.com> Message-ID: On 12/7/18 5:09 AM, coleen.phillimore at oracle.com wrote: > > > On 12/6/18 10:29 PM, dean.long at oracle.com wrote: >> It looks like we no longer use java_mirror() for >> is_unsafe_anonymous() classes? > > Hi Dean,? I'm glad you looked at this.? The ClassLoaderData::_holder > is the java mirror now, so we only have to look at that, and mark it > for the concurrent collectors. > OK, I missed the earlier ClassLoaderData::_holder change.? This change looks good. dl > Coleen > >> >> dl >> >> >> On 12/6/18 12:47 PM, coleen.phillimore at oracle.com wrote: >>> Summary: Fix klass_holder() and make all callers use it, remove >>> holder_phantom().* >>> >>> *This removes the confusion about which one to use, except for GC, >>> who has to know. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8214972.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8214972 >>> >>> Ran tier1-3 tests. >>> >>> Thanks, >>> Coleen >>> ** >> > From coleen.phillimore at oracle.com Fri Dec 7 20:05:08 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 7 Dec 2018 15:05:08 -0500 Subject: RFR (S) 8214972: Uses of klass_holder() except GC need to apply GC barriers In-Reply-To: References: <0d7642c1-c6a1-4b2a-bad0-7135c33a1cfd@oracle.com> <2145ba81-17aa-078b-b21b-fb4d1f596247@oracle.com> Message-ID: <8fd48fd2-49c9-e242-da31-4f72fae1e7d4@oracle.com> On 12/7/18 2:58 PM, dean.long at oracle.com wrote: > On 12/7/18 5:09 AM, coleen.phillimore at oracle.com wrote: >> >> >> On 12/6/18 10:29 PM, dean.long at oracle.com wrote: >>> It looks like we no longer use java_mirror() for >>> is_unsafe_anonymous() classes? >> >> Hi Dean,? I'm glad you looked at this.? The ClassLoaderData::_holder >> is the java mirror now, so we only have to look at that, and mark it >> for the concurrent collectors. >> > > OK, I missed the earlier ClassLoaderData::_holder change.? This change > looks good. Thank you very much Dean! Coleen > > dl > >> Coleen >> >>> >>> dl >>> >>> >>> On 12/6/18 12:47 PM, coleen.phillimore at oracle.com wrote: >>>> Summary: Fix klass_holder() and make all callers use it, remove >>>> holder_phantom().* >>>> >>>> *This removes the confusion about which one to use, except for GC, >>>> who has to know. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8214972.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8214972 >>>> >>>> Ran tier1-3 tests. >>>> >>>> Thanks, >>>> Coleen >>>> ** >>> >> > From kim.barrett at oracle.com Fri Dec 7 20:07:00 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Dec 2018 15:07:00 -0500 Subject: RFR: 8214315: G1: fatal error: acquiring lock SATB_Q_FL_lock/1 out of order with lock tty_lock/0 In-Reply-To: References: Message-ID: <860905A8-D943-43E1-8C44-DC8FB069046A@oracle.com> > On Dec 7, 2018, at 5:54 AM, Thomas Schatzl wrote: >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8214997 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8214315/open.00/ >> >> Testing: >> mach5 tier1-5 >> > > looks good. > > Thomas Thanks From kim.barrett at oracle.com Fri Dec 7 20:07:17 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Dec 2018 15:07:17 -0500 Subject: RFR: 8214315: G1: fatal error: acquiring lock SATB_Q_FL_lock/1 out of order with lock tty_lock/0 In-Reply-To: References: Message-ID: <411CF1AA-C812-4AA8-876D-2495CD2C7B55@oracle.com> > On Dec 7, 2018, at 2:38 AM, Erik ?sterlund wrote: > > Hi Kim, > > Looks good. > > Thanks, > /Erik > > On 2018-12-07 07:31, Kim Barrett wrote: >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8214997 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8214315/open.00/ >> Testing: >> mach5 tier1-5 Thanks. From amith.pawar at gmail.com Fri Dec 7 23:20:58 2018 From: amith.pawar at gmail.com (amith pawar) Date: Sat, 8 Dec 2018 04:50:58 +0530 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> Message-ID: Hi Thomas, Please check my inline comments and attached patch is updated as per your suggestion. On Mon, Dec 3, 2018 at 6:12 PM Thomas Schatzl wrote: > Hi, > > On Fri, 2018-11-30 at 11:11 +0100, Thomas Schatzl wrote: > > Hi Amit, > > > > [...] > > > > > > Sorry. Please check the attached patch. > > > > I created a webrev at > > http://cr.openjdk.java.net/~tschatzl/8213827/webrev/ . > > > > Let the reviews begin :) > > > > style issues: > > - random whitespaces in many places. > - all if's or if-then-else require braces > - only class members have a leading underscore > - do not declare local variables long before they are used. That > makes the code hard to read. > - do not introduce local variables for no reason. E.g. > > 2907 struct bitmask *bmp; > > [...unrelated code...] > > 2911 bmp = _numa_get_interleave_mask(); > 2912 set_numa_interleave_ptr(&bmp); > 2913 bmp = _numa_get_membind(); > 2914 set_numa_membind_ptr(&bmp); > > is much less readable than > > set_numa_interleave_ptr(_numa_get_interleave_mask()); > set_numa_membind_ptr(_numa_get_membind_mask()); > > - in the type declaration, please put the * next to the type. While > this is kind-of wrong, the majority of existing code does so (not only > in these files). > > code issues: > > - interleave/membind mode are mutually exclusive, which the code > kind-of tries to establish (and check) using special values for eg. the > value of Linux::_numa_interleave_ptr. > > I think the code gets a lot more readable if you introduce an enum and > an explicit global variable holding the current NUMA node. If you do > not want to do that, move the condition "Linux::_numa_interleave_ptr != > 0" into a new method. > > - move the new NUMA initialization code in os::init_2() into a new > method please. It's gotten very long now. > > - please avoid duplicating code; ie. > > 5039 // Check for membind mode. > 5040 bmp = Linux::_numa_membind_ptr; > 5041 for (int node = 0; node <= Linux::numa_max_node() ; node++) > { > 5042 if (Linux::_numa_bitmask_isbitset(bmp, node)) { > 5043 _is_membind = true; > 5044 } > 5045 } > > 5048 bmp = Linux::_numa_interleave_ptr; > 5049 for (int node = 0; node <= Linux::numa_max_node() ; node++) > { > 5050 if (Linux::_numa_bitmask_isbitset(bmp, node)) { > 5051 _is_interleaved = true; > 5052 // Set membind to false as interleave mode allows all > nodes to be used. > 5053 _is_membind = false; > 5054 } > 5055 } > > and right after: > > Something like: > > is_interleave = is_any_bit_in_bitmask_set(Linux::_numa_interleave_ptr); > is_membind = !is_interleave && > is_any_bit_in_bitmask_set(Linux::_numa_membind_ptr); > > with an appropriate is_any_bit_in_bitmask_set() helper method is much > more clear. > > (Even better, as suggested above use a global "NUMA mode" enum that is > filled in somewhere instead of multiple flags) > > - Use a LogStream to print the log messages instead of rolling your > own code to create a string containing the node number. The code that > is there exhibits a potential buffer overflow (with many NUMA nodes), > and a potential out-of-bounds write. > > - the log message for selected NUMA mode and nodes should be a single > log message. > > - partially pre-existing: the numa_set_xxx_ptr methods take a > "struct bitmask**". Could you explain why "**"? All of those store the > dereferenced result anyway, so why not pass the dereferenced pointer > right away? > Following bitmask pointers are defined in libnuma library. 'dlvsym' will return the address of pointer which needs to be de-referenced. Updated other methods as suggested. struct bitmask *numa_all_nodes_ptr; struct bitmask *numa_no_nodes_ptr; struct bitmask *numa_all_cpus_ptr; > > This also simplifies the code at the callers. > > - partially pre-existing: I would prefer if the numa_set_xx_ptr > methods were all renamed to numa_set_xx_bitmask. The "ptr" suffix > seems redundant, and most importantly very unspecific. > Newly introduced membind and interleave bitmask pointer names are changed. Also above mentioned three pointers are imported. If you suggest will rename these also. > > - in numa_interleave_memory(), why does the code not make the > membind/interleave mode distinction when the v1 API is used? > v2 API's works as expected for membind/interleave case but numa_get_membind_v1 API's always returns null . v1 and v2 prototypes are also not same. Please check below. nodemask_t numa_get_membind_v1(void) struct bitmask * numa_get_membind_v2(void) void numa_interleave_memory_v1(void *mem, size_t size, const nodemask_t *mask) void numa_interleave_memory_v2(void *mem, size_t size, struct bitmask *bmp) > > That method could probably cleaned up a bit, i.e. extracting common > code. > > - Pre-existing: numa_interleave_memory() could be cleaned up; why > do we have two pointers to _numa_interleave_memory_v2 and > _numa_interleave_memory around if their parameters are the same and > we only ever use one of them? > As explained above, API prototypes are not same so this is required. > > That might be an additional cleanup that needs to be done. > > - please split out the change that disables UseNUMA if there is > a single node only. It seems relatively unrelated, and so should imho > be looked at (evaluated) separately. > > While reviewing I already fixed a few of the above issues, see > > http://cr.openjdk.java.net/~tschatzl/8213827/webrev.0_to_1 (diff) > http://cr.openjdk.java.net/~tschatzl/8213827/webrev.1 (full) > > webrevs. You can use it as starting point. > > Thanks, > Thomas > > -- With best regards, amit pawar -------------- next part -------------- diff -r d537553ed639 src/hotspot/os/linux/os_linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp Wed Nov 28 22:29:35 2018 -0500 +++ b/src/hotspot/os/linux/os_linux.cpp Sat Dec 08 05:20:06 2018 +0530 @@ -33,6 +33,7 @@ #include "compiler/disassembler.hpp" #include "interpreter/interpreter.hpp" #include "logging/log.hpp" +#include "logging/logStream.hpp" #include "memory/allocation.inline.hpp" #include "memory/filemap.hpp" #include "oops/oop.inline.hpp" @@ -2888,11 +2889,15 @@ libnuma_dlsym(handle, "numa_distance"))); set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, libnuma_v2_dlsym(handle, "numa_get_membind"))); + set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, + libnuma_v2_dlsym(handle, "numa_get_interleave_mask"))); if (numa_available() != -1) { set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); set_numa_all_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_all_nodes_ptr")); set_numa_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_nodes_ptr")); + set_numa_interleave_bitmask(_numa_get_interleave_mask()); + set_numa_membind_bitmask(_numa_get_membind()); // Create an index -> node mapping, since nodes are not always consecutive _nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) GrowableArray(0, true); rebuild_nindex_to_node_map(); @@ -3019,9 +3024,13 @@ os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; os::Linux::numa_distance_func_t os::Linux::_numa_distance; os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; +os::Linux::numa_get_interleave_mask_func_t os::Linux::_numa_get_interleave_mask; +os::Linux::Numa_allocation_policy os::Linux::_current_numa_policy; unsigned long* os::Linux::_numa_all_nodes; struct bitmask* os::Linux::_numa_all_nodes_ptr; struct bitmask* os::Linux::_numa_nodes_ptr; +struct bitmask* os::Linux::_numa_interleave_bitmask; +struct bitmask* os::Linux::_numa_membind_bitmask; bool os::pd_uncommit_memory(char* addr, size_t size) { uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, @@ -4958,6 +4967,75 @@ OSContainer::init(); } +void os::Linux::numa_init() { + + // Java can be invoked as + // 1. Without numactl and heap will be allocated/configured on all nodes as + // per the system policy. + // 2. With numactl --interleave: + // Use numa_get_interleave_mask(v2) API to get nodes bitmask. The same + // API for membind case bitmask is reset. + // Interleave is only hint and Kernel can fallback to other nodes if + // no memory is available on the target nodes. + // 3. With numactl --membind: + // Use numa_get_membind(v2) API to get nodes bitmask. The same API for + // interleave case returns bitmask of all nodes. + // numa_all_nodes_ptr holds bitmask of all nodes. + // numa_get_interleave_mask(v2) and numa_get_membind(v2) APIs returns correct + // bitmask when externally configured to run on all or fewer nodes. + + if (!Linux::libnuma_init()) { + UseNUMA = false; + } else { + if ((Linux::numa_max_node() < 1) || Linux::isbound_to_single_node()) { + // If there's only one node (they start from 0) or if the process + // is bound explicitly to a single node using membind, disable NUMA. + UseNUMA = false; + } else { + + LogTarget(Info,os) log; + LogStream ls(log); + outputStream *st=&ls; + + Linux::set_configured_numa_policy (Linux::identify_numa_policy()); + + struct bitmask* bmp = Linux::_numa_membind_bitmask; + const char* numa_mode = "membind"; + + if (Linux::isrunning_in_interleave_mode()) { + bmp = Linux::_numa_interleave_bitmask; + numa_mode = "interleave"; + } + + st->print("UseNUMA is enabled and invoked in '%s' mode." + " Heap will be configured using NUMA memory nodes:", numa_mode); + + for (int node = 0; node < Linux::numa_max_node(); node++) { + if (Linux::_numa_bitmask_isbitset(bmp, node)) { + st->print(" %d", node); + } + } + } + } + + if (UseParallelGC && UseNUMA && UseLargePages && !can_commit_large_page_memory()) { + // With SHM and HugeTLBFS large pages we cannot uncommit a page, so there's no way + // we can make the adaptive lgrp chunk resizing work. If the user specified both + // UseNUMA and UseLargePages (or UseSHM/UseHugeTLBFS) on the command line - warn + // and disable adaptive resizing. + if (UseAdaptiveSizePolicy || UseAdaptiveNUMAChunkSizing) { + warning("UseNUMA is not fully compatible with SHM/HugeTLBFS large pages, " + "disabling adaptive resizing (-XX:-UseAdaptiveSizePolicy -XX:-UseAdaptiveNUMAChunkSizing)"); + UseAdaptiveSizePolicy = false; + UseAdaptiveNUMAChunkSizing = false; + } + } + + if (!UseNUMA && ForceNUMA) { + UseNUMA = true; + } +} + // this is called _after_ the global arguments have been parsed jint os::init_2(void) { @@ -5002,32 +5080,7 @@ Linux::glibc_version(), Linux::libpthread_version()); if (UseNUMA) { - if (!Linux::libnuma_init()) { - UseNUMA = false; - } else { - if ((Linux::numa_max_node() < 1) || Linux::isbound_to_single_node()) { - // If there's only one node (they start from 0) or if the process - // is bound explicitly to a single node using membind, disable NUMA. - UseNUMA = false; - } - } - - if (UseParallelGC && UseNUMA && UseLargePages && !can_commit_large_page_memory()) { - // With SHM and HugeTLBFS large pages we cannot uncommit a page, so there's no way - // we can make the adaptive lgrp chunk resizing work. If the user specified both - // UseNUMA and UseLargePages (or UseSHM/UseHugeTLBFS) on the command line - warn - // and disable adaptive resizing. - if (UseAdaptiveSizePolicy || UseAdaptiveNUMAChunkSizing) { - warning("UseNUMA is not fully compatible with SHM/HugeTLBFS large pages, " - "disabling adaptive resizing (-XX:-UseAdaptiveSizePolicy -XX:-UseAdaptiveNUMAChunkSizing)"); - UseAdaptiveSizePolicy = false; - UseAdaptiveNUMAChunkSizing = false; - } - } - - if (!UseNUMA && ForceNUMA) { - UseNUMA = true; - } + Linux::numa_init(); } if (MaxFDLimit) { diff -r d537553ed639 src/hotspot/os/linux/os_linux.hpp --- a/src/hotspot/os/linux/os_linux.hpp Wed Nov 28 22:29:35 2018 -0500 +++ b/src/hotspot/os/linux/os_linux.hpp Sat Dec 08 05:20:06 2018 +0530 @@ -211,6 +211,7 @@ // none present private: + static void numa_init(); static void expand_stack_to(address bottom); typedef int (*sched_getcpu_func_t)(void); @@ -222,6 +223,7 @@ typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); typedef struct bitmask* (*numa_get_membind_func_t)(void); + typedef struct bitmask* (*numa_get_interleave_mask_func_t)(void); typedef void (*numa_set_bind_policy_func_t)(int policy); typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); @@ -239,9 +241,12 @@ static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; static numa_distance_func_t _numa_distance; static numa_get_membind_func_t _numa_get_membind; + static numa_get_interleave_mask_func_t _numa_get_interleave_mask; static unsigned long* _numa_all_nodes; static struct bitmask* _numa_all_nodes_ptr; static struct bitmask* _numa_nodes_ptr; + static struct bitmask* _numa_interleave_bitmask; + static struct bitmask* _numa_membind_bitmask; static void set_sched_getcpu(sched_getcpu_func_t func) { _sched_getcpu = func; } static void set_numa_node_to_cpus(numa_node_to_cpus_func_t func) { _numa_node_to_cpus = func; } @@ -255,10 +260,21 @@ static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } + static void set_numa_get_interleave_mask(numa_get_interleave_mask_func_t func) { _numa_get_interleave_mask = func; } static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } + static void set_numa_interleave_bitmask(struct bitmask* ptr) { _numa_interleave_bitmask = ptr ; } + static void set_numa_membind_bitmask(struct bitmask* ptr) { _numa_membind_bitmask = ptr ; } static int sched_getcpu_syscall(void); + + enum Numa_allocation_policy{ + not_initialized, + membind, + interleave + }; + static Numa_allocation_policy _current_numa_policy; + public: static int sched_getcpu() { return _sched_getcpu != NULL ? _sched_getcpu() : -1; } static int numa_node_to_cpus(int node, unsigned long *buffer, int bufferlen) { @@ -272,11 +288,35 @@ static int numa_tonode_memory(void *start, size_t size, int node) { return _numa_tonode_memory != NULL ? _numa_tonode_memory(start, size, node) : -1; } + + static bool isrunning_in_interleave_mode() { + return _current_numa_policy == interleave ? true : false; + } + + static void set_configured_numa_policy(Numa_allocation_policy numa_policy) { + _current_numa_policy = numa_policy ; + } + + static Numa_allocation_policy identify_numa_policy() { + Numa_allocation_policy current_policy = membind; + for (int node = 0; node <= Linux::numa_max_node() ; node++) { + if (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_bitmask, node)) { + current_policy = interleave; + } + } + return current_policy ; + } + static void numa_interleave_memory(void *start, size_t size) { // Use v2 api if available - if (_numa_interleave_memory_v2 != NULL && _numa_all_nodes_ptr != NULL) { - _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr); - } else if (_numa_interleave_memory != NULL && _numa_all_nodes != NULL) { + if (_numa_interleave_memory_v2 != NULL && _numa_membind_bitmask != NULL) { + // Use interleave bitmask while running interleave mode. + if (isrunning_in_interleave_mode()) { + _numa_interleave_memory_v2(start, size, _numa_interleave_bitmask); + } else if (_numa_membind_bitmask != NULL) { + _numa_interleave_memory_v2(start, size, _numa_membind_bitmask); + } + } else { _numa_interleave_memory(start, size, _numa_all_nodes); } } @@ -318,11 +358,14 @@ } // Check if node is in bound node set. static bool isnode_in_bound_nodes(int node) { - if (_numa_get_membind != NULL && _numa_bitmask_isbitset != NULL) { - return _numa_bitmask_isbitset(_numa_get_membind(), node); - } else { - return false; + if (_numa_bitmask_isbitset != NULL) { + if (isrunning_in_interleave_mode()) { + return _numa_bitmask_isbitset(_numa_interleave_bitmask, node); + } else { + return _numa_membind_bitmask != NULL ? _numa_bitmask_isbitset(_numa_membind_bitmask, node) : false; + } } + return false; } // Check if bound to only one numa node. // Returns true if bound to a single numa node, otherwise returns false. From merkel05 at gmail.com Sat Dec 8 01:19:52 2018 From: merkel05 at gmail.com (Sergey) Date: Sat, 8 Dec 2018 02:19:52 +0100 Subject: [PATCH][TESTBUG] JDK-8214052: Wrong shell used In-Reply-To: References: Message-ID: Hi everyone, Could anyone please review that tiny fix for test script below? That would be extremely great if anyone could sponsor the change as well. Thanks and regards, Sergei On Thu, Dec 6, 2018, 08:37 Sergey Hi, > > I've posted the original mail to the wrong mailing list and thanks to > David for guidance. > > Problem goes as follows: as stated in the bug description [1] there are few > places in the tests where extended bash syntax is being used. > > First one is (as per ticket in the headline): > > - vmTestbase/vm/compiler/CodeCacheInfoOnCompilation > > I've made a quick fix for that one and patch could be found > inlined below. > > Moreover what I've noticed is that grep gives me > another test script: > > - GeneratePropertyPassword.sh [2] > > that seems to be linked with another ticket [3] which is closed, > though it still uses the bash "[[" extension. Perhaps this [4] fix > needs to be reapplied. > > In regards to the patch below, please let me know if I've missed > something. Changes were successfully tested with `dash v0.5.10.2-1`. > > [1] https://bugs.openjdk.java.net/browse/JDK-8214052 > [2] > http://hg.openjdk.java.net/jdk/sandbox/file/413c28945e0f/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh#l37 > [3] https://bugs.openjdk.java.net/browse/JDK-8025886 > [4] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/a2551e12a9ea0.5.10.2-1 > > Regards, > su - > > diff --git > a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > --- > a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > +++ > b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > @@ -27,7 +27,7 @@ > > ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) > isComp=$( echo ${ver} | grep -c "compiled mode") > -if [[ $isComp != 1 ]]; then > +if [ "$isComp" -ne 1 ]; then > echo "skipped. This test works only with -Xcomp" > exit > fi > @@ -40,7 +40,7 @@ > res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} > -XX:+PrintCodeCacheOnCompilation -XX:-Inline > vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce "${pattern}") > echo "res: " ${res} > > -if (( "${res}" != "0" )); then > +if [ "$res" != "0" ]; then > echo "passed" > true > else > From jesper.wilhelmsson at oracle.com Sat Dec 8 03:10:06 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Sat, 8 Dec 2018 04:10:06 +0100 Subject: [PATCH][TESTBUG] JDK-8214052: Wrong shell used In-Reply-To: References: Message-ID: Hi, Why do you replace != with -ne in one case and not in the other? Thanks, /Jesper > On 8 Dec 2018, at 02:19, Sergey wrote: > > Hi everyone, > > Could anyone please review that tiny fix for test script below? > That would be extremely great if anyone could sponsor the change as well. > > Thanks and regards, > Sergei > > On Thu, Dec 6, 2018, 08:37 Sergey >> Hi, >> >> I've posted the original mail to the wrong mailing list and thanks to >> David for guidance. >> >> Problem goes as follows: as stated in the bug description [1] there are few >> places in the tests where extended bash syntax is being used. >> >> First one is (as per ticket in the headline): >> >> - vmTestbase/vm/compiler/CodeCacheInfoOnCompilation >> >> I've made a quick fix for that one and patch could be found >> inlined below. >> >> Moreover what I've noticed is that grep gives me >> another test script: >> >> - GeneratePropertyPassword.sh [2] >> >> that seems to be linked with another ticket [3] which is closed, >> though it still uses the bash "[[" extension. Perhaps this [4] fix >> needs to be reapplied. >> >> In regards to the patch below, please let me know if I've missed >> something. Changes were successfully tested with `dash v0.5.10.2-1`. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8214052 >> [2] >> http://hg.openjdk.java.net/jdk/sandbox/file/413c28945e0f/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh#l37 >> [3] https://bugs.openjdk.java.net/browse/JDK-8025886 >> [4] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/a2551e12a9ea0.5.10.2-1 >> >> Regards, >> su - >> >> diff --git >> a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> --- >> a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> +++ >> b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> @@ -27,7 +27,7 @@ >> >> ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) >> isComp=$( echo ${ver} | grep -c "compiled mode") >> -if [[ $isComp != 1 ]]; then >> +if [ "$isComp" -ne 1 ]; then >> echo "skipped. This test works only with -Xcomp" >> exit >> fi >> @@ -40,7 +40,7 @@ >> res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} >> -XX:+PrintCodeCacheOnCompilation -XX:-Inline >> vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce "${pattern}") >> echo "res: " ${res} >> >> -if (( "${res}" != "0" )); then >> +if [ "$res" != "0" ]; then >> echo "passed" >> true >> else >> From merkel05 at gmail.com Sat Dec 8 03:24:31 2018 From: merkel05 at gmail.com (Sergey) Date: Sat, 8 Dec 2018 04:24:31 +0100 Subject: [PATCH][TESTBUG] JDK-8214052: Wrong shell used In-Reply-To: References: Message-ID: Hi Jesper, I?ve double checked with ?test?s man page http://man7.org/linux/man-pages/man1/test.1.html In particular this part: STRING1 != STRING2 the strings are not equal INTEGER1 *-ne *INTEGER2 INTEGER1 is not equal to INTEGER2 and applied those respectively. Do you think I better use ?!=? in both places? Regards, Sergei On Sat, 8 Dec 2018 at 04:10, wrote: > Hi, > > Why do you replace != with -ne in one case and not in the other? > > Thanks, > /Jesper > > > On 8 Dec 2018, at 02:19, Sergey wrote: > > > > Hi everyone, > > > > Could anyone please review that tiny fix for test script below? > > That would be extremely great if anyone could sponsor the change as well. > > > > Thanks and regards, > > Sergei > > > > On Thu, Dec 6, 2018, 08:37 Sergey > > >> Hi, > >> > >> I've posted the original mail to the wrong mailing list and thanks to > >> David for guidance. > >> > >> Problem goes as follows: as stated in the bug description [1] there are > few > >> places in the tests where extended bash syntax is being used. > >> > >> First one is (as per ticket in the headline): > >> > >> - vmTestbase/vm/compiler/CodeCacheInfoOnCompilation > >> > >> I've made a quick fix for that one and patch could be found > >> inlined below. > >> > >> Moreover what I've noticed is that grep gives me > >> another test script: > >> > >> - GeneratePropertyPassword.sh [2] > >> > >> that seems to be linked with another ticket [3] which is closed, > >> though it still uses the bash "[[" extension. Perhaps this [4] fix > >> needs to be reapplied. > >> > >> In regards to the patch below, please let me know if I've missed > >> something. Changes were successfully tested with `dash v0.5.10.2-1`. > >> > >> [1] https://bugs.openjdk.java.net/browse/JDK-8214052 > >> [2] > >> > http://hg.openjdk.java.net/jdk/sandbox/file/413c28945e0f/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh#l37 > >> [3] https://bugs.openjdk.java.net/browse/JDK-8025886 > >> [4] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/a2551e12a9ea0.5.10.2-1 > >> > >> Regards, > >> su - > >> > >> diff --git > >> > a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> > b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> --- > >> > a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> +++ > >> > b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> @@ -27,7 +27,7 @@ > >> > >> ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) > >> isComp=$( echo ${ver} | grep -c "compiled mode") > >> -if [[ $isComp != 1 ]]; then > >> +if [ "$isComp" -ne 1 ]; then > >> echo "skipped. This test works only with -Xcomp" > >> exit > >> fi > >> @@ -40,7 +40,7 @@ > >> res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} > >> -XX:+PrintCodeCacheOnCompilation -XX:-Inline > >> vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce > "${pattern}") > >> echo "res: " ${res} > >> > >> -if (( "${res}" != "0" )); then > >> +if [ "$res" != "0" ]; then > >> echo "passed" > >> true > >> else > >> > > From merkel05 at gmail.com Sat Dec 8 03:46:43 2018 From: merkel05 at gmail.com (Sergey) Date: Sat, 8 Dec 2018 04:46:43 +0100 Subject: [PATCH][TESTBUG] JDK-8214052: Wrong shell used In-Reply-To: References: Message-ID: Hi Jesper, I've missed that the $res variable is assigned as a result of `egrep -ce` so it is safe to treat both cases as integers. I hope I'm not overthinking here. Anyway, I've changed both places to use "-ne", please see updated patch inlined below diff --git a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh --- a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh +++ b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh @@ -27,7 +27,7 @@ ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) isComp=$( echo ${ver} | grep -c "compiled mode") -if [[ $isComp != 1 ]]; then +if [ "$isComp" -ne 1 ]; then echo "skipped. This test works only with -Xcomp" exit fi @@ -40,7 +40,7 @@ res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -XX:+PrintCodeCacheOnCompilation -XX:-Inline vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce "${pattern}") echo "res: " ${res} -if (( "${res}" != "0" )); then +if [ "$res" -ne 0 ]; then echo "passed" true else Regards, Sergei On Sat, 8 Dec 2018 at 04:24, Sergey wrote: > Hi Jesper, > > I?ve double checked with ?test?s man page > http://man7.org/linux/man-pages/man1/test.1.html > In particular this part: > > STRING1 != STRING2 > the strings are not equal > > INTEGER1 *-ne *INTEGER2 > INTEGER1 is not equal to INTEGER2 > > and applied those respectively. > > Do you think I better use ?!=? in both places? > > Regards, > Sergei > > On Sat, 8 Dec 2018 at 04:10, wrote: > >> Hi, >> >> Why do you replace != with -ne in one case and not in the other? >> >> Thanks, >> /Jesper >> >> > On 8 Dec 2018, at 02:19, Sergey wrote: >> > >> > Hi everyone, >> > >> > Could anyone please review that tiny fix for test script below? >> > That would be extremely great if anyone could sponsor the change as >> well. >> > >> > Thanks and regards, >> > Sergei >> > >> > On Thu, Dec 6, 2018, 08:37 Sergey > > >> >> Hi, >> >> >> >> I've posted the original mail to the wrong mailing list and thanks to >> >> David for guidance. >> >> >> >> Problem goes as follows: as stated in the bug description [1] there >> are few >> >> places in the tests where extended bash syntax is being used. >> >> >> >> First one is (as per ticket in the headline): >> >> >> >> - vmTestbase/vm/compiler/CodeCacheInfoOnCompilation >> >> >> >> I've made a quick fix for that one and patch could be found >> >> inlined below. >> >> >> >> Moreover what I've noticed is that grep gives me >> >> another test script: >> >> >> >> - GeneratePropertyPassword.sh [2] >> >> >> >> that seems to be linked with another ticket [3] which is closed, >> >> though it still uses the bash "[[" extension. Perhaps this [4] fix >> >> needs to be reapplied. >> >> >> >> In regards to the patch below, please let me know if I've missed >> >> something. Changes were successfully tested with `dash v0.5.10.2-1`. >> >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8214052 >> >> [2] >> >> >> http://hg.openjdk.java.net/jdk/sandbox/file/413c28945e0f/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh#l37 >> >> [3] https://bugs.openjdk.java.net/browse/JDK-8025886 >> >> [4] >> http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/a2551e12a9ea0.5.10.2-1 >> >> >> >> Regards, >> >> su - >> >> >> >> diff --git >> >> >> a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> >> >> b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> >> --- >> >> >> a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> >> +++ >> >> >> b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh >> >> @@ -27,7 +27,7 @@ >> >> >> >> ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) >> >> isComp=$( echo ${ver} | grep -c "compiled mode") >> >> -if [[ $isComp != 1 ]]; then >> >> +if [ "$isComp" -ne 1 ]; then >> >> echo "skipped. This test works only with -Xcomp" >> >> exit >> >> fi >> >> @@ -40,7 +40,7 @@ >> >> res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} >> >> -XX:+PrintCodeCacheOnCompilation -XX:-Inline >> >> vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce >> "${pattern}") >> >> echo "res: " ${res} >> >> >> >> -if (( "${res}" != "0" )); then >> >> +if [ "$res" != "0" ]; then >> >> echo "passed" >> >> true >> >> else >> >> >> >> From jesper.wilhelmsson at oracle.com Sat Dec 8 04:00:13 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Sat, 8 Dec 2018 05:00:13 +0100 Subject: [PATCH][TESTBUG] JDK-8214052: Wrong shell used In-Reply-To: References: Message-ID: <0CF64BE4-2327-4283-8E6A-6293E44716AA@oracle.com> > On 8 Dec 2018, at 04:46, Sergey wrote: > > Hi Jesper, > > I've missed that the $res variable is assigned as a result of > `egrep -ce` so it is safe to treat both cases as integers. > > I hope I'm not overthinking here. Anyway, I've changed > both places to use "-ne", please see updated patch inlined > below Yes, both places are comparing the result of a grep -c, so it seems they should be handled the same way. Wether it's using string comparison or integer comparison doesn't really matter to me, as long as they are the same. Thank you for providing this fix! The latest patch looks good to me. I'll sponsor it. /Jesper > > > diff --git a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > --- a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > +++ b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > @@ -27,7 +27,7 @@ > > ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) > isComp=$( echo ${ver} | grep -c "compiled mode") > -if [[ $isComp != 1 ]]; then > +if [ "$isComp" -ne 1 ]; then > echo "skipped. This test works only with -Xcomp" > exit > fi > @@ -40,7 +40,7 @@ > res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -XX:+PrintCodeCacheOnCompilation -XX:-Inline vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce "${pattern}") > echo "res: " ${res} > > -if (( "${res}" != "0" )); then > +if [ "$res" -ne 0 ]; then > echo "passed" > true > else > > Regards, > Sergei > > On Sat, 8 Dec 2018 at 04:24, Sergey > wrote: > Hi Jesper, > > I?ve double checked with ?test?s man page > http://man7.org/linux/man-pages/man1/test.1.html > In particular this part: > > STRING1 != STRING2 > the strings are not equal > > INTEGER1 -ne INTEGER2 > INTEGER1 is not equal to INTEGER2 > and applied those respectively. > > Do you think I better use ?!=? in both places? > > Regards, > Sergei > > On Sat, 8 Dec 2018 at 04:10, > wrote: > Hi, > > Why do you replace != with -ne in one case and not in the other? > > Thanks, > /Jesper > > > On 8 Dec 2018, at 02:19, Sergey > wrote: > > > > Hi everyone, > > > > Could anyone please review that tiny fix for test script below? > > That would be extremely great if anyone could sponsor the change as well. > > > > Thanks and regards, > > Sergei > > > > On Thu, Dec 6, 2018, 08:37 Sergey wrote: > > > >> Hi, > >> > >> I've posted the original mail to the wrong mailing list and thanks to > >> David for guidance. > >> > >> Problem goes as follows: as stated in the bug description [1] there are few > >> places in the tests where extended bash syntax is being used. > >> > >> First one is (as per ticket in the headline): > >> > >> - vmTestbase/vm/compiler/CodeCacheInfoOnCompilation > >> > >> I've made a quick fix for that one and patch could be found > >> inlined below. > >> > >> Moreover what I've noticed is that grep gives me > >> another test script: > >> > >> - GeneratePropertyPassword.sh [2] > >> > >> that seems to be linked with another ticket [3] which is closed, > >> though it still uses the bash "[[" extension. Perhaps this [4] fix > >> needs to be reapplied. > >> > >> In regards to the patch below, please let me know if I've missed > >> something. Changes were successfully tested with `dash v0.5.10.2-1`. > >> > >> [1] https://bugs.openjdk.java.net/browse/JDK-8214052 > >> [2] > >> http://hg.openjdk.java.net/jdk/sandbox/file/413c28945e0f/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh#l37 > >> [3] https://bugs.openjdk.java.net/browse/JDK-8025886 > >> [4] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/a2551e12a9ea0.5.10.2-1 > >> > >> Regards, > >> su - > >> > >> diff --git > >> a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> --- > >> a/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> +++ > >> b/test/hotspot/jtreg/vmTestbase/vm/compiler/CodeCacheInfoOnCompilation/run.sh > >> @@ -27,7 +27,7 @@ > >> > >> ver=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} -version 2>&1) > >> isComp=$( echo ${ver} | grep -c "compiled mode") > >> -if [[ $isComp != 1 ]]; then > >> +if [ "$isComp" -ne 1 ]; then > >> echo "skipped. This test works only with -Xcomp" > >> exit > >> fi > >> @@ -40,7 +40,7 @@ > >> res=$(${TESTED_JAVA_HOME}/bin/java ${JAVA_OPTS} > >> -XX:+PrintCodeCacheOnCompilation -XX:-Inline > >> vm.compiler.CodeCacheInfoOnCompilation.PrintOnCall | egrep -ce "${pattern}") > >> echo "res: " ${res} > >> > >> -if (( "${res}" != "0" )); then > >> +if [ "$res" != "0" ]; then > >> echo "passed" > >> true > >> else > >> > From rkennke at redhat.com Sat Dec 8 11:33:02 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 8 Dec 2018 12:33:02 +0100 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> Zhengyu's and Thomas' TaskQueue stuff landed in jdk/jdk and we integrated it into shenandoah/jdk and in the master patch and webrevs. I'm updating the webrevs in-place in: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ This is going to be the final webrev that I intend to push on Monday, unless somebody stops me or we face serious merge conflicts by then. I've just pushed this to jdk/submit for testing. I will also do more testing locally over the weekend. Thanks everybody for reviewing and helping and being patient! Cheers, Roman > Here comes round 6, possibly/hopefully the last round of webrevs for > upstreaming Shenandoah: > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ > > It incorporates: > - renames vm_operations_shenandoah* to shenandoahVMOperations*, as > suggested by Coleen > - reshuffles gcCause in shared-gc SA as suggested by Per > - print number of threads in SA, as suggested by Jini > - fixed a problem in webrev generation that did not track move of > CriticalNativeStress.java and CriticalNativeArgs.java > > The CSR has been approved, the JEP moved to target jdk12, and I got > positive reviews for all parts. I intend to push this early next week, > unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes > in by then, I'll incorporate it. > > Thanks everybody for reviewing and reviewing and reviewing again ;-) > > Cheers, > Roman > >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build (kvn, erikj, ihse, pliden) >> [ ] shared-gc (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests (lmesnik, pliden) >> [ ] shenandoah-gc >> [ ] shenandoah-tests >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ?*) shenandoah-gc >>>>>> >>>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ?*) shared-gc >>>>>> >>>>>> ??? - This is mostly boilerplate that is common to any GC >>>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>>> fail (next to CMS and G1) >>>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ?*) shared-serviceability >>>>>> >>>>>> ??? - The usual code to support another GC >>>>>> >>>>>> ?*) shared-runtime >>>>>> >>>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>>> hook up with, >>>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ??? - Warning and disabling JFR LeakProfiler >>>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ??? - The usual macros in macro.hpp >>>>>> >>>>>> ?*) shared-build >>>>>> >>>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>> files, which is >>>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>>> regression if we >>>>>> ????? don't) >>>>>> >>>>>> ?*) shared-tests >>>>>> >>>>>> ??? - Test infrastructure to support Shenandoah >>>>>> ??? - Shenandoah test groups >>>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>>> run with it >>>>>> >>>>>> ?*) shenandoah-tests >>>>>> >>>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>> ??? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ? - {Windows} x {x86_64}, >>>>>> ? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ? - the new Shenandoah tests >>>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> > From rkennke at redhat.com Sat Dec 8 13:57:53 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 8 Dec 2018 14:57:53 +0100 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> Message-ID: Anybody has an idea what's up with the test below? As far as I can tell, it passes when I run locally. Thanks, Roman mach5-one-rkennke-JDK-8214259-20181208-1149-13785: FAILED, Failed tests: 1 Build Details: 2018-12-08-1147316.roman.source 1 Failed Test Test Tier Platform Keywords Description Task compiler/aot/calls/fromAot/AotInvokeDynamic2AotTest.java tier1 linux-x64-debug othervm driver Exception: java.lang.AssertionError: duplicate classes for name Ljava/lang/Object;, fingerprints old: ..., new: ..., klass pointers old: meta{HotSpotType}, new: meta{HotSpotType} task Mach5 Tasks Results Summary FAILED: 0 EXECUTED_WITH_FAILURE: 1 NA: 0 KILLED: 0 PASSED: 75 UNABLE_TO_RUN: 0 Test 1 Executed with failure tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-x64-debug-37 Results: total: 73, passed: 72, failed: 1, skipped: 0 Am 08.12.18 um 12:33 schrieb Roman Kennke: > Zhengyu's and Thomas' TaskQueue stuff landed in jdk/jdk and we > integrated it into shenandoah/jdk and in the master patch and webrevs. > I'm updating the webrevs in-place in: > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ > > This is going to be the final webrev that I intend to push on Monday, > unless somebody stops me or we face serious merge conflicts by then. > > I've just pushed this to jdk/submit for testing. I will also do more > testing locally over the weekend. > > Thanks everybody for reviewing and helping and being patient! > > Cheers, > Roman > >> Here comes round 6, possibly/hopefully the last round of webrevs for >> upstreaming Shenandoah: >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ >> >> It incorporates: >> - renames vm_operations_shenandoah* to shenandoahVMOperations*, as >> suggested by Coleen >> - reshuffles gcCause in shared-gc SA as suggested by Per >> - print number of threads in SA, as suggested by Jini >> - fixed a problem in webrev generation that did not track move of >> CriticalNativeStress.java and CriticalNativeArgs.java >> >> The CSR has been approved, the JEP moved to target jdk12, and I got >> positive reviews for all parts. I intend to push this early next week, >> unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes >> in by then, I'll incorporate it. >> >> Thanks everybody for reviewing and reviewing and reviewing again ;-) >> >> Cheers, >> Roman >> >>> Round 5 of Shenandoah review includes: >>> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >>> correct now. We believe the CMS @requires was also not quite right and >>> fixed it the same. >>> >>> It reads now: Don't run this test if: >>> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >>> true, as set by harness >>> - Actual GC set by harness is Shenandoah *and* >>> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >>> default in Shenandoah, so this needs to be double-inverteed). >>> >>> The @requires for CMS was wrong before (we think), because it would also >>> filter defaultGC + ExplicitGCInvokesConcurrent. >>> >>> - Sorting of macros was fixed, as was pointed out by Per >>> - Some stuff was added to SA, as suggested by Jini >>> - Rebased on most current jdk/jdk code >>> >>> Webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >>> >>> I also need reviews from GC reviewers for the CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8214349 >>> >>> I already got reviews for: >>> [x] shared-runtime (coleenp) >>> [x] shared-compiler (kvn) >>> >>> I got reviews for shared-build, but an earlier version, so maybe makes >>> sense to look over this again. Erik J, Magnus? >>> >>> I still need approvals for: >>> [ ] shared-build (kvn, erikj, ihse, pliden) >>> [ ] shared-gc (pliden, kbarrett) >>> [ ] shared-serviceability (jgeorge, pliden) >>> [ ] shared-tests (lmesnik, pliden) >>> [ ] shenandoah-gc >>> [ ] shenandoah-tests >>> >>> >>> Thanks for your patience and ongoing support! >>> >>> Cheers, >>> Roman >>> >>> >>>> Hi all, >>>> >>>> here comes round 4 of Shenandoah upstreaming review: >>>> >>>> This includes fixes for the issues that Per brought up: >>>> - Verify and gracefully reject dangerous flags combinations that >>>> disables required barriers >>>> - Revisited @requires filters in tests >>>> - Trim unused code from Shenandoah's SA impl >>>> - Move ShenandoahGCTracer to gc/shenandoah >>>> - Fix ordering of GC names in various files >>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>> >>>> Thanks everybody for taking time to review this! >>>> Roman >>>> >>>>> Hello all, >>>>> >>>>> Thanks so far for all the reviews and support! >>>>> >>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>>> >>>>> Things we've changed today: >>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>> requires an addition in build machinery though, see >>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>> - Improved zero-disabling and build-code-simplification as suggested by >>>>> Magnus and Per >>>>> - Cleaned up some leftovers in C2 >>>>> - Improved C2 loop opts code by introducing another APIs in >>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>>> that. >>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>>> - Rebased on jdk-12+22 >>>>> >>>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>>> Alright, we fixed: >>>>>> - The minor issues that Kim reported in shared-gc >>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>> >>>>>> Some notes: >>>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>> >>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>> >>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>> those with ZGC? >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>> >>>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>>> next round). >>>>>> >>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>> the code and see what is ok, and especially what is not ok, so that we >>>>>> can focus our efforts on the relevant parts? >>>>>> >>>>>> Updated set of webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>> mainline. >>>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>> changes and >>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>>> eventually >>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>> >>>>>>> JEP: >>>>>>> ? https://openjdk.java.net/jeps/189 >>>>>>> Bug entry: >>>>>>> >>>>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>> >>>>>>> Webrevs: >>>>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>> >>>>>>> For those who want to see the full change, have a look at the >>>>>>> shenandoah-complete >>>>>>> >>>>>>> directory, >>>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>>> shenandoah-master.patch >>>>>>> , >>>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>>> 'shenandoah-complete' webrev). >>>>>>> >>>>>>> Sections to review (at this point) are the following: >>>>>>> ?*) shenandoah-gc >>>>>>> >>>>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>>>> gc/shenandoah >>>>>>> >>>>>>> ?*) shared-gc >>>>>>> >>>>>>> ??? - This is mostly boilerplate that is common to any GC >>>>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>>>> fail (next to CMS and G1) >>>>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>>> >>>>>>> ?*) shared-serviceability >>>>>>> >>>>>>> ??? - The usual code to support another GC >>>>>>> >>>>>>> ?*) shared-runtime >>>>>>> >>>>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>>>> hook up with, >>>>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>>>> ??? - Warning and disabling JFR LeakProfiler >>>>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>> Shenandoah C2 optimizations >>>>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>> Shenandoah's logging >>>>>>> ??? - The usual macros in macro.hpp >>>>>>> >>>>>>> ?*) shared-build >>>>>>> >>>>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>>>> Vladimir K. beforehand >>>>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>> Shenandoah's barriers >>>>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>>> files, which is >>>>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>>>> regression if we >>>>>>> ????? don't) >>>>>>> >>>>>>> ?*) shared-tests >>>>>>> >>>>>>> ??? - Test infrastructure to support Shenandoah >>>>>>> ??? - Shenandoah test groups >>>>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>>>> run with it >>>>>>> >>>>>>> ?*) shenandoah-tests >>>>>>> >>>>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>>> ??? - A couple of tests configurations have been added, e.g. >>>>>>> TestGCBasherWithShenandoah.java >>>>>>> >>>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>>> work left to do >>>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>>> want to take >>>>>>> a peek at it. >>>>>>> >>>>>>> We have regular builds on: >>>>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>> ? - {Windows} x {x86_64}, >>>>>>> ? - {MacOS X} x {x86_64} >>>>>>> >>>>>>> This also routinely passes: >>>>>>> ? - the new Shenandoah tests >>>>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>> >>>>>>> >>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>>> related changes, and of course the many early adopters for reporting >>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>> without any of you! >>>>>>> >>>>>>> Best regards, >>>>>>> Roman >>>>>>> >>>>>> >>>>> >>>> >>> >> > From kim.barrett at oracle.com Sat Dec 8 18:30:18 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 8 Dec 2018 13:30:18 -0500 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> Message-ID: <0A259B61-A4CA-4E2E-8FFF-E855E9285D6E@oracle.com> > On Dec 8, 2018, at 8:57 AM, Roman Kennke wrote: > > Anybody has an idea what's up with the test below? As far as I can tell, > it passes when I run locally. It appears to be an instance of https://bugs.openjdk.java.net/browse/JDK-8201248. > Thanks, > Roman > > mach5-one-rkennke-JDK-8214259-20181208-1149-13785: FAILED, Failed tests: 1 > > Build Details: 2018-12-08-1147316.roman.source > 1 Failed Test > Test Tier Platform Keywords Description Task > compiler/aot/calls/fromAot/AotInvokeDynamic2AotTest.java tier1 > linux-x64-debug othervm driver Exception: java.lang.AssertionError: > duplicate classes for name Ljava/lang/Object;, fingerprints old: ..., > new: ..., klass pointers old: meta{HotSpotType}, new: > meta{HotSpotType} task > Mach5 Tasks Results Summary > > FAILED: 0 > EXECUTED_WITH_FAILURE: 1 > NA: 0 > KILLED: 0 > PASSED: 75 > UNABLE_TO_RUN: 0 > Test > > 1 Executed with failure > > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-x64-debug-37 > Results: total: 73, passed: 72, failed: 1, skipped: 0 From merkel05 at gmail.com Sat Dec 8 20:42:31 2018 From: merkel05 at gmail.com (Sergey) Date: Sat, 8 Dec 2018 21:42:31 +0100 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests Message-ID: Hi, I've recently been working on bug https://bugs.openjdk.java.net/browse/JDK-8214052 that is mainly about cleaning up the [[ and == bash extensions from the tests. I've discovered, that there's a leftover after hg forest consolidation. GeneratePropertyPassword.sh script still uses beforementioned bash extensions. That has been fixed within the scope of this bug: https://bugs.openjdk.java.net/browse/JDK-8025886 Though fix didn't get to the consolidated forest. The fix is trivial and just reapplies the patch from the JBS' comments. Change has been tested with `dash` and works fine. Please find the patch inlined below. That would also be great if anyone could sponsor it. diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh @@ -34,12 +34,13 @@ OS=`uname -s` UMASK=`umask` -if [[ $OS == CYGWIN_NT* ]] ; then +case $OS in +CYGWIN_NT*) OS="Windows_NT" if [ -z "$SystemRoot" ] ; then - SystemRoot=`cygpath $SYSTEMROOT` + SystemRoot=$SYSTEMROOT fi -fi +esac case $OS in SunOS | Linux | Darwin | AIX ) Thanks, Sergei From kim.barrett at oracle.com Sun Dec 9 08:12:32 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 9 Dec 2018 03:12:32 -0500 Subject: 8214976: Warn about uses of functions replaced for portability - was : RE: RFR: 8214944: replace strerror by os::strerror In-Reply-To: References: Message-ID: > On Dec 7, 2018, at 7:00 AM, Baesken, Matthias wrote: > > Hi Kim, this looks great, thanks ! > > Maybe strtok could be considered too for the list of __attribute__ declarations to cause warnings , > because this got pushed recently : > > 8214773: Replace use of thread unsafe strtok > > https://hg.openjdk.java.net/jdk/jdk/rev/8df9cf767f79 I've added a suggested list of functions (including strtok) to JDK-8214976. From kim.barrett at oracle.com Sun Dec 9 08:30:34 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 9 Dec 2018 03:30:34 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet Message-ID: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Please review this change to move the construction of some work gang threads by G1 and CMS to after they've created the barrier set. This allows the removal of some bootstrapping code needed to support that construction order. There isn't any requirement for the old order, it seems to just be a historical artifact. CR: https://bugs.openjdk.java.net/browse/JDK-8215097 Webrev: http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ Testing: mach5 tier1-5 From doko at ubuntu.com Sun Dec 9 10:58:15 2018 From: doko at ubuntu.com (Matthias Klose) Date: Sun, 9 Dec 2018 11:58:15 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> Message-ID: <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> On 06.12.18 22:03, Erik Joelsson wrote: > Nothing strange in there. Is it only printed once? I wouldn't be surprised if > this got printed more than one time during a normal make execution (due to > reloads caused by -include). If it is, then perhaps there is something different > in a later print? no, it's only printed once. And it seems to be independent of the test-image target. just the bootcycle-images target is enough to trigger that [1]. Also not architecture specific for the hotspot targets. Builds without the bootcycle target succeed [2]. Anything else wrong with the configury in [1]? [1] https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=amd64&ver=12~23-2&stamp=1544145960&raw=0 [2] https://buildd.debian.org/status/package.php?p=openjdk-12 > > /Erik > > On 2018-12-06 11:23, Matthias Klose wrote: >> On 06.12.18 20:04, Erik Joelsson wrote: >>> Could you insert this before line 1087 in make/Main.gmk and post the output? >>> >>> $(call PrintVar, ALL_NAMED_TESTS) >>> >>> /Erik >>> >>> On 2018-12-06 10:31, Matthias Klose wrote: >>>> On 06.12.18 17:41, Aleksey Shipilev wrote: >>>>> On 12/6/18 4:34 PM, Matthias Klose wrote: >>>>>> my bad, that happens in the test-image target: >>>>>> >>>>>> In the build log I see: >>>>>> >>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>>>>> >>>>>> >>>>>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>> make/Main.gmk:1087: *** target pattern contains no '%'.? Stop. >>>>>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >>>>> I wonder if "~" in your path screws things up. >>>> that worked at least for the openjdk-11 development cycle, and replacing the >>>> tilde with a dash makes no difference. >> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all >> hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co >> mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc >> hotspot_handshake hotspot_misc hotspot_native_sanity ho >> tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime >> hotspot_runtime hotspot_runtime_minimalvm ho >> tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime >> hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru >> ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt >> jdk_beans jdk_client_sanity jdk_collections jd >> k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core >> jdk_imageio jdk_instrument jdk_instrument_sanity >> jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity >> jdk_lang jdk_launcher jdk_management jdk_mana >> gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi >> jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 >> jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity >> jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other >> jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common tier1_compiler >> tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms >> tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher >> tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime >> tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 >> tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages >> vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi >> vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick >> vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring >> vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict >> vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth >> vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact >> vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle >> vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick >> vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro >> make-make-base make-java-compilation make-copy-files make-idea >> make-compile-commands failure-handler make< From david.holmes at oracle.com Sun Dec 9 22:05:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Dec 2018 08:05:02 +1000 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Message-ID: Hi Kim, Didn't realize you were going to tackle this so soon. I was just ironing out the wrinkles in 8214097 before sending it for review later today. :) On 9/12/2018 6:30 pm, Kim Barrett wrote: > Please review this change to move the construction of some work gang > threads by G1 and CMS to after they've created the barrier set. This > allows the removal of some bootstrapping code needed to support that > construction order. There isn't any requirement for the old order, > it seems to just be a historical artifact. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8215097 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ Moving the GC thread creation out of the heap constructor into the heap initialize function seems quite reasonable. Does ZGC already defer the thread creation? Will this impact the merge of Shenandoah? Everything else seems fine. (I'll be reworking the BarrierSet creation assertion as part of 8214097.) Thanks, David > Testing: > mach5 tier1-5 > From per.liden at oracle.com Sun Dec 9 22:26:15 2018 From: per.liden at oracle.com (Per Liden) Date: Sun, 9 Dec 2018 23:26:15 +0100 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Message-ID: Hi, On 12/09/2018 11:05 PM, David Holmes wrote: > Hi Kim, > > Didn't realize you were going to tackle this so soon. I was just ironing > out the wrinkles in 8214097 before sending it for review later today. :) > > On 9/12/2018 6:30 pm, Kim Barrett wrote: >> Please review this change to move the construction of some work gang >> threads by G1 and CMS to after they've created the barrier set. This >> allows the removal of some bootstrapping code needed to support that >> construction order. There isn't any requirement for the old order, >> it seems to just be a historical artifact. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8215097 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ > > Moving the GC thread creation out of the heap constructor into the heap > initialize function seems quite reasonable. Does ZGC already defer the > thread creation? Will this impact the merge of Shenandoah? ZGC creates the workers in the constructor, and frankly I don't think we want to change that, unless there's some very good reason. I haven't checked Shenandoah. cheers, Per > > Everything else seems fine. (I'll be reworking the BarrierSet creation > assertion as part of 8214097.) > > Thanks, > David > >> Testing: >> mach5 tier1-5 >> From per.liden at oracle.com Sun Dec 9 22:31:43 2018 From: per.liden at oracle.com (Per Liden) Date: Sun, 9 Dec 2018 23:31:43 +0100 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Message-ID: On 12/09/2018 11:26 PM, Per Liden wrote: > Hi, > > On 12/09/2018 11:05 PM, David Holmes wrote: >> Hi Kim, >> >> Didn't realize you were going to tackle this so soon. I was just >> ironing out the wrinkles in 8214097 before sending it for review later >> today. :) >> >> On 9/12/2018 6:30 pm, Kim Barrett wrote: >>> Please review this change to move the construction of some work gang >>> threads by G1 and CMS to after they've created the barrier set. This >>> allows the removal of some bootstrapping code needed to support that >>> construction order. There isn't any requirement for the old order, >>> it seems to just be a historical artifact. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8215097 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ >> >> Moving the GC thread creation out of the heap constructor into the >> heap initialize function seems quite reasonable. Does ZGC already >> defer the thread creation? Will this impact the merge of Shenandoah? > > ZGC creates the workers in the constructor, and frankly I don't think we > want to change that, unless there's some very good reason. I haven't > checked Shenandoah. But I should also add that ZGC registers its barrier set in the constructor too, before workers are created, so I don't think Kim's patch will cause any problems really. Other collectors typically registers their barrier set in initialize(), which I guess is the root of the bootstrapping issue. cheers, Per > > cheers, > Per > >> >> Everything else seems fine. (I'll be reworking the BarrierSet creation >> assertion as part of 8214097.) >> >> Thanks, >> David >> >>> Testing: >>> mach5 tier1-5 >>> From rkennke at redhat.com Sun Dec 9 22:37:10 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 9 Dec 2018 23:37:10 +0100 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Message-ID: <1c4c5dcd-cf0e-faf7-c207-e3a630fb4c7c@redhat.com> >> Didn't realize you were going to tackle this so soon. I was just >> ironing out the wrinkles in 8214097 before sending it for review later >> today. :) >> >> On 9/12/2018 6:30 pm, Kim Barrett wrote: >>> Please review this change to move the construction of some work gang >>> threads by G1 and CMS to after they've created the barrier set.? This >>> allows the removal of some bootstrapping code needed to support that >>> construction order.? There isn't any requirement for the old order, >>> it seems to just be a historical artifact. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8215097 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ >> >> Moving the GC thread creation out of the heap constructor into the >> heap initialize function seems quite reasonable. Does ZGC already >> defer the thread creation? Will this impact the merge of Shenandoah? > > ZGC creates the workers in the constructor, and frankly I don't think we > want to change that, unless there's some very good reason. I haven't > checked Shenandoah. Shenandoah also creates workers in constructors. May I request to hold this off a little bit to avoid making a mess out of the very soon upstream push? I'm open to moving this into initialize() but I cannot tell real quick if it has or hasn't any funny side-effects. Thanks, Roman From david.holmes at oracle.com Mon Dec 10 07:07:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Dec 2018 17:07:32 +1000 Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic Message-ID: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> This is a preliminary RFR to get feedback. I won't be pushing until 13 and the final RFR will have to sync with other changes that may impact what I'm doing here - e.g. Shenandoah GC. @Markus: does this help with your JFR event hooks? Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ The primary change is that we introduce initialization and tear-down hooks for the different thread types so that we can consistently apply common actions - like adding/removing NJTs to the NJT::list - and ensure consistent lifecycle management. This is done by introducing virtual pre_run() and post_run() hooks so that we arrange the non-virtual Thread::call_run() so that it does: void Thread::call_run() { // Perform common initialization actions register_thread_stack_with_NMT(); ... // Perform initialization actions this->pre_run(); // Invoke ::run() this->run(); // Perform common tear-down actions // Perform tear-down actions this->post_run(); } And establish some basic lifecycle rules, such as: - no deletion of the thread is allowed prior to post_run() - after post_run() 'this' should not be referenced - Thread::current must be cleared before Thread deletion I refactored a little bit of common setup by getting rid of NamedThread::initialize_named_thread(), and having NJT::pre_run() set the name.** Added some comments about how the different thread entry points and hooks connect together. Clarified some initialization related interactions between thread creation and BarrierSet creation (which Kim is trying to clean up under 8215097). Reverted the change in management.cpp put in under JDK-8212207 as it is no longer needed now that NJTs only get added to the list in pre_run() and so must be valid. And somewhat related I added an assert on Linux that osthread->pthread_id() has been set before the call to call_run(). Simplified gtest/threadHelper.inline.hpp as the hooks introduced are now supplied by Thread. Thanks to Kim Barrett for the updated NJT list management code and discussion on different approaches. Thanks, David ** The role of NamedThread seems quite tenous now, but it's still being used to store "stuff" that seems unrelated to being a NamedThread. Anyway that would be a future cleanup RFE. From david.holmes at oracle.com Mon Dec 10 07:25:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Dec 2018 17:25:17 +1000 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: <12fb2633-7643-d87b-8578-7aa0ff62b07e@oracle.com> Hi Sergey, Sorry to be a pain but you do need to send the request to the appropriate mailing list for the "owner" of the code being changed. In this case: test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh is not hotspot code but JDK Monitoring&Management code, which I think is covered by serviceability-dev. Cheers, David On 9/12/2018 6:42 am, Sergey wrote: > Hi, > > I've recently been working on bug > https://bugs.openjdk.java.net/browse/JDK-8214052 > that is mainly about cleaning up the [[ and == bash > extensions from the tests. > > I've discovered, that there's a leftover after hg forest > consolidation. GeneratePropertyPassword.sh script > still uses beforementioned bash extensions. That has > been fixed within the scope of this bug: > https://bugs.openjdk.java.net/browse/JDK-8025886 > Though fix didn't get to the consolidated forest. > > The fix is trivial and just reapplies the patch from the > JBS' comments. Change has been tested with `dash` > and works fine. > > Please find the patch inlined below. That would also be > great if anyone could sponsor it. > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -34,12 +34,13 @@ > OS=`uname -s` > UMASK=`umask` > > -if [[ $OS == CYGWIN_NT* ]] ; then > +case $OS in > +CYGWIN_NT*) > OS="Windows_NT" > if [ -z "$SystemRoot" ] ; then > - SystemRoot=`cygpath $SYSTEMROOT` > + SystemRoot=$SYSTEMROOT > fi > -fi > +esac > > case $OS in > SunOS | Linux | Darwin | AIX ) > > > Thanks, > Sergei > From thomas.schatzl at oracle.com Mon Dec 10 10:05:01 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 10 Dec 2018 11:05:01 +0100 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: Message-ID: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> Hi, On Fri, 2018-12-07 at 22:42 +0800, Man Cao wrote: > Hi Thomas, > > Thanks for the review. No worries about the delay. Responding to some > comments below, will address other comments soon. > > > - not sure about the usefulness of the OverheadTester interface. At > > least in the current implementation its implementation could be > > easily replaced by static functions returning a bool. That would > > save a lot of boiler plate code. Is there some reason to have > > explicit classes for that? > > The OverheadTester interface provides a way to pass functions as > parameters to OverheadChecker::check_gc_overhead_limit(), and hide > details about what parameters are required to compute is_exceeded(). > We could change OverheadChecker::check_gc_overhead_limit() to take > "bool time_overhead_exceeded, bool space_overhead_exceeded" > parameters instead of two pointers to OverheadTester. > Then we can have two static functions instead of > the OverheadTester interface. However, it would introduce a > small behavior change > for AdaptiveSizePolicy::check_gc_overhead_limit(). > Specifically, it would compute whether time and space overhead limits > are exceeded every time the function is called. Currently it only > needs to compute these under the following condition: > !GCCause::is_user_requested_gc(gc_cause) && > !GCCause::is_serviceability_requested_gc(gc_cause) && is_full_gc > If this behavior change is OK, I can replace the OverheadTester > interface. Given this explanation it seems okay to use the extra classes. > > > Assuming that all collectors want to implement this, and actually > > need to I am leaning towards doing so. However the ZGC/Shenandoah > > people might object to this. > > I think it is reasonable for all collectors to support this, but the > low-pause collectors may have a very different definition for GC > overhead. I also received internal user complaints that the current > UseGCOverheadLimit for ParallelGC and CMS is too difficult to get > triggered, because of the requirement for 5 consecutive full GCs. > It might also be a problem for G1, as it avoids full GCs eagerly. I > think we might need to change the 5-full-GC requirement in the future > to make it more useful. We can discuss this in a separate RFE. > > > Okay. However I do not know how to rename them and what are the > > requirements here. Probably a CSR is needed, although I have seen > > translation tables used (in > > src/jdk.internal.jvmstat/share/classes/sun/jvmstat/perfdata/resourc > > es/aliasmap). I cc'ed hotspot-dev in the assumption somebody else > > there knows more about this. > > If the process is too complicated, I can live with the current > imperfect name. It is not causing much ambiguity. I can just move > it's definition to GCPolicyCounters. Creating a CSR and getting it approved is not a big deal; it may even be useful as it clearly communicates the change to the users. Additionally I think due to that translation table I mentioned, the old name can still be used I think. > Keeping its name unchanged also reduces users' work when they upgrade > to new JDK, if they are already monitoring this counter using their > own tools. In fact, the more useful counter for monitoring purpose is > the raw value for gc_cost(). > We have a local patch to export it for ParallelGC and CMS, but it > doesn't work for G1 yet. Perhaps we can upstream it in the future > when it works for G1. Thanks, Thomas From thomas.schatzl at oracle.com Mon Dec 10 10:07:50 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 10 Dec 2018 11:07:50 +0100 Subject: RFR(S) [downport 11]: 8213410: UseCompressedOops requirement check fails fails on 32-bit system In-Reply-To: <0a52708b571f4c729664900b4a7e5bd1@sap.com> References: <0a52708b571f4c729664900b4a7e5bd1@sap.com> Message-ID: Hi, On Wed, 2018-11-28 at 09:00 +0000, Lindenmaier, Goetz wrote: > Hi, > > I need a review to downport 8213410. > 8213410 fixes disabling tests for UseCompressedOops on 32-bit > platforms. > > In jdk12, "@requires vm.bits == 64" was introduced to > CompressedClassPointers.java > by "8208655: use JTreg skipped status in hotspot tests". Because of > this change, 8213410 does not apply properly. > To fix this, I introduce "@requires vm.bits == 64" to this test in > jdk11, too. > http://cr.openjdk.java.net/~goetz/wr18/8213410-32bit_cOops-jdk11/01/ change looks good. Thomas From magnus.ihse.bursie at oracle.com Mon Dec 10 10:19:24 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 10 Dec 2018 11:19:24 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> Message-ID: <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> On 2018-12-09 11:58, Matthias Klose wrote: > On 06.12.18 22:03, Erik Joelsson wrote: >> Nothing strange in there. Is it only printed once? I wouldn't be surprised if >> this got printed more than one time during a normal make execution (due to >> reloads caused by -include). If it is, then perhaps there is something different >> in a later print? > no, it's only printed once. And it seems to be independent of the test-image > target. just the bootcycle-images target is enough to trigger that [1]. Also not > architecture specific for the hotspot targets. Builds without the bootcycle > target succeed [2]. > > Anything else wrong with the configury in [1]? It's a bit hard to say. I'm reacting to "make -C build", maybe the -C does not play well with bootcycle builds? I don't think that's a very well tested combination, and bootcycle builds is really like running the build twice in different directories. But I don't know, it shouldn't affect things... You are also running with a very detailed log level, it hardly makes it easier to read the log. I recommend using "info,cmdlines" instead of "debug" unless actively debugging a hard-to-find issue. ... Found it! Erik's intuition was correct: ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 [...snip...] gtest micro make-make-base make-java-compilation make-copy-files make-idea make-compile-commands make-make[5]: make-Leaving make-directory make-'/<>' failure-handler make< make[5]: Leaving directory '/<>' make/Main.gmk:1088: *** target pattern contains no '%'. Stop. So once again we're being bit by the make changedir message. And maybe that's triggered in a new way due to the -C? Try adding --no-print-directory to your top-level make invocation, as a workaround. We should probably send that as a flag to make, always. /Magnus > > [1] > https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=amd64&ver=12~23-2&stamp=1544145960&raw=0 > [2] https://buildd.debian.org/status/package.php?p=openjdk-12 > > >> /Erik >> >> On 2018-12-06 11:23, Matthias Klose wrote: >>> On 06.12.18 20:04, Erik Joelsson wrote: >>>> Could you insert this before line 1087 in make/Main.gmk and post the output? >>>> >>>> $(call PrintVar, ALL_NAMED_TESTS) >>>> >>>> /Erik >>>> >>>> On 2018-12-06 10:31, Matthias Klose wrote: >>>>> On 06.12.18 17:41, Aleksey Shipilev wrote: >>>>>> On 12/6/18 4:34 PM, Matthias Klose wrote: >>>>>>> my bad, that happens in the test-image target: >>>>>>> >>>>>>> In the build log I see: >>>>>>> >>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>>>>>> >>>>>>> >>>>>>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>> make/Main.gmk:1087: *** target pattern contains no '%'. Stop. >>>>>>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >>>>>> I wonder if "~" in your path screws things up. >>>>> that worked at least for the openjdk-11 development cycle, and replacing the >>>>> tilde with a dash makes no difference. >>> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all >>> hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co >>> mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc >>> hotspot_handshake hotspot_misc hotspot_native_sanity ho >>> tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime >>> hotspot_runtime hotspot_runtime_minimalvm ho >>> tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime >>> hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru >>> ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt >>> jdk_beans jdk_client_sanity jdk_collections jd >>> k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core >>> jdk_imageio jdk_instrument jdk_instrument_sanity >>> jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity >>> jdk_lang jdk_launcher jdk_management jdk_mana >>> gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi >>> jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 >>> jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity >>> jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other >>> jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common tier1_compiler >>> tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms >>> tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher >>> tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime >>> tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 >>> tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages >>> vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi >>> vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick >>> vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring >>> vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict >>> vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth >>> vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact >>> vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle >>> vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick >>> vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro >>> make-make-base make-java-compilation make-copy-files make-idea >>> make-compile-commands failure-handler make< From thomas.schatzl at oracle.com Mon Dec 10 10:20:45 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 10 Dec 2018 11:20:45 +0100 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Message-ID: <43c4783a6caf1f1d8d0716731d842b4ae3c02afd.camel@oracle.com> Hi, On Sun, 2018-12-09 at 03:30 -0500, Kim Barrett wrote: > Please review this change to move the construction of some work gang > threads by G1 and CMS to after they've created the barrier set. This > allows the removal of some bootstrapping code needed to support that > construction order. There isn't any requirement for the old order, > it seems to just be a historical artifact. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8215097 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ > > Testing: > mach5 tier1-5 looks good. Thomas From sgehwolf at redhat.com Mon Dec 10 10:31:21 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 10 Dec 2018 11:31:21 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> Message-ID: <5a0fc37c20a5c628faad6cb0ad20ec02a21d65f7.camel@redhat.com> On Mon, 2018-12-10 at 11:19 +0100, Magnus Ihse Bursie wrote: > > On 2018-12-09 11:58, Matthias Klose wrote: > > On 06.12.18 22:03, Erik Joelsson wrote: > > > Nothing strange in there. Is it only printed once? I wouldn't be surprised if > > > this got printed more than one time during a normal make execution (due to > > > reloads caused by -include). If it is, then perhaps there is something different > > > in a later print? > > > > no, it's only printed once. And it seems to be independent of the test-image > > target. just the bootcycle-images target is enough to trigger that [1]. Also not > > architecture specific for the hotspot targets. Builds without the bootcycle > > target succeed [2]. > > > > Anything else wrong with the configury in [1]? > > It's a bit hard to say. I'm reacting to "make -C build", maybe the -C > does not play well with bootcycle builds? I don't think that's a very > well tested combination, and bootcycle builds is really like running the > build twice in different directories. But I don't know, it shouldn't > affect things... > You are also running with a very detailed log level, it hardly makes it > easier to read the log. I recommend using "info,cmdlines" instead of > "debug" unless actively debugging a hard-to-find issue. > > ... Found it! Erik's intuition was correct: > > ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 [...snip...] gtest > micro make-make-base make-java-compilation make-copy-files make-idea > make-compile-commands make-make[5]: make-Leaving make-directory > make-'/<>' failure-handler make< > make[5]: Leaving directory '/<>' > make/Main.gmk:1088: *** target pattern contains no '%'. Stop. > > So once again we're being bit by the make changedir message. And maybe > that's triggered in a new way due to the -C? Try adding > --no-print-directory to your top-level make invocation, as a workaround. > > We should probably send that as a flag to make, always. Wasn't this fixed with? https://bugs.openjdk.java.net/browse/JDK-8213736 Thanks, Severin > /Magnus > > > > > [1] > > https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=amd64&ver=12~23-2&stamp=1544145960&raw=0 > > [2] https://buildd.debian.org/status/package.php?p=openjdk-12 > > > > > > > /Erik > > > > > > On 2018-12-06 11:23, Matthias Klose wrote: > > > > On 06.12.18 20:04, Erik Joelsson wrote: > > > > > Could you insert this before line 1087 in make/Main.gmk and post the output? > > > > > > > > > > $(call PrintVar, ALL_NAMED_TESTS) > > > > > > > > > > /Erik > > > > > > > > > > On 2018-12-06 10:31, Matthias Klose wrote: > > > > > > On 06.12.18 17:41, Aleksey Shipilev wrote: > > > > > > > On 12/6/18 4:34 PM, Matthias Klose wrote: > > > > > > > > my bad, that happens in the test-image target: > > > > > > > > > > > > > > > > In the build log I see: > > > > > > > > > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp > > > > > > > > > > > > > > > > > > > > > > > > make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' > > > > > > > > make/Main.gmk:1087: *** target pattern contains no '%'. Stop. > > > > > > > > make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' > > > > > > > > make[5]: warning: -j1 forced in submake: resetting jobserver mode. > > > > > > > > > > > > > > I wonder if "~" in your path screws things up. > > > > > > > > > > > > that worked at least for the openjdk-11 development cycle, and replacing the > > > > > > tilde with a dash makes no difference. > > > > > > > > ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all > > > > hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co > > > > mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc > > > > hotspot_handshake hotspot_misc hotspot_native_sanity ho > > > > tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime > > > > hotspot_runtime hotspot_runtime_minimalvm ho > > > > tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime > > > > hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru > > > > ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt > > > > jdk_beans jdk_client_sanity jdk_collections jd > > > > k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core > > > > jdk_imageio jdk_instrument jdk_instrument_sanity > > > > jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity > > > > jdk_lang jdk_launcher jdk_management jdk_mana > > > > gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi > > > > jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 > > > > jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity > > > > jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other > > > > jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common tier1_compiler > > > > tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms > > > > tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher > > > > tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime > > > > tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 > > > > tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages > > > > vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi > > > > vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick > > > > vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring > > > > vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict > > > > vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth > > > > vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact > > > > vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle > > > > vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick > > > > vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro > > > > make-make-base make-java-compilation make-copy-files make-idea > > > > make-compile-commands failure-handler make< > > From magnus.ihse.bursie at oracle.com Mon Dec 10 10:49:25 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 10 Dec 2018 11:49:25 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <5a0fc37c20a5c628faad6cb0ad20ec02a21d65f7.camel@redhat.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> <5a0fc37c20a5c628faad6cb0ad20ec02a21d65f7.camel@redhat.com> Message-ID: <99e0ec6c-2655-803b-b87b-f2fc323cd9f1@oracle.com> On 2018-12-10 11:31, Severin Gehwolf wrote: > On Mon, 2018-12-10 at 11:19 +0100, Magnus Ihse Bursie wrote: >> On 2018-12-09 11:58, Matthias Klose wrote: >>> On 06.12.18 22:03, Erik Joelsson wrote: >>>> Nothing strange in there. Is it only printed once? I wouldn't be surprised if >>>> this got printed more than one time during a normal make execution (due to >>>> reloads caused by -include). If it is, then perhaps there is something different >>>> in a later print? >>> no, it's only printed once. And it seems to be independent of the test-image >>> target. just the bootcycle-images target is enough to trigger that [1]. Also not >>> architecture specific for the hotspot targets. Builds without the bootcycle >>> target succeed [2]. >>> >>> Anything else wrong with the configury in [1]? >> It's a bit hard to say. I'm reacting to "make -C build", maybe the -C >> does not play well with bootcycle builds? I don't think that's a very >> well tested combination, and bootcycle builds is really like running the >> build twice in different directories. But I don't know, it shouldn't >> affect things... >> You are also running with a very detailed log level, it hardly makes it >> easier to read the log. I recommend using "info,cmdlines" instead of >> "debug" unless actively debugging a hard-to-find issue. >> >> ... Found it! Erik's intuition was correct: >> >> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 [...snip...] gtest >> micro make-make-base make-java-compilation make-copy-files make-idea >> make-compile-commands make-make[5]: make-Leaving make-directory >> make-'/<>' failure-handler make< >> make[5]: Leaving directory '/<>' >> make/Main.gmk:1088: *** target pattern contains no '%'. Stop. >> >> So once again we're being bit by the make changedir message. And maybe >> that's triggered in a new way due to the -C? Try adding >> --no-print-directory to your top-level make invocation, as a workaround. >> >> We should probably send that as a flag to make, always. > Wasn't this fixed with? > https://bugs.openjdk.java.net/browse/JDK-8213736 Unless Matthias is running with an outdated source (Matthias: Are you?), I'm afraid that the solution in JDK-8213736 was not complete. :( The makefile bootstraping logic is quite hairy, and I'm sure there's ways to still trigger the same issue, but differently. /Magnus > > Thanks, > Severin > >> /Magnus >> >>> [1] >>> https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=amd64&ver=12~23-2&stamp=1544145960&raw=0 >>> [2] https://buildd.debian.org/status/package.php?p=openjdk-12 >>> >>> >>>> /Erik >>>> >>>> On 2018-12-06 11:23, Matthias Klose wrote: >>>>> On 06.12.18 20:04, Erik Joelsson wrote: >>>>>> Could you insert this before line 1087 in make/Main.gmk and post the output? >>>>>> >>>>>> $(call PrintVar, ALL_NAMED_TESTS) >>>>>> >>>>>> /Erik >>>>>> >>>>>> On 2018-12-06 10:31, Matthias Klose wrote: >>>>>>> On 06.12.18 17:41, Aleksey Shipilev wrote: >>>>>>>> On 12/6/18 4:34 PM, Matthias Klose wrote: >>>>>>>>> my bad, that happens in the test-image target: >>>>>>>>> >>>>>>>>> In the build log I see: >>>>>>>>> >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>>>>>>>> >>>>>>>>> >>>>>>>>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>>>> make/Main.gmk:1087: *** target pattern contains no '%'. Stop. >>>>>>>>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>>>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >>>>>>>> I wonder if "~" in your path screws things up. >>>>>>> that worked at least for the openjdk-11 development cycle, and replacing the >>>>>>> tilde with a dash makes no difference. >>>>> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all >>>>> hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co >>>>> mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc >>>>> hotspot_handshake hotspot_misc hotspot_native_sanity ho >>>>> tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime >>>>> hotspot_runtime hotspot_runtime_minimalvm ho >>>>> tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime >>>>> hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru >>>>> ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt >>>>> jdk_beans jdk_client_sanity jdk_collections jd >>>>> k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core >>>>> jdk_imageio jdk_instrument jdk_instrument_sanity >>>>> jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity >>>>> jdk_lang jdk_launcher jdk_management jdk_mana >>>>> gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi >>>>> jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 >>>>> jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity >>>>> jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other >>>>> jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common tier1_compiler >>>>> tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms >>>>> tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher >>>>> tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime >>>>> tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 >>>>> tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages >>>>> vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi >>>>> vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick >>>>> vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring >>>>> vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict >>>>> vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth >>>>> vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact >>>>> vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle >>>>> vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick >>>>> vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro >>>>> make-make-base make-java-compilation make-copy-files make-idea >>>>> make-compile-commands failure-handler make< >> From magnus.ihse.bursie at oracle.com Mon Dec 10 11:44:23 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 10 Dec 2018 12:44:23 +0100 Subject: RFR (S): 8214850: Rename vm_operations* to vmOperations* In-Reply-To: References: <6d0c700f01192c1ad56e8c6ae2dc5f80733e72cd.camel@oracle.com> <0abe8a03-35c3-1472-4ca5-c3571446b0c4@oracle.com> Message-ID: <0ce2da68-14eb-cfd1-ca0d-9b1dae9718b9@oracle.com> On 2018-12-05 22:51, David Holmes wrote: > On 6/12/2018 12:27 am, coleen.phillimore at oracle.com wrote: >> >> This change looks good to me. I type this name wrong every time. >> On 12/5/18 6:08 AM, David Holmes wrote: >>> Hi Thomas, >>> >>> On 5/12/2018 8:47 pm, Thomas Schatzl wrote: >>>> Hi all, >>>> >>>> can I have reviews for this change that renames the vm_operations >>>> files to vmOperations? >>>> >>>> This has been requested in the JDK-8214791 review but I did not >>>> want to >>>> merge these. >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8214850 >>>> Webrev: >>>> http://cr.openjdk.java.net/~tschatzl/8214850/webrev/ >>>> Testing: >>>> build on all (Oracle) platforms >>> >>> I'm not a fan of gratuitous renames as it makes it harder to query >>> the history of a file. >>> >> >> hg mv will keep the history of the file moved. > > Yes but you have to tell hg to follow across the rename; and graphical > tools like TortoiseHg don't seem to follow; and if you only know the > old name of the file (eg from before the module system) I have yet to > determine how to get hg or Tortoise to show it to me. David, Add: [defaults] log = --follow to your .hgrc. (I actually don't understand why this is not the default behavior; there seem to be no downside to it.) In my experience TortoiseHg follows files across renames, but maybe that's due to the .hgrc setting above? /Magnus > > Cheers, > David > >> >>> But this seems to be an accurate renaming so in that sense this is >>> Reviewed. >>> >>> But if we're going to impose a particular naming scheme perhaps we >>> should bite the bullet and bring everything into line? >>> arguments_ext, os_ext, os_perf, vm_version ... >>> >> >> Yes, eventually, depending on distracting the misnaming is, and how >> difficult to find the file. vm_version seems like a candidate. >> >> thanks, >> Coleen >>> Cheers, >>> David >>> >>>> Thanks, >>>> Thomas >>>> >>>> >> From magnus.ihse.bursie at oracle.com Mon Dec 10 13:19:09 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 10 Dec 2018 14:19:09 +0100 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT Message-ID: I propose that we introduce a new define, available to all JDK native files (Hotspot included), called JDK_EXPORT. The behavior of this symbol will be very similar (as of now, in fact identical) to JNIEXPORT; however, the semantics will not. Currently, we "mis-use" the JNIEXPORT define to mark a function for exporting from the library. The problem with this is that JNIEXPORT is part of the JNI interface, and is supposed to be used when C programs interact with Java. And, when doing this, the function should be fully decorated like this: "JNIEXPORT foo JNICALL". We do have many such JNI exports in our native libraries, but we also have a lot of other, non-JNI exports, where one native library just provides an interface to other libraries. In these cases, we have still used JNIEXPORT for the functionality of getting the function exported, but we have not been consistent in our use of JNICALL. This has caused us way too much trouble for something that should Just Work. I therefore propose that we define "JDK_EXPORT", with the same behavior as JNIEXPORT (that is, flagging the function for external visibility in the resulting native library), but which is *not* supposed to be exported to Java code using JNI, nor supposed to be decorated with JNICALL. All current instances of JNIEXPORT which is not pure JNI native functions should be changed to use JDK_EXPORT instead. I further propose that this macro should reside in a new file "jdk.h", placed in the new directory src/java.base/share/native/include/internal. This header file path will automatically be provided to all native libraries, but not copied to the JDK being built. (The existence of a "include/internal" directory with this behavior has been discussed before. There are more files that ought to be moved there, if/when it is created.) I believe in many cases the #include "jni.h" can be just modified to #include "#jdk.h", since most native code will not require "jni.h" unless actually doing JNI calls -- most have included this file to get the JNIEXPORT macro, which would explain the pervasive use of #include "jni.h" in our code base. Thoughts? /Magnus From gg5070 at gmail.com Mon Dec 10 13:47:53 2018 From: gg5070 at gmail.com (Gidon Gershinsky) Date: Mon, 10 Dec 2018 15:47:53 +0200 Subject: Problems with AES-GCM native acceleration In-Reply-To: References: <92a16009b9e047eacf102b006bc0612bac4a2cfb.camel@redhat.com> <1b1fe9e4-ad71-7d48-13eb-48dbe267962e@oracle.com> <864a763d-0d67-d894-234c-fbbef78e0320@oracle.com> Message-ID: I've run my decryption benchmarks on Java with this patch, with excellent results. The benchmarks reach the top speed right away, no long warm-up anymore. Also, there is no need to split the operation into multiple updates, the doFinal works just fine. Moreover, the maximal decryption throughput is actually higher than in Java11 after warm-up. On one thread, I get 930MB/sec instead of 850MB/s. On 8 threads, 350x8 instead of 230x8. This capability will be important in the Spark/Parquet workloads. Cheers, Gidon On Fri, Nov 30, 2018 at 6:10 PM Andrew Haley wrote: > On 11/21/18 5:37 PM, Andrew Haley wrote: > > On 11/15/18 10:42 AM, Gidon Gershinsky wrote: > >> Having the decryption optimized in the HotSpot engine would be ideal. > > > > I agree with you. I did a few experiments, and it can take a very > > long time for C2 compilation to kick in, especially because GCM is > > glacially slow until the intrinsics are used. > > > > I think this would be a generally useful enhancement to HotSpot, > > and I'm kicking around an experimental patch which adds the > > intrinsics to c1 and the interpreter. > > There's a proof-of-concept patch at http://cr.openjdk.java.net/~aph/gctr/ > It's all rather hacky but it works. > > The patch is rather more complicated than I would have liked. We > could simplify it somewhat by getting rid of the C1 intrinsic, and > instead making C1 call the interpreter implementation. > > There also a jmh benchmark in that directory. Test results for 1Mb > look like this: > > Interp: > > Benchmark Mode Cnt Score Error Units > AESGCMUpdateAAD2.test avgt 5 1426.275 ? 8.778 us/op > > C1: > > Benchmark Mode Cnt Score Error Units > AESGCMUpdateAAD2.test avgt 5 1359.367 ? 8.196 us/op > > C2: > > Benchmark Mode Cnt Score Error Units > AESGCMUpdateAAD2.test avgt 5 1333.863 ? 18.385 us/op > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > From markus.gronlund at oracle.com Mon Dec 10 14:49:31 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Mon, 10 Dec 2018 06:49:31 -0800 (PST) Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> References: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> Message-ID: <4a3d5c3e-3dfd-4ba8-b929-374e971565aa@default> Hi David, Thank you for looking into this. I think this approach is in the right direction. Specially as it gives a form and a vocabulary to reason about thread initialization and teardown states. I like the fact you provide both shared as well as thread-specific locations for transitions, like: Thread::call_run() // shared pre-run // specific pre-run // shared run // specific run // shared post-run // specific post-run This means we can talk about this better now. I can for example say, "the hook is thread generic so it can go into "shared post-run"". In addition, we can start to bind properties to them, much like you have done with the assertions for thread locals. It will help a lot to have the guarantee using pre-and post-conditions that a thread will not be deleted somewhere deeper in the call stack. I could easily consolidate JFR_ONLY(Jfr::on_thread_exit();), originally located in JavaThread::exit() and NonJavaThread::post_run() into "shared post-run". I also elaborated with moving "this->exit(false);" from JavaThread::thread_main_inner() to JavaThread::post_run(), and that seemed to work quite well, although I understand that we are not moving all components at once. The most important thing is that we get a protocol and locations in place for where things could be moved (which is what you are addressing). Thanks again for doing this Markus -----Original Message----- From: David Holmes Sent: den 10 december 2018 08:08 To: hotspot-dev developers Cc: Kim Barrett ; Markus Gr?nlund Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic This is a preliminary RFR to get feedback. I won't be pushing until 13 and the final RFR will have to sync with other changes that may impact what I'm doing here - e.g. Shenandoah GC. @Markus: does this help with your JFR event hooks? Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ The primary change is that we introduce initialization and tear-down hooks for the different thread types so that we can consistently apply common actions - like adding/removing NJTs to the NJT::list - and ensure consistent lifecycle management. This is done by introducing virtual pre_run() and post_run() hooks so that we arrange the non-virtual Thread::call_run() so that it does: void Thread::call_run() { // Perform common initialization actions register_thread_stack_with_NMT(); ... // Perform initialization actions this->pre_run(); // Invoke ::run() this->run(); // Perform common tear-down actions // Perform tear-down actions this->post_run(); } And establish some basic lifecycle rules, such as: - no deletion of the thread is allowed prior to post_run() - after post_run() 'this' should not be referenced - Thread::current must be cleared before Thread deletion I refactored a little bit of common setup by getting rid of NamedThread::initialize_named_thread(), and having NJT::pre_run() set the name.** Added some comments about how the different thread entry points and hooks connect together. Clarified some initialization related interactions between thread creation and BarrierSet creation (which Kim is trying to clean up under 8215097). Reverted the change in management.cpp put in under JDK-8212207 as it is no longer needed now that NJTs only get added to the list in pre_run() and so must be valid. And somewhat related I added an assert on Linux that osthread->pthread_id() has been set before the call to call_run(). Simplified gtest/threadHelper.inline.hpp as the hooks introduced are now supplied by Thread. Thanks to Kim Barrett for the updated NJT list management code and discussion on different approaches. Thanks, David ** The role of NamedThread seems quite tenous now, but it's still being used to store "stuff" that seems unrelated to being a NamedThread. Anyway that would be a future cleanup RFE. From aph at redhat.com Mon Dec 10 16:15:21 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 10 Dec 2018 16:15:21 +0000 Subject: Problems with AES-GCM native acceleration In-Reply-To: References: <92a16009b9e047eacf102b006bc0612bac4a2cfb.camel@redhat.com> <1b1fe9e4-ad71-7d48-13eb-48dbe267962e@oracle.com> <864a763d-0d67-d894-234c-fbbef78e0320@oracle.com> Message-ID: <737aad13-273a-4295-e835-da766e004a93@redhat.com> On 12/10/18 1:47 PM, Gidon Gershinsky wrote: > I've run my decryption benchmarks on Java with this patch, with excellent > results. > The benchmarks reach the top speed right away, no long warm-up anymore. > Also, there is no need to split the operation into multiple updates, the > doFinal works just fine. > > Moreover, the maximal decryption throughput is actually higher than in > Java11 after warm-up. > On one thread, I get 930MB/sec instead of 850MB/s. > On 8 threads, 350x8 instead of 230x8. > > This capability will be important in the Spark/Parquet workloads. The downloadable binary build is at https://aph.fedorapeople.org/jdk-12-internal+0-adhoc.aph.jdk-jdk.tar.gz -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martinrb at google.com Mon Dec 10 16:43:40 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 10 Dec 2018 08:43:40 -0800 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: > I would not try to remove all bash-isms from openjdk. Instead I would find > instances of /bin/sh that need to be changed to /bin/bash. > From erik.joelsson at oracle.com Mon Dec 10 17:25:57 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 10 Dec 2018 09:25:57 -0800 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: References: Message-ID: <7f36135b-4c6d-4b14-9477-1cd4d969d3ac@oracle.com> Sounds good to me. /Erik On 2018-12-10 05:19, Magnus Ihse Bursie wrote: > I propose that we introduce a new define, available to all JDK native > files (Hotspot included), called JDK_EXPORT. The behavior of this > symbol will be very similar (as of now, in fact identical) to > JNIEXPORT; however, the semantics will not. > > Currently, we "mis-use" the JNIEXPORT define to mark a function for > exporting from the library. The problem with this is that JNIEXPORT is > part of the JNI interface, and is supposed to be used when C programs > interact with Java. And, when doing this, the function should be fully > decorated like this: "JNIEXPORT foo JNICALL". > > We do have many such JNI exports in our native libraries, but we also > have a lot of other, non-JNI exports, where one native library just > provides an interface to other libraries. In these cases, we have > still used JNIEXPORT for the functionality of getting the function > exported, but we have not been consistent in our use of JNICALL. This > has caused us way too much trouble for something that should Just > Work. > > I therefore propose that we define "JDK_EXPORT", with the same > behavior as JNIEXPORT (that is, flagging the function for external > visibility in the resulting native library), but which is *not* > supposed to be exported to Java code using JNI, nor supposed to be > decorated with JNICALL. All current instances of JNIEXPORT which is > not pure JNI native functions should be changed to use JDK_EXPORT > instead. > > I further propose that this macro should reside in a new file "jdk.h", > placed in the new directory > src/java.base/share/native/include/internal. This header file path > will automatically be provided to all native libraries, but not copied > to the JDK being built. (The existence of a "include/internal" > directory with this behavior has been discussed before. There are more > files that ought to be moved there, if/when it is created.) I believe > in many cases the #include "jni.h" can be just modified to #include > "#jdk.h", since most native code will not require "jni.h" unless > actually doing JNI calls -- most have included this file to get the > JNIEXPORT macro, which would explain the pervasive use of #include > "jni.h" in our code base. > > Thoughts? > > /Magnus > From jcbeyler at google.com Mon Dec 10 17:57:59 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 09:57:59 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> Message-ID: Hi all, Friendly ping for a re-review? :) Jc On Wed, Dec 5, 2018 at 11:36 AM JC Beyler wrote: > Hi all, > > My apologies to having to come back for another review for this change: I > ran into a snag when trying to pull the latest changes compared to the base > I was working on. I basically forgot that there was an issue with snprintf > and that I had solved it via JDK-8213622. > > Could I have a new review of this webrev: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > Incremental from the port of webrev.03 that got LGTMs: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ > > A few comments on this because it took me a while to get things in a state > I thought was good: > - I had to implement an itoa method, do we have something like that in > the test base (remember that JDK-8213622 could not use sprintf due to being > in the test code)? > > - The differences here compared to the one you all reviewed are: > - I found that adding to the strlen/memcpy error prone and thought > that I would try to make it less so. If you want to compare, I extended the > strlen/memcpy with the new format to show you if you prefer [1] > - Note that the diff between the "old extended way from [1]" > to the webrev.04 can be found in [2] > > - I added a test to test the exception wrapper in tests :); I'm not > sure it is deemed useful or not but helped me assure myself that I was not > doing things wrong; you can find the base test file here [3]; should we > have this or not? (I know that normally we don't add tests to vmTestbase > but thought this might be an exception) > > Thanks for your help and my apologies for the snag, > Jc > > [1]: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > [2]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 > [3] > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > > On Mon, Dec 3, 2018 at 11:29 PM David Holmes > wrote: > >> Looks fine to me. >> >> Thanks, >> David >> >> On 4/12/2018 4:04 pm, JC Beyler wrote: >> > Hi both, >> > >> > Thanks for the reviews! Since Serguei did not insist on get_basename, I >> > went for get_dirname since the method is a local static method and >> won't >> > have its name start spreading, I think it's ok too. >> > >> > For the naming of the local variable, the idea initially was to use the >> > same name as the local variable for JNIEnv already used to reduce the >> > code change. Since I'm now adding the line macro at the end anyway, >> this >> > does not matter anymore so I converged all local variables to "jni". >> > >> > So, without further ado, here is the new version: >> > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >> > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >> > This passes the various tests changed by the webrev on my dev machine. >> > >> > Let me know what you think, >> > Jc >> > >> > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com >> > > > > wrote: >> > >> > On 12/3/18 20:15, Chris Plummer wrote: >> > > Hi JC, >> > > >> > > Overall it looks good. A few naming nits thought: >> > > >> > > In bi01t001.cpp, why have you declared the >> > ExceptionCheckingJniEnvPtr >> > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename the >> > > method argument passed in from jni to jni_env. >> > > >> > > Related to this, I also noticed in some files that already are >> using >> > > ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, >> you >> > > delcared it as env(jni_env). So that means there are 3 different >> > names >> > > you have used for the ExceptionCheckingJniEnvPtr local variable. >> > They >> > > should be consistent. >> > > >> > > Also, can you rename get_basename() to get_dirname()? I know >> Serguei >> > > suggested get_basename() a while back, but unless "basename" is >> > > commonly used for this purpose, I think "dirname" is more self >> > > explanatory. >> > >> > In general, I'm Okay with get_dirname(). >> > Just to mention dirname can be both short or full, so it is a little >> > confusing as well. >> > It is the reason why the get_basename() was suggested. >> > However, I do not insist on get_basename() nor get_full_dirname(). >> :) >> > >> > Thanks, >> > Serguei >> > >> > >> > > thanks, >> > > >> > > Chris >> > > >> > > On 12/2/18 10:29 PM, David Holmes wrote: >> > >> Hi Jc, >> > >> >> > >> I've been lurking on this one and have had a look through. I'm >> okay >> > >> with the FatalError approach for the tests - we don't expect >> > anything >> > >> to go wrong in a well written test in a correctly functioning >> VM. >> > >> >> > >> Thanks, >> > >> David >> > >> >> > >> >> > >> >> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: >> > >>> Hi all, >> > >>> >> > >>> Would someone on the GC or runtime team be motivated to give >> > this a >> > >>> review? :) >> > >>> >> > >>> It would be much appreciated! >> > >>> >> > >>> Webrev: >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>> >> > >>> Thanks for your help, >> > >>> Jc >> > >>> >> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > > >> > >>> >> >> wrote: >> > >>> >> > >>> Hi Chris, >> > >>> >> > >>> Yes I was waiting for another review since you had >> explicitly >> > >>> asked :) >> > >>> >> > >>> And sounds good that when someone from GC or runtime gives >> a >> > >>> review, >> > >>> I'll wait for your full review on the webrev.02! >> > >>> >> > >>> Thanks again for your help, >> > >>> Jc >> > >>> >> > >>> >> > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >> > >>> > > >> > > >>> >> > wrote: >> > >>> >> > >>> Hi JC, >> > >>> >> > >>> I think it would be good to get a review from the gc or >> > runtime >> > >>> teams, since this also affects their tests. >> > >>> >> > >>> Also, once we are settled on this FatalError approach, >> > I still >> > >>> need to give your webrev-02 a full review. I only >> > skimmed over >> > >>> parts of it (I did look at all the changes in >> webrevo-01). >> > >>> >> > >>> thanks, >> > >>> >> > >>> Chris >> > >>> >> > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >> > >> > >>> > > > wrote: >> > >>>> Hi Jc, >> > >>>> >> > >>>> I've already reviewed this too. >> > >>>> >> > >>>> Thanks, >> > >>>> Serguei >> > >>>> >> > >>>> >> > >>>> On 11/27/18 06:56, JC Beyler wrote: >> > >>>>> Thanks Chris, >> > >>>>> >> > >>>>> Anybody else motivated to look at this and review >> it? :) >> > >>>>> Jc >> > >>>>> >> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >> > >>>>> > > > > >> >> > >>>>> wrote: >> > >>>>> >> > >>>>> Hi JC, >> > >>>>> >> > >>>>> I'm ok with the FatalError approach, but would >> > like to >> > >>>>> hear opinions from others also. >> > >>>>> >> > >>>>> thanks, >> > >>>>> >> > >>>>> Chris >> > >>>>> >> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >> > >>>>>> Hi Chris, >> > >>>>>> >> > >>>>>> Thanks for taking the time to look at it and >> yes you >> > >>>>>> have raised exactly why the webrev is between >> two >> > >>>>>> worlds: in cases where a fatal error on failure >> is >> > >>>>>> wanted, should we simplify the code to remove >> > the return >> > >>>>>> tests since we do them internally? Now that I've >> > looked >> > >>>>>> around for non-fatal cases, I think the answer >> > is yes, >> > >>>>>> it simplifies the code while maintaining the >> checks. >> > >>>>>> >> > >>>>>> I looked a bit and it seems that I can't find >> > easily a >> > >>>>>> case where the test accepts a JNI failure to >> > then move >> > >>>>>> on. Therefore, perhaps, for now, the fail with a >> > Fatal >> > >>>>>> is enough and we can work on the tests to clean >> > them up? >> > >>>>>> >> > >>>>>> That means that this is the new webrev with only >> > Fatal >> > >>>>>> and cleans up the tests so that it is no longer >> in >> > >>>>>> between two worlds: >> > >>>>>> >> > >>>>>> Webrev: >> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> > >>>>>> >> > >>>>>> Bug: >> > https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>> >> > >>>>>> (This passes testing on my dev machine for all >> the >> > >>>>>> modified tests) >> > >>>>>> >> > >>>>>> with the example you provided, it now looks >> like: >> > >>>>>> >> > >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >>>>>> >> > >>>>>> >> > < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >> > >>>>>> >> > >>>>>> >> > >>>>>> Where it does, to me at least, seem cleaner and >> less >> > >>>>>> "noisy". >> > >>>>>> >> > >>>>>> Let me know what you think, >> > >>>>>> Jc >> > >>>>>> >> > >>>>>> >> > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >> > >>>>>> > > >> > >>>>>> > > >> wrote: >> > >>>>>> >> > >>>>>> Hi JC, >> > >>>>>> >> > >>>>>> Sorry about the delay. I had to go back an >> > look at >> > >>>>>> the initial 8210842 webrev and RFR thread >> to see >> > >>>>>> what this was initially all about. >> > >>>>>> >> > >>>>>> In general the changes look good. >> > >>>>>> >> > >>>>>> I don't have a good answer to your >> > >>>>>> FatalError/NonFatalError question. It makes >> > the code >> > >>>>>> a lot cleaner to use FatalError, but then it >> > is a >> > >>>>>> behavior change, and you also need to deal >> with >> > >>>>>> tests that intentionally induce errors (do >> > you have >> > >>>>>> an example of that). >> > >>>>>> >> > >>>>>> In any case, right now your webrev seems to >> be >> > >>>>>> between two worlds. You are producing >> > FatalError, >> > >>>>>> but still checking results. Here's a good >> > example: >> > >>>>>> >> > >>>>>> >> > >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >>>>>> >> > >>>>>> >> > < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >> > >>>>>> >> > >>>>>> >> > >>>>>> I'm not sure if this is just a temporary >> > state until >> > >>>>>> it was decided which approach to take. >> > >>>>>> >> > >>>>>> thanks, >> > >>>>>> >> > >>>>>> Chris >> > >>>>>> >> > >>>>>> >> > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: >> > >>>>>>> Hi all, >> > >>>>>>> >> > >>>>>>> Chris thought it made sense to have more >> > eyes on >> > >>>>>>> this change than just serviceability as it >> will >> > >>>>>>> modify to tests that are not only >> > serviceability >> > >>>>>>> tests so I've moved this to conversation >> > here :) >> > >>>>>>> >> > >>>>>>> For convenience, I've copy-pasted the >> > initial RFR: >> > >>>>>>> >> > >>>>>>> Could I have a review for the extension and >> > usage >> > >>>>>>> of the ExceptionJniWrapper. This adds >> lines and >> > >>>>>>> filenames to the end of the wrapper JNI >> > methods, >> > >>>>>>> adds tracing, and throws an error if need >> > be. I've >> > >>>>>>> ported the gc/lock files to use the new >> > >>>>>>> TRACE_JNI_CALL add-on and I've ported a few >> > of the >> > >>>>>>> tests that were already changed for the >> > assignment >> > >>>>>>> webrev for JDK-8212884. >> > >>>>>>> >> > >>>>>>> Webrev: >> > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >> > >>>>>>> >> > >>>>>>> Bug: >> > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>>> >> > >>>>>>> For illustration, if I force an error to >> the >> > >>>>>>> AP04/ap04t03 test and set the verbosity on, >> > I get >> > >>>>>>> something like: >> > >>>>>>> >> > >>>>>>> >> Calling JNI method FindClass from >> > >>>>>>> ap04t003.cpp:343 >> > >>>>>>> >> Calling with these parameter(s): >> > >>>>>>> java/lang/Threadd >> > >>>>>>> Wait for thread to finish >> > >>>>>>> << Called JNI method FindClass from >> > >>>>>>> ap04t003.cpp:343 >> > >>>>>>> Exception in thread "Thread-0" >> > >>>>>>> java.lang.NoClassDefFoundError: >> > java/lang/Threadd >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>> >> > >>>>>>> Method) >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>> >> > >>>>>>> Caused by: >> java.lang.ClassNotFoundException: >> > >>>>>>> java.lang.Threadd >> > >>>>>>> at >> > >>>>>>> >> > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> > >>>>>>> ... 3 more >> > >>>>>>> FATAL ERROR in native method: JNI method >> > FindClass >> > >>>>>>> : internal error from ap04t003.cpp:343 >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>> >> > >>>>>>> Method) >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> Questions/comments I have about this are: >> > >>>>>>> - Do we want to force fatal errors when >> a JNI >> > >>>>>>> call fails in general? Most of these tests >> > do the >> > >>>>>>> right thing and test the return of the JNI >> > calls, >> > >>>>>>> for example: >> > >>>>>>> thrClass = >> > jni->FindClass("java/lang/Threadd", >> > >>>>>>> TRACE_JNI_CALL); >> > >>>>>>> if (thrClass == NULL) { >> > >>>>>>> >> > >>>>>>> but now the wrapper actually would do a >> > fatal if >> > >>>>>>> the FindClass call would return a nullptr, >> > so we >> > >>>>>>> could remove that test altogether. What do >> you >> > >>>>>>> think? >> > >>>>>>> - I prefer to leave them as the tests >> then >> > >>>>>>> become closer to what real users would >> have in >> > >>>>>>> their code and is the "recommended" way of >> > doing it >> > >>>>>>> >> > >>>>>>> - The alternative is to use the >> > NonFatalError I >> > >>>>>>> added which then just prints out that >> something >> > >>>>>>> went wrong, letting the test continue. >> Question >> > >>>>>>> will be what should be the default? The >> > fatal or >> > >>>>>>> the non-fatal error handling? >> > >>>>>>> >> > >>>>>>> On a different subject: >> > >>>>>>> - On the new tests, I've removed the >> > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper >> > handles the >> > >>>>>>> tracing and the verify in almost the same >> > way; only >> > >>>>>>> difference I can really tell is that the >> > complain >> > >>>>>>> method from NSK has a max complain before >> > stopping >> > >>>>>>> to "complain"; I have not added that part >> > of the >> > >>>>>>> code in this webrev >> > >>>>>>> >> > >>>>>>> Once we decide on these, I can continue on >> the >> > >>>>>>> files from JDK-8212884 and then do both the >> > >>>>>>> assignment in an if extraction followed-by >> this >> > >>>>>>> type of webrev in an easier fashion. >> > Depending on >> > >>>>>>> decisions here, NSK*VERIFY can be >> deprecated as >> > >>>>>>> well as we go forward. >> > >>>>>>> >> > >>>>>>> Thanks! >> > >>>>>>> Jc >> > >>>>>>> >> > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris >> Plummer >> > >>>>>>> > > >> > >>>>>>> > > >> wrote: >> > >>>>>>> >> > >>>>>>> On 11/19/18 10:07 AM, JC Beyler wrote: >> > >>>>>>>> Hi all, >> > >>>>>>>> >> > >>>>>>>> @David/Chris: should I then push this >> > RFR to >> > >>>>>>>> the hotspot mailing or the runtime >> > one? For >> > >>>>>>>> what it's worth, a lot of the tests >> > under the >> > >>>>>>>> vmTestbase are jvmti so the review >> also >> > >>>>>>>> affects serviceability; it just turns >> > out I >> > >>>>>>>> started with the GC originally and >> > then hit >> > >>>>>>>> some other tests I had touched via the >> > >>>>>>>> assignment extraction. >> > >>>>>>> I think hotspot would be best. >> > >>>>>>> >> > >>>>>>> Chris >> > >>>>>>>> >> > >>>>>>>> @Serguei: Done for the method >> > renaming, for >> > >>>>>>>> the indent, are you talking about >> > going from >> > >>>>>>>> the 8-indent to 4-indent? If so, would >> > it not >> > >>>>>>>> just be better to do a new JBS bug and >> > do the >> > >>>>>>>> whole files in one go? I ask because >> > >>>>>>>> otherwise, it will look a bit weird to >> > have >> > >>>>>>>> parts of the file as 8-indent and >> others >> > >>>>>>>> 4-indent? >> > >>>>>>>> >> > >>>>>>>> Thanks for looking at it! >> > >>>>>>>> Jc >> > >>>>>>>> >> > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM >> > >>>>>>>> serguei.spitsyn at oracle.com > serguei.spitsyn at oracle.com> >> > >>>>>>>> > > > >> > >>>>>>>> > > >> > >>>>>>>> > > >> wrote: >> > >>>>>>>> >> > >>>>>>>> Hi Jc, >> > >>>>>>>> >> > >>>>>>>> We have to start this review >> > anyway. :) >> > >>>>>>>> It looks good to me in general. >> > >>>>>>>> Thank you for your consistency in >> this >> > >>>>>>>> refactoring! >> > >>>>>>>> >> > >>>>>>>> Some minor comments. >> > >>>>>>>> >> > >>>>>>>> >> > >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >> > >> > >>>>>>>> >> > >>>>>>>> +static const char* >> > remove_folders(const >> > >>>>>>>> char* fullname) { I'd suggest to >> > rename >> > >>>>>>>> the function name to something >> > traditional >> > >>>>>>>> like get_basename. Otherwise, it >> > sounds >> > >>>>>>>> like this function has to really >> > remove >> > >>>>>>>> folders. :) Also, all *Locker.cpp >> have >> > >>>>>>>> wrong indent in the bodies of if >> > and while >> > >>>>>>>> statements. Could this be fixed >> > with the >> > >>>>>>>> refactoring? I did not look on how >> > this >> > >>>>>>>> impacts the tests other than >> > >>>>>>>> serviceability. Thanks, Serguei >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> On 11/16/18 19:43, JC Beyler >> wrote: >> > >>>>>>>>> Hi all, >> > >>>>>>>>> >> > >>>>>>>>> Anybody motivated to review >> this? :) >> > >>>>>>>>> Jc >> > >>>>>>>>> >> > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 PM JC >> > Beyler >> > >>>>>>>>> > > >> > >>>>>>>>> > > >> wrote: >> > >>>>>>>>> >> > >>>>>>>>> Hi all, >> > >>>>>>>>> >> > >>>>>>>>> Could I have a review for the >> > >>>>>>>>> extension and usage of the >> > >>>>>>>>> ExceptionJniWrapper. This >> > adds lines >> > >>>>>>>>> and filenames to the end of >> the >> > >>>>>>>>> wrapper JNI methods, adds >> > tracing, >> > >>>>>>>>> and throws an error if need >> > be. I've >> > >>>>>>>>> ported the gc/lock files to >> > use the >> > >>>>>>>>> new TRACE_JNI_CALL add-on and >> > I've >> > >>>>>>>>> ported a few of the tests >> > that were >> > >>>>>>>>> already changed for the >> > assignment >> > >>>>>>>>> webrev for JDK-8212884. >> > >>>>>>>>> >> > >>>>>>>>> Webrev: >> > >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >> > >>>>>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/> >> > >>>>>>>>> Bug: >> > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>>>>> >> > >>>>>>>>> For illustration, if I force >> > an error >> > >>>>>>>>> to the AP04/ap04t03 test and >> > set the >> > >>>>>>>>> verbosity on, I get something >> > like: >> > >>>>>>>>> >> > >>>>>>>>> >> Calling JNI method >> > FindClass from >> > >>>>>>>>> ap04t003.cpp:343 >> > >>>>>>>>> >> Calling with these >> > parameter(s): >> > >>>>>>>>> java/lang/Threadd >> > >>>>>>>>> Wait for thread to finish >> > >>>>>>>>> << Called JNI method >> > FindClass from >> > >>>>>>>>> ap04t003.cpp:343 >> > >>>>>>>>> Exception in thread >> "Thread-0" >> > >>>>>>>>> java.lang.NoClassDefFoundError: >> > >>>>>>>>> java/lang/Threadd >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>>>> >> > >>>>>>>>> Method) >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>>>> >> > >>>>>>>>> Caused by: >> > >>>>>>>>> java.lang.ClassNotFoundException: >> > >>>>>>>>> java.lang.Threadd >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> > >>>>>>>>> ... 3 more >> > >>>>>>>>> FATAL ERROR in native >> method: JNI >> > >>>>>>>>> method FindClass : internal >> error >> > >>>>>>>>> from ap04t003.cpp:343 >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>>>> >> > >>>>>>>>> Method) >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> Questions/comments I have >> about >> > >>>>>>>>> this are: >> > >>>>>>>>> - Do we want to force fatal >> > errors >> > >>>>>>>>> when a JNI call fails in >> general? >> > >>>>>>>>> Most of these tests do the >> right >> > >>>>>>>>> thing and test the return of >> > the JNI >> > >>>>>>>>> calls, for example: >> > >>>>>>>>> thrClass = >> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >> > >>>>>>>>> TRACE_JNI_CALL); >> > >>>>>>>>> if (thrClass == NULL) { >> > >>>>>>>>> >> > >>>>>>>>> but now the wrapper actually >> > would do >> > >>>>>>>>> a fatal if the FindClass call >> > would >> > >>>>>>>>> return a nullptr, so we could >> > remove >> > >>>>>>>>> that test altogether. What do >> > you >> > >>>>>>>>> think? >> > >>>>>>>>> - I prefer to leave them >> > as the >> > >>>>>>>>> tests then become closer to >> > what real >> > >>>>>>>>> users would have in their >> > code and is >> > >>>>>>>>> the "recommended" way of >> doing it >> > >>>>>>>>> >> > >>>>>>>>> - The alternative is to >> > use the >> > >>>>>>>>> NonFatalError I added which >> > then just >> > >>>>>>>>> prints out that something >> > went wrong, >> > >>>>>>>>> letting the test continue. >> > Question >> > >>>>>>>>> will be what should be the >> > default? >> > >>>>>>>>> The fatal or the non-fatal >> error >> > >>>>>>>>> handling? >> > >>>>>>>>> >> > >>>>>>>>> On a different subject: >> > >>>>>>>>> - On the new tests, I've >> > removed >> > >>>>>>>>> the NSK_JNI_VERIFY since the >> JNI >> > >>>>>>>>> wrapper handles the tracing >> > and the >> > >>>>>>>>> verify in almost the same >> > way; only >> > >>>>>>>>> difference I can really tell >> > is that >> > >>>>>>>>> the complain method from NSK >> > has a >> > >>>>>>>>> max complain before stopping >> to >> > >>>>>>>>> "complain"; I have not added >> that >> > >>>>>>>>> part of the code in this >> webrev >> > >>>>>>>>> >> > >>>>>>>>> Once we decide on these, I >> can >> > >>>>>>>>> continue on the files from >> > >>>>>>>>> JDK-8212884 and then do both >> the >> > >>>>>>>>> assignment in an if >> extraction >> > >>>>>>>>> followed-by this type of >> > webrev in an >> > >>>>>>>>> easier fashion. Depending on >> > >>>>>>>>> decisions here, NSK*VERIFY >> can be >> > >>>>>>>>> deprecated as well as we go >> > forward. >> > >>>>>>>>> >> > >>>>>>>>> Thank you for the >> > reviews/comments :) >> > >>>>>>>>> Jc >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> -- >> > >>>>>>>>> Thanks, >> > >>>>>>>>> Jc >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> -- >> > >>>>>>>> Thanks, >> > >>>>>>>> Jc >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> -- >> > >>>>>>> Thanks, >> > >>>>>>> Jc >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> -- >> > >>>>>> Thanks, >> > >>>>>> Jc >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> -- >> > >>>>> Thanks, >> > >>>>> Jc >> > >>>> >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Thanks, >> > >>> Jc >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> >> > >>> Thanks, >> > >>> Jc >> > > >> > > >> > >> > >> > >> > -- >> > >> > Thanks, >> > Jc >> > > > -- > > Thanks, > Jc > -- Thanks, Jc From doko at ubuntu.com Mon Dec 10 18:02:54 2018 From: doko at ubuntu.com (Matthias Klose) Date: Mon, 10 Dec 2018 19:02:54 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> Message-ID: On 10.12.18 11:19, Magnus Ihse Bursie wrote: > > > On 2018-12-09 11:58, Matthias Klose wrote: >> On 06.12.18 22:03, Erik Joelsson wrote: >>> Nothing strange in there. Is it only printed once? I wouldn't be surprised if >>> this got printed more than one time during a normal make execution (due to >>> reloads caused by -include). If it is, then perhaps there is something different >>> in a later print? >> no, it's only printed once. And it seems to be independent of the test-image >> target. just the bootcycle-images target is enough to trigger that [1]. Also not >> architecture specific for the hotspot targets. Builds without the bootcycle >> target succeed [2]. >> >> Anything else wrong with the configury in [1]? > It's a bit hard to say. I'm reacting to "make -C build", maybe the -C does not > play well with bootcycle builds? I don't think that's a very well tested > combination, and bootcycle builds is really like running the build twice in > different directories. But I don't know, it shouldn't affect things... > You are also running with a very detailed log level, it hardly makes it easier > to read the log. I recommend using "info,cmdlines" instead of "debug" unless > actively debugging a hard-to-find issue. I'm using the separate build dir to build the server and zero jvm variants. In the past --with-jvm-variants=server,zero was not supported, and you can't do bootcycle builds for both variants. Replacing LOG=debug with LOG=info,cmdlines still lets the build fail, or was that expected? > ... Found it! Erik's intuition was correct: > > ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 [...snip...] gtest micro > make-make-base make-java-compilation make-copy-files make-idea > make-compile-commands make-make[5]: make-Leaving make-directory > make-'/<>' failure-handler make< > make[5]: Leaving directory '/<>' > make/Main.gmk:1088: *** target pattern contains no '%'.? Stop. > > So once again we're being bit by the make changedir message. And maybe that's > triggered in a new way due to the -C? Try adding --no-print-directory to your > top-level make invocation, as a workaround. Yes, this lets the bootcycle build succeed. Thanks for the pointer! Matthias > > We should probably send that as a flag to make, always. > > /Magnus > >> >> [1] >> https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=amd64&ver=12~23-2&stamp=1544145960&raw=0 >> >> [2] https://buildd.debian.org/status/package.php?p=openjdk-12 >> >> >>> /Erik >>> >>> On 2018-12-06 11:23, Matthias Klose wrote: >>>> On 06.12.18 20:04, Erik Joelsson wrote: >>>>> Could you insert this before line 1087 in make/Main.gmk and post the output? >>>>> >>>>> $(call PrintVar, ALL_NAMED_TESTS) >>>>> >>>>> /Erik >>>>> >>>>> On 2018-12-06 10:31, Matthias Klose wrote: >>>>>> On 06.12.18 17:41, Aleksey Shipilev wrote: >>>>>>> On 12/6/18 4:34 PM, Matthias Klose wrote: >>>>>>>> my bad, that happens in the test-image target: >>>>>>>> >>>>>>>> In the build log I see: >>>>>>>> >>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>>>>>>> >>>>>>>> >>>>>>>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>>> make/Main.gmk:1087: *** target pattern contains no '%'.? Stop. >>>>>>>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >>>>>>> I wonder if "~" in your path screws things up. >>>>>> that worked at least for the openjdk-11 development cycle, and replacing the >>>>>> tilde with a dash makes no difference. >>>> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all >>>> hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co >>>> mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc >>>> hotspot_handshake hotspot_misc hotspot_native_sanity ho >>>> tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime >>>> hotspot_runtime hotspot_runtime_minimalvm ho >>>> tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime >>>> hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru >>>> ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt >>>> jdk_beans jdk_client_sanity jdk_collections jd >>>> k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core >>>> jdk_imageio jdk_instrument jdk_instrument_sanity >>>> jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity >>>> jdk_lang jdk_launcher jdk_management jdk_mana >>>> gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi >>>> jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 >>>> jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity >>>> jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other >>>> jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common >>>> tier1_compiler >>>> tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms >>>> tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher >>>> tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime >>>> tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 >>>> tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages >>>> vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi >>>> vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick >>>> vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring >>>> vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict >>>> vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth >>>> vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact >>>> vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle >>>> vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick >>>> vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro >>>> make-make-base make-java-compilation make-copy-files make-idea >>>> make-compile-commands failure-handler make< > From doko at ubuntu.com Mon Dec 10 18:05:00 2018 From: doko at ubuntu.com (Matthias Klose) Date: Mon, 10 Dec 2018 19:05:00 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: <99e0ec6c-2655-803b-b87b-f2fc323cd9f1@oracle.com> References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> <5a0fc37c20a5c628faad6cb0ad20ec02a21d65f7.camel@redhat.com> <99e0ec6c-2655-803b-b87b-f2fc323cd9f1@oracle.com> Message-ID: <44b9de4d-2bb1-1196-8aca-119205373ab0@ubuntu.com> On 10.12.18 11:49, Magnus Ihse Bursie wrote: > > > On 2018-12-10 11:31, Severin Gehwolf wrote: >> On Mon, 2018-12-10 at 11:19 +0100, Magnus Ihse Bursie wrote: >>> On 2018-12-09 11:58, Matthias Klose wrote: >>>> On 06.12.18 22:03, Erik Joelsson wrote: >>>>> Nothing strange in there. Is it only printed once? I wouldn't be surprised if >>>>> this got printed more than one time during a normal make execution (due to >>>>> reloads caused by -include). If it is, then perhaps there is something >>>>> different >>>>> in a later print? >>>> no, it's only printed once. And it seems to be independent of the test-image >>>> target. just the bootcycle-images target is enough to trigger that [1]. Also >>>> not >>>> architecture specific for the hotspot targets. Builds without the bootcycle >>>> target succeed [2]. >>>> >>>> Anything else wrong with the configury in [1]? >>> It's a bit hard to say. I'm reacting to "make -C build", maybe the -C >>> does not play well with bootcycle builds? I don't think that's a very >>> well tested combination, and bootcycle builds is really like running the >>> build twice in different directories. But I don't know, it shouldn't >>> affect things... >>> You are also running with a very detailed log level, it hardly makes it >>> easier to read the log. I recommend using "info,cmdlines" instead of >>> "debug" unless actively debugging a hard-to-find issue. >>> >>> ... Found it! Erik's intuition was correct: >>> >>> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 [...snip...] gtest >>> micro make-make-base make-java-compilation make-copy-files make-idea >>> make-compile-commands make-make[5]: make-Leaving make-directory >>> make-'/<>' failure-handler make< >>> make[5]: Leaving directory '/<>' >>> make/Main.gmk:1088: *** target pattern contains no '%'.? Stop. >>> >>> So once again we're being bit by the make changedir message. And maybe >>> that's triggered in a new way due to the -C? Try adding >>> --no-print-directory to your top-level make invocation, as a workaround. >>> >>> We should probably send that as a flag to make, always. >> Wasn't this fixed with? >> https://bugs.openjdk.java.net/browse/JDK-8213736 > Unless Matthias is running with an outdated source (Matthias: Are you?), I'm > afraid that the solution in JDK-8213736 was not complete. :( The makefile > bootstraping logic is quite hairy, and I'm sure there's ways to still trigger > the same issue, but differently. I'm basing these packages on the last tag, this one was jdk-12+23. From jcbeyler at google.com Mon Dec 10 20:54:27 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 12:54:27 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> Message-ID: Hi Chris, Thanks for looking again. That's a great point. I looked at the TEST.groups file and it seems to me that maybe we could put it in vmTestbase_nsk_jvmti or vmTestbase_nsk_monitoring? The issue I have in finding the right spot is that there is no "general vmTestbase" spot. Does anyone have a better test group where it could fit? Thanks! Jc On Mon, Dec 10, 2018 at 11:34 AM Chris Plummer wrote: > Hi JC, > > The changes look fine. With regard to your new test, does it get run with > any test group? It seems no. If we are going to keep it, it should be run > automatically with one of the test groups. > > thanks, > > Chris > > On 12/5/18 11:36 AM, JC Beyler wrote: > > Hi all, > > My apologies to having to come back for another review for this change: I > ran into a snag when trying to pull the latest changes compared to the base > I was working on. I basically forgot that there was an issue with snprintf > and that I had solved it via JDK-8213622. > > Could I have a new review of this webrev: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > Incremental from the port of webrev.03 that got LGTMs: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ > > A few comments on this because it took me a while to get things in a state > I thought was good: > - I had to implement an itoa method, do we have something like that in > the test base (remember that JDK-8213622 could not use sprintf due to being > in the test code)? > > - The differences here compared to the one you all reviewed are: > - I found that adding to the strlen/memcpy error prone and thought > that I would try to make it less so. If you want to compare, I extended the > strlen/memcpy with the new format to show you if you prefer [1] > - Note that the diff between the "old extended way from [1]" > to the webrev.04 can be found in [2] > > - I added a test to test the exception wrapper in tests :); I'm not > sure it is deemed useful or not but helped me assure myself that I was not > doing things wrong; you can find the base test file here [3]; should we > have this or not? (I know that normally we don't add tests to vmTestbase > but thought this might be an exception) > > Thanks for your help and my apologies for the snag, > Jc > > [1]: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > [2]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 > [3] > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > > On Mon, Dec 3, 2018 at 11:29 PM David Holmes > wrote: > >> Looks fine to me. >> >> Thanks, >> David >> >> On 4/12/2018 4:04 pm, JC Beyler wrote: >> > Hi both, >> > >> > Thanks for the reviews! Since Serguei did not insist on get_basename, I >> > went for get_dirname since the method is a local static method and >> won't >> > have its name start spreading, I think it's ok too. >> > >> > For the naming of the local variable, the idea initially was to use the >> > same name as the local variable for JNIEnv already used to reduce the >> > code change. Since I'm now adding the line macro at the end anyway, >> this >> > does not matter anymore so I converged all local variables to "jni". >> > >> > So, without further ado, here is the new version: >> > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >> > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >> > This passes the various tests changed by the webrev on my dev machine. >> > >> > Let me know what you think, >> > Jc >> > >> > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com >> > > > > wrote: >> > >> > On 12/3/18 20:15, Chris Plummer wrote: >> > > Hi JC, >> > > >> > > Overall it looks good. A few naming nits thought: >> > > >> > > In bi01t001.cpp, why have you declared the >> > ExceptionCheckingJniEnvPtr >> > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename the >> > > method argument passed in from jni to jni_env. >> > > >> > > Related to this, I also noticed in some files that already are >> using >> > > ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, >> you >> > > delcared it as env(jni_env). So that means there are 3 different >> > names >> > > you have used for the ExceptionCheckingJniEnvPtr local variable. >> > They >> > > should be consistent. >> > > >> > > Also, can you rename get_basename() to get_dirname()? I know >> Serguei >> > > suggested get_basename() a while back, but unless "basename" is >> > > commonly used for this purpose, I think "dirname" is more self >> > > explanatory. >> > >> > In general, I'm Okay with get_dirname(). >> > Just to mention dirname can be both short or full, so it is a little >> > confusing as well. >> > It is the reason why the get_basename() was suggested. >> > However, I do not insist on get_basename() nor get_full_dirname(). >> :) >> > >> > Thanks, >> > Serguei >> > >> > >> > > thanks, >> > > >> > > Chris >> > > >> > > On 12/2/18 10:29 PM, David Holmes wrote: >> > >> Hi Jc, >> > >> >> > >> I've been lurking on this one and have had a look through. I'm >> okay >> > >> with the FatalError approach for the tests - we don't expect >> > anything >> > >> to go wrong in a well written test in a correctly functioning >> VM. >> > >> >> > >> Thanks, >> > >> David >> > >> >> > >> >> > >> >> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: >> > >>> Hi all, >> > >>> >> > >>> Would someone on the GC or runtime team be motivated to give >> > this a >> > >>> review? :) >> > >>> >> > >>> It would be much appreciated! >> > >>> >> > >>> Webrev: >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>> >> > >>> Thanks for your help, >> > >>> Jc >> > >>> >> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > > >> > >>> >> >> wrote: >> > >>> >> > >>> Hi Chris, >> > >>> >> > >>> Yes I was waiting for another review since you had >> explicitly >> > >>> asked :) >> > >>> >> > >>> And sounds good that when someone from GC or runtime gives >> a >> > >>> review, >> > >>> I'll wait for your full review on the webrev.02! >> > >>> >> > >>> Thanks again for your help, >> > >>> Jc >> > >>> >> > >>> >> > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >> > >>> > > >> > > >>> >> > wrote: >> > >>> >> > >>> Hi JC, >> > >>> >> > >>> I think it would be good to get a review from the gc or >> > runtime >> > >>> teams, since this also affects their tests. >> > >>> >> > >>> Also, once we are settled on this FatalError approach, >> > I still >> > >>> need to give your webrev-02 a full review. I only >> > skimmed over >> > >>> parts of it (I did look at all the changes in >> webrevo-01). >> > >>> >> > >>> thanks, >> > >>> >> > >>> Chris >> > >>> >> > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >> > >> > >>> > > > wrote: >> > >>>> Hi Jc, >> > >>>> >> > >>>> I've already reviewed this too. >> > >>>> >> > >>>> Thanks, >> > >>>> Serguei >> > >>>> >> > >>>> >> > >>>> On 11/27/18 06:56, JC Beyler wrote: >> > >>>>> Thanks Chris, >> > >>>>> >> > >>>>> Anybody else motivated to look at this and review >> it? :) >> > >>>>> Jc >> > >>>>> >> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >> > >>>>> > > > > >> >> > >>>>> wrote: >> > >>>>> >> > >>>>> Hi JC, >> > >>>>> >> > >>>>> I'm ok with the FatalError approach, but would >> > like to >> > >>>>> hear opinions from others also. >> > >>>>> >> > >>>>> thanks, >> > >>>>> >> > >>>>> Chris >> > >>>>> >> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >> > >>>>>> Hi Chris, >> > >>>>>> >> > >>>>>> Thanks for taking the time to look at it and >> yes you >> > >>>>>> have raised exactly why the webrev is between >> two >> > >>>>>> worlds: in cases where a fatal error on failure >> is >> > >>>>>> wanted, should we simplify the code to remove >> > the return >> > >>>>>> tests since we do them internally? Now that I've >> > looked >> > >>>>>> around for non-fatal cases, I think the answer >> > is yes, >> > >>>>>> it simplifies the code while maintaining the >> checks. >> > >>>>>> >> > >>>>>> I looked a bit and it seems that I can't find >> > easily a >> > >>>>>> case where the test accepts a JNI failure to >> > then move >> > >>>>>> on. Therefore, perhaps, for now, the fail with a >> > Fatal >> > >>>>>> is enough and we can work on the tests to clean >> > them up? >> > >>>>>> >> > >>>>>> That means that this is the new webrev with only >> > Fatal >> > >>>>>> and cleans up the tests so that it is no longer >> in >> > >>>>>> between two worlds: >> > >>>>>> >> > >>>>>> Webrev: >> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> > >>>>>> >> > >>>>>> Bug: >> > https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>> >> > >>>>>> (This passes testing on my dev machine for all >> the >> > >>>>>> modified tests) >> > >>>>>> >> > >>>>>> with the example you provided, it now looks >> like: >> > >>>>>> >> > >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >>>>>> >> > >>>>>> >> > < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >> > >>>>>> >> > >>>>>> >> > >>>>>> Where it does, to me at least, seem cleaner and >> less >> > >>>>>> "noisy". >> > >>>>>> >> > >>>>>> Let me know what you think, >> > >>>>>> Jc >> > >>>>>> >> > >>>>>> >> > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >> > >>>>>> > > >> > >>>>>> > > >> wrote: >> > >>>>>> >> > >>>>>> Hi JC, >> > >>>>>> >> > >>>>>> Sorry about the delay. I had to go back an >> > look at >> > >>>>>> the initial 8210842 webrev and RFR thread >> to see >> > >>>>>> what this was initially all about. >> > >>>>>> >> > >>>>>> In general the changes look good. >> > >>>>>> >> > >>>>>> I don't have a good answer to your >> > >>>>>> FatalError/NonFatalError question. It makes >> > the code >> > >>>>>> a lot cleaner to use FatalError, but then it >> > is a >> > >>>>>> behavior change, and you also need to deal >> with >> > >>>>>> tests that intentionally induce errors (do >> > you have >> > >>>>>> an example of that). >> > >>>>>> >> > >>>>>> In any case, right now your webrev seems to >> be >> > >>>>>> between two worlds. You are producing >> > FatalError, >> > >>>>>> but still checking results. Here's a good >> > example: >> > >>>>>> >> > >>>>>> >> > >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >>>>>> >> > >>>>>> >> > < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >> > >>>>>> >> > >>>>>> >> > >>>>>> I'm not sure if this is just a temporary >> > state until >> > >>>>>> it was decided which approach to take. >> > >>>>>> >> > >>>>>> thanks, >> > >>>>>> >> > >>>>>> Chris >> > >>>>>> >> > >>>>>> >> > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: >> > >>>>>>> Hi all, >> > >>>>>>> >> > >>>>>>> Chris thought it made sense to have more >> > eyes on >> > >>>>>>> this change than just serviceability as it >> will >> > >>>>>>> modify to tests that are not only >> > serviceability >> > >>>>>>> tests so I've moved this to conversation >> > here :) >> > >>>>>>> >> > >>>>>>> For convenience, I've copy-pasted the >> > initial RFR: >> > >>>>>>> >> > >>>>>>> Could I have a review for the extension and >> > usage >> > >>>>>>> of the ExceptionJniWrapper. This adds >> lines and >> > >>>>>>> filenames to the end of the wrapper JNI >> > methods, >> > >>>>>>> adds tracing, and throws an error if need >> > be. I've >> > >>>>>>> ported the gc/lock files to use the new >> > >>>>>>> TRACE_JNI_CALL add-on and I've ported a few >> > of the >> > >>>>>>> tests that were already changed for the >> > assignment >> > >>>>>>> webrev for JDK-8212884. >> > >>>>>>> >> > >>>>>>> Webrev: >> > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >> > >>>>>>> >> > >>>>>>> Bug: >> > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>>> >> > >>>>>>> For illustration, if I force an error to >> the >> > >>>>>>> AP04/ap04t03 test and set the verbosity on, >> > I get >> > >>>>>>> something like: >> > >>>>>>> >> > >>>>>>> >> Calling JNI method FindClass from >> > >>>>>>> ap04t003.cpp:343 >> > >>>>>>> >> Calling with these parameter(s): >> > >>>>>>> java/lang/Threadd >> > >>>>>>> Wait for thread to finish >> > >>>>>>> << Called JNI method FindClass from >> > >>>>>>> ap04t003.cpp:343 >> > >>>>>>> Exception in thread "Thread-0" >> > >>>>>>> java.lang.NoClassDefFoundError: >> > java/lang/Threadd >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>> >> > >>>>>>> Method) >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>> >> > >>>>>>> Caused by: >> java.lang.ClassNotFoundException: >> > >>>>>>> java.lang.Threadd >> > >>>>>>> at >> > >>>>>>> >> > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> > >>>>>>> ... 3 more >> > >>>>>>> FATAL ERROR in native method: JNI method >> > FindClass >> > >>>>>>> : internal error from ap04t003.cpp:343 >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>> >> > >>>>>>> Method) >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> Questions/comments I have about this are: >> > >>>>>>> - Do we want to force fatal errors when >> a JNI >> > >>>>>>> call fails in general? Most of these tests >> > do the >> > >>>>>>> right thing and test the return of the JNI >> > calls, >> > >>>>>>> for example: >> > >>>>>>> thrClass = >> > jni->FindClass("java/lang/Threadd", >> > >>>>>>> TRACE_JNI_CALL); >> > >>>>>>> if (thrClass == NULL) { >> > >>>>>>> >> > >>>>>>> but now the wrapper actually would do a >> > fatal if >> > >>>>>>> the FindClass call would return a nullptr, >> > so we >> > >>>>>>> could remove that test altogether. What do >> you >> > >>>>>>> think? >> > >>>>>>> - I prefer to leave them as the tests >> then >> > >>>>>>> become closer to what real users would >> have in >> > >>>>>>> their code and is the "recommended" way of >> > doing it >> > >>>>>>> >> > >>>>>>> - The alternative is to use the >> > NonFatalError I >> > >>>>>>> added which then just prints out that >> something >> > >>>>>>> went wrong, letting the test continue. >> Question >> > >>>>>>> will be what should be the default? The >> > fatal or >> > >>>>>>> the non-fatal error handling? >> > >>>>>>> >> > >>>>>>> On a different subject: >> > >>>>>>> - On the new tests, I've removed the >> > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper >> > handles the >> > >>>>>>> tracing and the verify in almost the same >> > way; only >> > >>>>>>> difference I can really tell is that the >> > complain >> > >>>>>>> method from NSK has a max complain before >> > stopping >> > >>>>>>> to "complain"; I have not added that part >> > of the >> > >>>>>>> code in this webrev >> > >>>>>>> >> > >>>>>>> Once we decide on these, I can continue on >> the >> > >>>>>>> files from JDK-8212884 and then do both the >> > >>>>>>> assignment in an if extraction followed-by >> this >> > >>>>>>> type of webrev in an easier fashion. >> > Depending on >> > >>>>>>> decisions here, NSK*VERIFY can be >> deprecated as >> > >>>>>>> well as we go forward. >> > >>>>>>> >> > >>>>>>> Thanks! >> > >>>>>>> Jc >> > >>>>>>> >> > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris >> Plummer >> > >>>>>>> > > >> > >>>>>>> > > >> wrote: >> > >>>>>>> >> > >>>>>>> On 11/19/18 10:07 AM, JC Beyler wrote: >> > >>>>>>>> Hi all, >> > >>>>>>>> >> > >>>>>>>> @David/Chris: should I then push this >> > RFR to >> > >>>>>>>> the hotspot mailing or the runtime >> > one? For >> > >>>>>>>> what it's worth, a lot of the tests >> > under the >> > >>>>>>>> vmTestbase are jvmti so the review >> also >> > >>>>>>>> affects serviceability; it just turns >> > out I >> > >>>>>>>> started with the GC originally and >> > then hit >> > >>>>>>>> some other tests I had touched via the >> > >>>>>>>> assignment extraction. >> > >>>>>>> I think hotspot would be best. >> > >>>>>>> >> > >>>>>>> Chris >> > >>>>>>>> >> > >>>>>>>> @Serguei: Done for the method >> > renaming, for >> > >>>>>>>> the indent, are you talking about >> > going from >> > >>>>>>>> the 8-indent to 4-indent? If so, would >> > it not >> > >>>>>>>> just be better to do a new JBS bug and >> > do the >> > >>>>>>>> whole files in one go? I ask because >> > >>>>>>>> otherwise, it will look a bit weird to >> > have >> > >>>>>>>> parts of the file as 8-indent and >> others >> > >>>>>>>> 4-indent? >> > >>>>>>>> >> > >>>>>>>> Thanks for looking at it! >> > >>>>>>>> Jc >> > >>>>>>>> >> > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM >> > >>>>>>>> serguei.spitsyn at oracle.com > serguei.spitsyn at oracle.com> >> > >>>>>>>> > > > >> > >>>>>>>> > > >> > >>>>>>>> > > >> wrote: >> > >>>>>>>> >> > >>>>>>>> Hi Jc, >> > >>>>>>>> >> > >>>>>>>> We have to start this review >> > anyway. :) >> > >>>>>>>> It looks good to me in general. >> > >>>>>>>> Thank you for your consistency in >> this >> > >>>>>>>> refactoring! >> > >>>>>>>> >> > >>>>>>>> Some minor comments. >> > >>>>>>>> >> > >>>>>>>> >> > >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >> > >> > >>>>>>>> >> > >>>>>>>> +static const char* >> > remove_folders(const >> > >>>>>>>> char* fullname) { I'd suggest to >> > rename >> > >>>>>>>> the function name to something >> > traditional >> > >>>>>>>> like get_basename. Otherwise, it >> > sounds >> > >>>>>>>> like this function has to really >> > remove >> > >>>>>>>> folders. :) Also, all *Locker.cpp >> have >> > >>>>>>>> wrong indent in the bodies of if >> > and while >> > >>>>>>>> statements. Could this be fixed >> > with the >> > >>>>>>>> refactoring? I did not look on how >> > this >> > >>>>>>>> impacts the tests other than >> > >>>>>>>> serviceability. Thanks, Serguei >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> On 11/16/18 19:43, JC Beyler >> wrote: >> > >>>>>>>>> Hi all, >> > >>>>>>>>> >> > >>>>>>>>> Anybody motivated to review >> this? :) >> > >>>>>>>>> Jc >> > >>>>>>>>> >> > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 PM JC >> > Beyler >> > >>>>>>>>> > > >> > >>>>>>>>> > > >> wrote: >> > >>>>>>>>> >> > >>>>>>>>> Hi all, >> > >>>>>>>>> >> > >>>>>>>>> Could I have a review for the >> > >>>>>>>>> extension and usage of the >> > >>>>>>>>> ExceptionJniWrapper. This >> > adds lines >> > >>>>>>>>> and filenames to the end of >> the >> > >>>>>>>>> wrapper JNI methods, adds >> > tracing, >> > >>>>>>>>> and throws an error if need >> > be. I've >> > >>>>>>>>> ported the gc/lock files to >> > use the >> > >>>>>>>>> new TRACE_JNI_CALL add-on and >> > I've >> > >>>>>>>>> ported a few of the tests >> > that were >> > >>>>>>>>> already changed for the >> > assignment >> > >>>>>>>>> webrev for JDK-8212884. >> > >>>>>>>>> >> > >>>>>>>>> Webrev: >> > >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >> > >>>>>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/> >> > >>>>>>>>> Bug: >> > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>>>>> >> > >>>>>>>>> For illustration, if I force >> > an error >> > >>>>>>>>> to the AP04/ap04t03 test and >> > set the >> > >>>>>>>>> verbosity on, I get something >> > like: >> > >>>>>>>>> >> > >>>>>>>>> >> Calling JNI method >> > FindClass from >> > >>>>>>>>> ap04t003.cpp:343 >> > >>>>>>>>> >> Calling with these >> > parameter(s): >> > >>>>>>>>> java/lang/Threadd >> > >>>>>>>>> Wait for thread to finish >> > >>>>>>>>> << Called JNI method >> > FindClass from >> > >>>>>>>>> ap04t003.cpp:343 >> > >>>>>>>>> Exception in thread >> "Thread-0" >> > >>>>>>>>> java.lang.NoClassDefFoundError: >> > >>>>>>>>> java/lang/Threadd >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>>>> >> > >>>>>>>>> Method) >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>>>> >> > >>>>>>>>> Caused by: >> > >>>>>>>>> java.lang.ClassNotFoundException: >> > >>>>>>>>> java.lang.Threadd >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> > >>>>>>>>> ... 3 more >> > >>>>>>>>> FATAL ERROR in native >> method: JNI >> > >>>>>>>>> method FindClass : internal >> error >> > >>>>>>>>> from ap04t003.cpp:343 >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>>>> >> > >>>>>>>>> Method) >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> Questions/comments I have >> about >> > >>>>>>>>> this are: >> > >>>>>>>>> - Do we want to force fatal >> > errors >> > >>>>>>>>> when a JNI call fails in >> general? >> > >>>>>>>>> Most of these tests do the >> right >> > >>>>>>>>> thing and test the return of >> > the JNI >> > >>>>>>>>> calls, for example: >> > >>>>>>>>> thrClass = >> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >> > >>>>>>>>> TRACE_JNI_CALL); >> > >>>>>>>>> if (thrClass == NULL) { >> > >>>>>>>>> >> > >>>>>>>>> but now the wrapper actually >> > would do >> > >>>>>>>>> a fatal if the FindClass call >> > would >> > >>>>>>>>> return a nullptr, so we could >> > remove >> > >>>>>>>>> that test altogether. What do >> > you >> > >>>>>>>>> think? >> > >>>>>>>>> - I prefer to leave them >> > as the >> > >>>>>>>>> tests then become closer to >> > what real >> > >>>>>>>>> users would have in their >> > code and is >> > >>>>>>>>> the "recommended" way of >> doing it >> > >>>>>>>>> >> > >>>>>>>>> - The alternative is to >> > use the >> > >>>>>>>>> NonFatalError I added which >> > then just >> > >>>>>>>>> prints out that something >> > went wrong, >> > >>>>>>>>> letting the test continue. >> > Question >> > >>>>>>>>> will be what should be the >> > default? >> > >>>>>>>>> The fatal or the non-fatal >> error >> > >>>>>>>>> handling? >> > >>>>>>>>> >> > >>>>>>>>> On a different subject: >> > >>>>>>>>> - On the new tests, I've >> > removed >> > >>>>>>>>> the NSK_JNI_VERIFY since the >> JNI >> > >>>>>>>>> wrapper handles the tracing >> > and the >> > >>>>>>>>> verify in almost the same >> > way; only >> > >>>>>>>>> difference I can really tell >> > is that >> > >>>>>>>>> the complain method from NSK >> > has a >> > >>>>>>>>> max complain before stopping >> to >> > >>>>>>>>> "complain"; I have not added >> that >> > >>>>>>>>> part of the code in this >> webrev >> > >>>>>>>>> >> > >>>>>>>>> Once we decide on these, I >> can >> > >>>>>>>>> continue on the files from >> > >>>>>>>>> JDK-8212884 and then do both >> the >> > >>>>>>>>> assignment in an if >> extraction >> > >>>>>>>>> followed-by this type of >> > webrev in an >> > >>>>>>>>> easier fashion. Depending on >> > >>>>>>>>> decisions here, NSK*VERIFY >> can be >> > >>>>>>>>> deprecated as well as we go >> > forward. >> > >>>>>>>>> >> > >>>>>>>>> Thank you for the >> > reviews/comments :) >> > >>>>>>>>> Jc >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> -- >> > >>>>>>>>> Thanks, >> > >>>>>>>>> Jc >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> -- >> > >>>>>>>> Thanks, >> > >>>>>>>> Jc >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> -- >> > >>>>>>> Thanks, >> > >>>>>>> Jc >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> -- >> > >>>>>> Thanks, >> > >>>>>> Jc >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> -- >> > >>>>> Thanks, >> > >>>>> Jc >> > >>>> >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Thanks, >> > >>> Jc >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> >> > >>> Thanks, >> > >>> Jc >> > > >> > > >> > >> > >> > >> > -- >> > >> > Thanks, >> > Jc >> > > > -- > > Thanks, > Jc > > > -- Thanks, Jc From david.holmes at oracle.com Mon Dec 10 21:32:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 07:32:00 +1000 Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <4a3d5c3e-3dfd-4ba8-b929-374e971565aa@default> References: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> <4a3d5c3e-3dfd-4ba8-b929-374e971565aa@default> Message-ID: Hi Markus, Glad to hear this may be of use - thanks for taking a look. With regard to thread->exit(false) placement, it seems somewhat arbitrary as to whether something goes at the end of run() or the beginning of post_run(). So unless there was a strong reason to move something (like no delete before post_run()) I left things as they are. Cheers, David On 11/12/2018 12:49 am, Markus Gronlund wrote: > Hi David, > > Thank you for looking into this. > > I think this approach is in the right direction. Specially as it gives a form and a vocabulary to reason about thread initialization and teardown states. > > I like the fact you provide both shared as well as thread-specific locations for transitions, like: > > Thread::call_run() > > // shared pre-run > // specific pre-run > // shared run > // specific run > // shared post-run > // specific post-run > > This means we can talk about this better now. > > I can for example say, "the hook is thread generic so it can go into "shared post-run"". > > In addition, we can start to bind properties to them, much like you have done with the assertions for thread locals. > It will help a lot to have the guarantee using pre-and post-conditions that a thread will not be deleted somewhere deeper in the call stack. > > I could easily consolidate JFR_ONLY(Jfr::on_thread_exit();), originally located in JavaThread::exit() and NonJavaThread::post_run() into "shared post-run". > > I also elaborated with moving "this->exit(false);" from JavaThread::thread_main_inner() to JavaThread::post_run(), and that seemed to work quite well, although I understand that we are not moving all components at once. > > The most important thing is that we get a protocol and locations in place for where things could be moved (which is what you are addressing). > > Thanks again for doing this > Markus > > > -----Original Message----- > From: David Holmes > Sent: den 10 december 2018 08:08 > To: hotspot-dev developers > Cc: Kim Barrett ; Markus Gr?nlund > Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic > > This is a preliminary RFR to get feedback. I won't be pushing until 13 and the final RFR will have to sync with other changes that may impact what I'm doing here - e.g. Shenandoah GC. > > @Markus: does this help with your JFR event hooks? > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 > Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ > > The primary change is that we introduce initialization and tear-down hooks for the different thread types so that we can consistently apply common actions - like adding/removing NJTs to the NJT::list - and ensure consistent lifecycle management. This is done by introducing virtual > pre_run() and post_run() hooks so that we arrange the non-virtual > Thread::call_run() so that it does: > > void Thread::call_run() { > > // Perform common initialization actions > > register_thread_stack_with_NMT(); > ... > > // Perform initialization actions > this->pre_run(); > > // Invoke ::run() > this->run(); > > // Perform common tear-down actions > > // Perform tear-down actions > this->post_run(); > > } > > And establish some basic lifecycle rules, such as: > - no deletion of the thread is allowed prior to post_run() > - after post_run() 'this' should not be referenced > - Thread::current must be cleared before Thread deletion > > I refactored a little bit of common setup by getting rid of NamedThread::initialize_named_thread(), and having NJT::pre_run() set the name.** > > Added some comments about how the different thread entry points and hooks connect together. > > Clarified some initialization related interactions between thread creation and BarrierSet creation (which Kim is trying to clean up under 8215097). > > Reverted the change in management.cpp put in under JDK-8212207 as it is no longer needed now that NJTs only get added to the list in pre_run() and so must be valid. And somewhat related I added an assert on Linux that osthread->pthread_id() has been set before the call to call_run(). > > Simplified gtest/threadHelper.inline.hpp as the hooks introduced are now supplied by Thread. > > Thanks to Kim Barrett for the updated NJT list management code and discussion on different approaches. > > Thanks, > David > > ** The role of NamedThread seems quite tenous now, but it's still being used to store "stuff" that seems unrelated to being a NamedThread. > Anyway that would be a future cleanup RFE. > From kim.barrett at oracle.com Mon Dec 10 21:38:03 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 16:38:03 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Message-ID: <989AD83F-F734-4DD8-B160-D614ECB0DB11@oracle.com> > On Dec 9, 2018, at 5:05 PM, David Holmes wrote: > > Didn't realize you were going to tackle this so soon. I was just ironing out the wrinkles in 8214097 before sending it for review later today. :) It turned out to be pretty easy. And I thought there was some chance that it would help with 8214097. > On 9/12/2018 6:30 pm, Kim Barrett wrote: >> Please review this change to move the construction of some work gang >> threads by G1 and CMS to after they've created the barrier set. This >> allows the removal of some bootstrapping code needed to support that >> construction order. There isn't any requirement for the old order, >> it seems to just be a historical artifact. >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8215097 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ > > Moving the GC thread creation out of the heap constructor into the heap initialize function seems quite reasonable. Does ZGC already defer the thread creation? Will this impact the merge of Shenandoah? As noted in the CR, ZGC already has the right ordering. Later in this thread Roman says Shenandoah will also need to be changed. That part is up to the Red Hat folks, of course. > Everything else seems fine. (I'll be reworking the BarrierSet creation assertion as part of 8214097.) Thanks. Is there actually any further change to BarrierSet installation needed for 8214097? I guess I will be finding out shortly, when your RFR comes out. From markus.gronlund at oracle.com Mon Dec 10 21:41:15 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Mon, 10 Dec 2018 13:41:15 -0800 (PST) Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: References: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> <4a3d5c3e-3dfd-4ba8-b929-374e971565aa@default> Message-ID: <1d3d2ab8-7e90-4247-9b0f-885e2f49bffc@default> Understood. The reason for suggesting that move is so that "shared post-run" can work with proper JavaThreads (not having oops for threadObj and threadGroup disappearing from underneath). We can take a closer look later. Cheers Markus -----Original Message----- From: David Holmes Sent: den 10 december 2018 22:32 To: Markus Gronlund Cc: Kim Barrett ; hotspot-dev developers Subject: Re: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic Hi Markus, Glad to hear this may be of use - thanks for taking a look. With regard to thread->exit(false) placement, it seems somewhat arbitrary as to whether something goes at the end of run() or the beginning of post_run(). So unless there was a strong reason to move something (like no delete before post_run()) I left things as they are. Cheers, David On 11/12/2018 12:49 am, Markus Gronlund wrote: > Hi David, > > Thank you for looking into this. > > I think this approach is in the right direction. Specially as it gives a form and a vocabulary to reason about thread initialization and teardown states. > > I like the fact you provide both shared as well as thread-specific locations for transitions, like: > > Thread::call_run() > > // shared pre-run > // specific pre-run > // shared run > // specific run > // shared post-run > // specific post-run > > This means we can talk about this better now. > > I can for example say, "the hook is thread generic so it can go into "shared post-run"". > > In addition, we can start to bind properties to them, much like you have done with the assertions for thread locals. > It will help a lot to have the guarantee using pre-and post-conditions that a thread will not be deleted somewhere deeper in the call stack. > > I could easily consolidate JFR_ONLY(Jfr::on_thread_exit();), originally located in JavaThread::exit() and NonJavaThread::post_run() into "shared post-run". > > I also elaborated with moving "this->exit(false);" from JavaThread::thread_main_inner() to JavaThread::post_run(), and that seemed to work quite well, although I understand that we are not moving all components at once. > > The most important thing is that we get a protocol and locations in place for where things could be moved (which is what you are addressing). > > Thanks again for doing this > Markus > > > -----Original Message----- > From: David Holmes > Sent: den 10 december 2018 08:08 > To: hotspot-dev developers > Cc: Kim Barrett ; Markus Gr?nlund > > Subject: Prelim RFR: JDK-8214097: Rework thread initialization and > teardown logic > > This is a preliminary RFR to get feedback. I won't be pushing until 13 and the final RFR will have to sync with other changes that may impact what I'm doing here - e.g. Shenandoah GC. > > @Markus: does this help with your JFR event hooks? > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 > Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ > > The primary change is that we introduce initialization and tear-down > hooks for the different thread types so that we can consistently apply > common actions - like adding/removing NJTs to the NJT::list - and > ensure consistent lifecycle management. This is done by introducing > virtual > pre_run() and post_run() hooks so that we arrange the non-virtual > Thread::call_run() so that it does: > > void Thread::call_run() { > > // Perform common initialization actions > > register_thread_stack_with_NMT(); > ... > > // Perform initialization actions > this->pre_run(); > > // Invoke ::run() > this->run(); > > // Perform common tear-down actions > > // Perform tear-down actions > this->post_run(); > > } > > And establish some basic lifecycle rules, such as: > - no deletion of the thread is allowed prior to post_run() > - after post_run() 'this' should not be referenced > - Thread::current must be cleared before Thread deletion > > I refactored a little bit of common setup by getting rid of > NamedThread::initialize_named_thread(), and having NJT::pre_run() set > the name.** > > Added some comments about how the different thread entry points and hooks connect together. > > Clarified some initialization related interactions between thread creation and BarrierSet creation (which Kim is trying to clean up under 8215097). > > Reverted the change in management.cpp put in under JDK-8212207 as it is no longer needed now that NJTs only get added to the list in pre_run() and so must be valid. And somewhat related I added an assert on Linux that osthread->pthread_id() has been set before the call to call_run(). > > Simplified gtest/threadHelper.inline.hpp as the hooks introduced are now supplied by Thread. > > Thanks to Kim Barrett for the updated NJT list management code and discussion on different approaches. > > Thanks, > David > > ** The role of NamedThread seems quite tenous now, but it's still being used to store "stuff" that seems unrelated to being a NamedThread. > Anyway that would be a future cleanup RFE. > From kim.barrett at oracle.com Mon Dec 10 21:47:59 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 16:47:59 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> Message-ID: <27985338-635C-4331-9010-E65CB44FA6EB@oracle.com> > On Dec 9, 2018, at 5:31 PM, Per Liden wrote: > > On 12/09/2018 11:26 PM, Per Liden wrote: >> Hi, >> On 12/09/2018 11:05 PM, David Holmes wrote: >>> Moving the GC thread creation out of the heap constructor into the heap initialize function seems quite reasonable. Does ZGC already defer the thread creation? Will this impact the merge of Shenandoah? >> ZGC creates the workers in the constructor, and frankly I don't think we want to change that, unless there's some very good reason. I haven't checked Shenandoah. Allocation in the constructor might be a mistake if we were ever going to support VM initialization failure not terminating the process. But that ship sailed a long time ago, and got about as far as the Vasa :) > But I should also add that ZGC registers its barrier set in the constructor too, before workers are created, so I don't think Kim's patch will cause any problems really. Other collectors typically registers their barrier set in initialize(), which I guess is the root of the bootstrapping issue. As noted in the CR, ZGC already does the BarrierSet and thread creation in the desired order. (Observed both by inspection and testing.) It was pretty obvious in ZGCs case; the relevant code is much nicer than for some of the other GCs. From jcbeyler at google.com Mon Dec 10 21:57:47 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 13:57:47 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> References: <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> Message-ID: Hi Serguei, The variables and functions are in a anonymous namespace; my understanding of C++ is that this is equivalent to putting it as static. Hence, I didn't add them there. Does that make sense? Thanks! Jc On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > Hi Jc, > > It looks good in general. > One question though. > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > > I wonder if the variables and functions have to be static. > > Thanks, > Serguei > > > On 12/5/18 11:36, JC Beyler wrote: > > Hi all, > > My apologies to having to come back for another review for this change: I > ran into a snag when trying to pull the latest changes compared to the base > I was working on. I basically forgot that there was an issue with snprintf > and that I had solved it via JDK-8213622. > > Could I have a new review of this webrev: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > Incremental from the port of webrev.03 that got LGTMs: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ > > A few comments on this because it took me a while to get things in a state > I thought was good: > - I had to implement an itoa method, do we have something like that in > the test base (remember that JDK-8213622 could not use sprintf due to being > in the test code)? > > - The differences here compared to the one you all reviewed are: > - I found that adding to the strlen/memcpy error prone and thought > that I would try to make it less so. If you want to compare, I extended the > strlen/memcpy with the new format to show you if you prefer [1] > - Note that the diff between the "old extended way from [1]" > to the webrev.04 can be found in [2] > > - I added a test to test the exception wrapper in tests :); I'm not > sure it is deemed useful or not but helped me assure myself that I was not > doing things wrong; you can find the base test file here [3]; should we > have this or not? (I know that normally we don't add tests to vmTestbase > but thought this might be an exception) > > Thanks for your help and my apologies for the snag, > Jc > > [1]: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > [2]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 > [3] > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > > On Mon, Dec 3, 2018 at 11:29 PM David Holmes > wrote: > >> Looks fine to me. >> >> Thanks, >> David >> >> On 4/12/2018 4:04 pm, JC Beyler wrote: >> > Hi both, >> > >> > Thanks for the reviews! Since Serguei did not insist on get_basename, I >> > went for get_dirname since the method is a local static method and >> won't >> > have its name start spreading, I think it's ok too. >> > >> > For the naming of the local variable, the idea initially was to use the >> > same name as the local variable for JNIEnv already used to reduce the >> > code change. Since I'm now adding the line macro at the end anyway, >> this >> > does not matter anymore so I converged all local variables to "jni". >> > >> > So, without further ado, here is the new version: >> > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >> > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >> > This passes the various tests changed by the webrev on my dev machine. >> > >> > Let me know what you think, >> > Jc >> > >> > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com >> > > > > wrote: >> > >> > On 12/3/18 20:15, Chris Plummer wrote: >> > > Hi JC, >> > > >> > > Overall it looks good. A few naming nits thought: >> > > >> > > In bi01t001.cpp, why have you declared the >> > ExceptionCheckingJniEnvPtr >> > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename the >> > > method argument passed in from jni to jni_env. >> > > >> > > Related to this, I also noticed in some files that already are >> using >> > > ExceptionCheckingJniEnvPtr, such as CharArrayCriticalLocker.cpp, >> you >> > > delcared it as env(jni_env). So that means there are 3 different >> > names >> > > you have used for the ExceptionCheckingJniEnvPtr local variable. >> > They >> > > should be consistent. >> > > >> > > Also, can you rename get_basename() to get_dirname()? I know >> Serguei >> > > suggested get_basename() a while back, but unless "basename" is >> > > commonly used for this purpose, I think "dirname" is more self >> > > explanatory. >> > >> > In general, I'm Okay with get_dirname(). >> > Just to mention dirname can be both short or full, so it is a little >> > confusing as well. >> > It is the reason why the get_basename() was suggested. >> > However, I do not insist on get_basename() nor get_full_dirname(). >> :) >> > >> > Thanks, >> > Serguei >> > >> > >> > > thanks, >> > > >> > > Chris >> > > >> > > On 12/2/18 10:29 PM, David Holmes wrote: >> > >> Hi Jc, >> > >> >> > >> I've been lurking on this one and have had a look through. I'm >> okay >> > >> with the FatalError approach for the tests - we don't expect >> > anything >> > >> to go wrong in a well written test in a correctly functioning >> VM. >> > >> >> > >> Thanks, >> > >> David >> > >> >> > >> >> > >> >> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: >> > >>> Hi all, >> > >>> >> > >>> Would someone on the GC or runtime team be motivated to give >> > this a >> > >>> review? :) >> > >>> >> > >>> It would be much appreciated! >> > >>> >> > >>> Webrev: >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>> >> > >>> Thanks for your help, >> > >>> Jc >> > >>> >> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > > >> > >>> >> >> wrote: >> > >>> >> > >>> Hi Chris, >> > >>> >> > >>> Yes I was waiting for another review since you had >> explicitly >> > >>> asked :) >> > >>> >> > >>> And sounds good that when someone from GC or runtime gives >> a >> > >>> review, >> > >>> I'll wait for your full review on the webrev.02! >> > >>> >> > >>> Thanks again for your help, >> > >>> Jc >> > >>> >> > >>> >> > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >> > >>> > > >> > > >>> >> > wrote: >> > >>> >> > >>> Hi JC, >> > >>> >> > >>> I think it would be good to get a review from the gc or >> > runtime >> > >>> teams, since this also affects their tests. >> > >>> >> > >>> Also, once we are settled on this FatalError approach, >> > I still >> > >>> need to give your webrev-02 a full review. I only >> > skimmed over >> > >>> parts of it (I did look at all the changes in >> webrevo-01). >> > >>> >> > >>> thanks, >> > >>> >> > >>> Chris >> > >>> >> > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >> > >> > >>> > > > wrote: >> > >>>> Hi Jc, >> > >>>> >> > >>>> I've already reviewed this too. >> > >>>> >> > >>>> Thanks, >> > >>>> Serguei >> > >>>> >> > >>>> >> > >>>> On 11/27/18 06:56, JC Beyler wrote: >> > >>>>> Thanks Chris, >> > >>>>> >> > >>>>> Anybody else motivated to look at this and review >> it? :) >> > >>>>> Jc >> > >>>>> >> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >> > >>>>> > > > > >> >> > >>>>> wrote: >> > >>>>> >> > >>>>> Hi JC, >> > >>>>> >> > >>>>> I'm ok with the FatalError approach, but would >> > like to >> > >>>>> hear opinions from others also. >> > >>>>> >> > >>>>> thanks, >> > >>>>> >> > >>>>> Chris >> > >>>>> >> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >> > >>>>>> Hi Chris, >> > >>>>>> >> > >>>>>> Thanks for taking the time to look at it and >> yes you >> > >>>>>> have raised exactly why the webrev is between >> two >> > >>>>>> worlds: in cases where a fatal error on failure >> is >> > >>>>>> wanted, should we simplify the code to remove >> > the return >> > >>>>>> tests since we do them internally? Now that I've >> > looked >> > >>>>>> around for non-fatal cases, I think the answer >> > is yes, >> > >>>>>> it simplifies the code while maintaining the >> checks. >> > >>>>>> >> > >>>>>> I looked a bit and it seems that I can't find >> > easily a >> > >>>>>> case where the test accepts a JNI failure to >> > then move >> > >>>>>> on. Therefore, perhaps, for now, the fail with a >> > Fatal >> > >>>>>> is enough and we can work on the tests to clean >> > them up? >> > >>>>>> >> > >>>>>> That means that this is the new webrev with only >> > Fatal >> > >>>>>> and cleans up the tests so that it is no longer >> in >> > >>>>>> between two worlds: >> > >>>>>> >> > >>>>>> Webrev: >> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> > >>>>>> >> > >>>>>> Bug: >> > https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>> >> > >>>>>> (This passes testing on my dev machine for all >> the >> > >>>>>> modified tests) >> > >>>>>> >> > >>>>>> with the example you provided, it now looks >> like: >> > >>>>>> >> > >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >>>>>> >> > >>>>>> >> > < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >> > >>>>>> >> > >>>>>> >> > >>>>>> Where it does, to me at least, seem cleaner and >> less >> > >>>>>> "noisy". >> > >>>>>> >> > >>>>>> Let me know what you think, >> > >>>>>> Jc >> > >>>>>> >> > >>>>>> >> > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >> > >>>>>> > > >> > >>>>>> > > >> wrote: >> > >>>>>> >> > >>>>>> Hi JC, >> > >>>>>> >> > >>>>>> Sorry about the delay. I had to go back an >> > look at >> > >>>>>> the initial 8210842 webrev and RFR thread >> to see >> > >>>>>> what this was initially all about. >> > >>>>>> >> > >>>>>> In general the changes look good. >> > >>>>>> >> > >>>>>> I don't have a good answer to your >> > >>>>>> FatalError/NonFatalError question. It makes >> > the code >> > >>>>>> a lot cleaner to use FatalError, but then it >> > is a >> > >>>>>> behavior change, and you also need to deal >> with >> > >>>>>> tests that intentionally induce errors (do >> > you have >> > >>>>>> an example of that). >> > >>>>>> >> > >>>>>> In any case, right now your webrev seems to >> be >> > >>>>>> between two worlds. You are producing >> > FatalError, >> > >>>>>> but still checking results. Here's a good >> > example: >> > >>>>>> >> > >>>>>> >> > >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >>>>>> >> > >>>>>> >> > < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> > >> > >> > >>>>>> >> > >>>>>> >> > >>>>>> I'm not sure if this is just a temporary >> > state until >> > >>>>>> it was decided which approach to take. >> > >>>>>> >> > >>>>>> thanks, >> > >>>>>> >> > >>>>>> Chris >> > >>>>>> >> > >>>>>> >> > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: >> > >>>>>>> Hi all, >> > >>>>>>> >> > >>>>>>> Chris thought it made sense to have more >> > eyes on >> > >>>>>>> this change than just serviceability as it >> will >> > >>>>>>> modify to tests that are not only >> > serviceability >> > >>>>>>> tests so I've moved this to conversation >> > here :) >> > >>>>>>> >> > >>>>>>> For convenience, I've copy-pasted the >> > initial RFR: >> > >>>>>>> >> > >>>>>>> Could I have a review for the extension and >> > usage >> > >>>>>>> of the ExceptionJniWrapper. This adds >> lines and >> > >>>>>>> filenames to the end of the wrapper JNI >> > methods, >> > >>>>>>> adds tracing, and throws an error if need >> > be. I've >> > >>>>>>> ported the gc/lock files to use the new >> > >>>>>>> TRACE_JNI_CALL add-on and I've ported a few >> > of the >> > >>>>>>> tests that were already changed for the >> > assignment >> > >>>>>>> webrev for JDK-8212884. >> > >>>>>>> >> > >>>>>>> Webrev: >> > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >> > >>>>>>> >> > >>>>>>> Bug: >> > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>>> >> > >>>>>>> For illustration, if I force an error to >> the >> > >>>>>>> AP04/ap04t03 test and set the verbosity on, >> > I get >> > >>>>>>> something like: >> > >>>>>>> >> > >>>>>>> >> Calling JNI method FindClass from >> > >>>>>>> ap04t003.cpp:343 >> > >>>>>>> >> Calling with these parameter(s): >> > >>>>>>> java/lang/Threadd >> > >>>>>>> Wait for thread to finish >> > >>>>>>> << Called JNI method FindClass from >> > >>>>>>> ap04t003.cpp:343 >> > >>>>>>> Exception in thread "Thread-0" >> > >>>>>>> java.lang.NoClassDefFoundError: >> > java/lang/Threadd >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>> >> > >>>>>>> Method) >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>> >> > >>>>>>> Caused by: >> java.lang.ClassNotFoundException: >> > >>>>>>> java.lang.Threadd >> > >>>>>>> at >> > >>>>>>> >> > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> > >>>>>>> ... 3 more >> > >>>>>>> FATAL ERROR in native method: JNI method >> > FindClass >> > >>>>>>> : internal error from ap04t003.cpp:343 >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>> >> > >>>>>>> Method) >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>> >> > >>>>>>> at >> > >>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> Questions/comments I have about this are: >> > >>>>>>> - Do we want to force fatal errors when >> a JNI >> > >>>>>>> call fails in general? Most of these tests >> > do the >> > >>>>>>> right thing and test the return of the JNI >> > calls, >> > >>>>>>> for example: >> > >>>>>>> thrClass = >> > jni->FindClass("java/lang/Threadd", >> > >>>>>>> TRACE_JNI_CALL); >> > >>>>>>> if (thrClass == NULL) { >> > >>>>>>> >> > >>>>>>> but now the wrapper actually would do a >> > fatal if >> > >>>>>>> the FindClass call would return a nullptr, >> > so we >> > >>>>>>> could remove that test altogether. What do >> you >> > >>>>>>> think? >> > >>>>>>> - I prefer to leave them as the tests >> then >> > >>>>>>> become closer to what real users would >> have in >> > >>>>>>> their code and is the "recommended" way of >> > doing it >> > >>>>>>> >> > >>>>>>> - The alternative is to use the >> > NonFatalError I >> > >>>>>>> added which then just prints out that >> something >> > >>>>>>> went wrong, letting the test continue. >> Question >> > >>>>>>> will be what should be the default? The >> > fatal or >> > >>>>>>> the non-fatal error handling? >> > >>>>>>> >> > >>>>>>> On a different subject: >> > >>>>>>> - On the new tests, I've removed the >> > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper >> > handles the >> > >>>>>>> tracing and the verify in almost the same >> > way; only >> > >>>>>>> difference I can really tell is that the >> > complain >> > >>>>>>> method from NSK has a max complain before >> > stopping >> > >>>>>>> to "complain"; I have not added that part >> > of the >> > >>>>>>> code in this webrev >> > >>>>>>> >> > >>>>>>> Once we decide on these, I can continue on >> the >> > >>>>>>> files from JDK-8212884 and then do both the >> > >>>>>>> assignment in an if extraction followed-by >> this >> > >>>>>>> type of webrev in an easier fashion. >> > Depending on >> > >>>>>>> decisions here, NSK*VERIFY can be >> deprecated as >> > >>>>>>> well as we go forward. >> > >>>>>>> >> > >>>>>>> Thanks! >> > >>>>>>> Jc >> > >>>>>>> >> > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris >> Plummer >> > >>>>>>> > > >> > >>>>>>> > > >> wrote: >> > >>>>>>> >> > >>>>>>> On 11/19/18 10:07 AM, JC Beyler wrote: >> > >>>>>>>> Hi all, >> > >>>>>>>> >> > >>>>>>>> @David/Chris: should I then push this >> > RFR to >> > >>>>>>>> the hotspot mailing or the runtime >> > one? For >> > >>>>>>>> what it's worth, a lot of the tests >> > under the >> > >>>>>>>> vmTestbase are jvmti so the review >> also >> > >>>>>>>> affects serviceability; it just turns >> > out I >> > >>>>>>>> started with the GC originally and >> > then hit >> > >>>>>>>> some other tests I had touched via the >> > >>>>>>>> assignment extraction. >> > >>>>>>> I think hotspot would be best. >> > >>>>>>> >> > >>>>>>> Chris >> > >>>>>>>> >> > >>>>>>>> @Serguei: Done for the method >> > renaming, for >> > >>>>>>>> the indent, are you talking about >> > going from >> > >>>>>>>> the 8-indent to 4-indent? If so, would >> > it not >> > >>>>>>>> just be better to do a new JBS bug and >> > do the >> > >>>>>>>> whole files in one go? I ask because >> > >>>>>>>> otherwise, it will look a bit weird to >> > have >> > >>>>>>>> parts of the file as 8-indent and >> others >> > >>>>>>>> 4-indent? >> > >>>>>>>> >> > >>>>>>>> Thanks for looking at it! >> > >>>>>>>> Jc >> > >>>>>>>> >> > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM >> > >>>>>>>> serguei.spitsyn at oracle.com > serguei.spitsyn at oracle.com> >> > >>>>>>>> > > > >> > >>>>>>>> > > >> > >>>>>>>> > > >> wrote: >> > >>>>>>>> >> > >>>>>>>> Hi Jc, >> > >>>>>>>> >> > >>>>>>>> We have to start this review >> > anyway. :) >> > >>>>>>>> It looks good to me in general. >> > >>>>>>>> Thank you for your consistency in >> this >> > >>>>>>>> refactoring! >> > >>>>>>>> >> > >>>>>>>> Some minor comments. >> > >>>>>>>> >> > >>>>>>>> >> > >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >> > >> > >>>>>>>> >> > >>>>>>>> +static const char* >> > remove_folders(const >> > >>>>>>>> char* fullname) { I'd suggest to >> > rename >> > >>>>>>>> the function name to something >> > traditional >> > >>>>>>>> like get_basename. Otherwise, it >> > sounds >> > >>>>>>>> like this function has to really >> > remove >> > >>>>>>>> folders. :) Also, all *Locker.cpp >> have >> > >>>>>>>> wrong indent in the bodies of if >> > and while >> > >>>>>>>> statements. Could this be fixed >> > with the >> > >>>>>>>> refactoring? I did not look on how >> > this >> > >>>>>>>> impacts the tests other than >> > >>>>>>>> serviceability. Thanks, Serguei >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> On 11/16/18 19:43, JC Beyler >> wrote: >> > >>>>>>>>> Hi all, >> > >>>>>>>>> >> > >>>>>>>>> Anybody motivated to review >> this? :) >> > >>>>>>>>> Jc >> > >>>>>>>>> >> > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 PM JC >> > Beyler >> > >>>>>>>>> > > >> > >>>>>>>>> > > >> wrote: >> > >>>>>>>>> >> > >>>>>>>>> Hi all, >> > >>>>>>>>> >> > >>>>>>>>> Could I have a review for the >> > >>>>>>>>> extension and usage of the >> > >>>>>>>>> ExceptionJniWrapper. This >> > adds lines >> > >>>>>>>>> and filenames to the end of >> the >> > >>>>>>>>> wrapper JNI methods, adds >> > tracing, >> > >>>>>>>>> and throws an error if need >> > be. I've >> > >>>>>>>>> ported the gc/lock files to >> > use the >> > >>>>>>>>> new TRACE_JNI_CALL add-on and >> > I've >> > >>>>>>>>> ported a few of the tests >> > that were >> > >>>>>>>>> already changed for the >> > assignment >> > >>>>>>>>> webrev for JDK-8212884. >> > >>>>>>>>> >> > >>>>>>>>> Webrev: >> > >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >> > >>>>>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/> >> > >>>>>>>>> Bug: >> > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> > >>>>>>>>> >> > >>>>>>>>> For illustration, if I force >> > an error >> > >>>>>>>>> to the AP04/ap04t03 test and >> > set the >> > >>>>>>>>> verbosity on, I get something >> > like: >> > >>>>>>>>> >> > >>>>>>>>> >> Calling JNI method >> > FindClass from >> > >>>>>>>>> ap04t003.cpp:343 >> > >>>>>>>>> >> Calling with these >> > parameter(s): >> > >>>>>>>>> java/lang/Threadd >> > >>>>>>>>> Wait for thread to finish >> > >>>>>>>>> << Called JNI method >> > FindClass from >> > >>>>>>>>> ap04t003.cpp:343 >> > >>>>>>>>> Exception in thread >> "Thread-0" >> > >>>>>>>>> java.lang.NoClassDefFoundError: >> > >>>>>>>>> java/lang/Threadd >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>>>> >> > >>>>>>>>> Method) >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>>>> >> > >>>>>>>>> Caused by: >> > >>>>>>>>> java.lang.ClassNotFoundException: >> > >>>>>>>>> java.lang.Threadd >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> > >>>>>>>>> ... 3 more >> > >>>>>>>>> FATAL ERROR in native >> method: JNI >> > >>>>>>>>> method FindClass : internal >> error >> > >>>>>>>>> from ap04t003.cpp:343 >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> > >>>>>>>>> >> > >>>>>>>>> Method) >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> > >> > >>>>>>>>> >> > >>>>>>>>> at >> > >>>>>>>>> >> > >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> > >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> Questions/comments I have >> about >> > >>>>>>>>> this are: >> > >>>>>>>>> - Do we want to force fatal >> > errors >> > >>>>>>>>> when a JNI call fails in >> general? >> > >>>>>>>>> Most of these tests do the >> right >> > >>>>>>>>> thing and test the return of >> > the JNI >> > >>>>>>>>> calls, for example: >> > >>>>>>>>> thrClass = >> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >> > >>>>>>>>> TRACE_JNI_CALL); >> > >>>>>>>>> if (thrClass == NULL) { >> > >>>>>>>>> >> > >>>>>>>>> but now the wrapper actually >> > would do >> > >>>>>>>>> a fatal if the FindClass call >> > would >> > >>>>>>>>> return a nullptr, so we could >> > remove >> > >>>>>>>>> that test altogether. What do >> > you >> > >>>>>>>>> think? >> > >>>>>>>>> - I prefer to leave them >> > as the >> > >>>>>>>>> tests then become closer to >> > what real >> > >>>>>>>>> users would have in their >> > code and is >> > >>>>>>>>> the "recommended" way of >> doing it >> > >>>>>>>>> >> > >>>>>>>>> - The alternative is to >> > use the >> > >>>>>>>>> NonFatalError I added which >> > then just >> > >>>>>>>>> prints out that something >> > went wrong, >> > >>>>>>>>> letting the test continue. >> > Question >> > >>>>>>>>> will be what should be the >> > default? >> > >>>>>>>>> The fatal or the non-fatal >> error >> > >>>>>>>>> handling? >> > >>>>>>>>> >> > >>>>>>>>> On a different subject: >> > >>>>>>>>> - On the new tests, I've >> > removed >> > >>>>>>>>> the NSK_JNI_VERIFY since the >> JNI >> > >>>>>>>>> wrapper handles the tracing >> > and the >> > >>>>>>>>> verify in almost the same >> > way; only >> > >>>>>>>>> difference I can really tell >> > is that >> > >>>>>>>>> the complain method from NSK >> > has a >> > >>>>>>>>> max complain before stopping >> to >> > >>>>>>>>> "complain"; I have not added >> that >> > >>>>>>>>> part of the code in this >> webrev >> > >>>>>>>>> >> > >>>>>>>>> Once we decide on these, I >> can >> > >>>>>>>>> continue on the files from >> > >>>>>>>>> JDK-8212884 and then do both >> the >> > >>>>>>>>> assignment in an if >> extraction >> > >>>>>>>>> followed-by this type of >> > webrev in an >> > >>>>>>>>> easier fashion. Depending on >> > >>>>>>>>> decisions here, NSK*VERIFY >> can be >> > >>>>>>>>> deprecated as well as we go >> > forward. >> > >>>>>>>>> >> > >>>>>>>>> Thank you for the >> > reviews/comments :) >> > >>>>>>>>> Jc >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> -- >> > >>>>>>>>> Thanks, >> > >>>>>>>>> Jc >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> -- >> > >>>>>>>> Thanks, >> > >>>>>>>> Jc >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> -- >> > >>>>>>> Thanks, >> > >>>>>>> Jc >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> -- >> > >>>>>> Thanks, >> > >>>>>> Jc >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> -- >> > >>>>> Thanks, >> > >>>>> Jc >> > >>>> >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Thanks, >> > >>> Jc >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> >> > >>> Thanks, >> > >>> Jc >> > > >> > > >> > >> > >> > >> > -- >> > >> > Thanks, >> > Jc >> > > > -- > > Thanks, > Jc > > > -- Thanks, Jc From kim.barrett at oracle.com Mon Dec 10 21:59:13 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 16:59:13 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <43c4783a6caf1f1d8d0716731d842b4ae3c02afd.camel@oracle.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <43c4783a6caf1f1d8d0716731d842b4ae3c02afd.camel@oracle.com> Message-ID: > On Dec 10, 2018, at 5:20 AM, Thomas Schatzl wrote: > > Hi, > > On Sun, 2018-12-09 at 03:30 -0500, Kim Barrett wrote: >> Please review this change to move the construction of some work gang >> threads by G1 and CMS to after they've created the barrier set. This >> allows the removal of some bootstrapping code needed to support that >> construction order. There isn't any requirement for the old order, >> it seems to just be a historical artifact. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8215097 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8215097/open.00/ >> >> Testing: >> mach5 tier1-5 > > looks good. > > Thomas Thanks. From kim.barrett at oracle.com Mon Dec 10 21:59:01 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 16:59:01 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <1c4c5dcd-cf0e-faf7-c207-e3a630fb4c7c@redhat.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <1c4c5dcd-cf0e-faf7-c207-e3a630fb4c7c@redhat.com> Message-ID: <9F9C227D-3437-4B3C-B308-971C9CAC8846@oracle.com> > On Dec 9, 2018, at 5:37 PM, Roman Kennke wrote: > Shenandoah also creates workers in constructors. May I request to hold > this off a little bit to avoid making a mess out of the very soon > upstream push? I'm open to moving this into initialize() but I cannot > tell real quick if it has or hasn't any funny side-effects. Looks like Shenandoah has been pushed. From kim.barrett at oracle.com Mon Dec 10 22:59:39 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 17:59:39 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <989AD83F-F734-4DD8-B160-D614ECB0DB11@oracle.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <989AD83F-F734-4DD8-B160-D614ECB0DB11@oracle.com> Message-ID: > On Dec 10, 2018, at 4:38 PM, Kim Barrett wrote: > >> On Dec 9, 2018, at 5:05 PM, David Holmes wrote: >> >> Didn't realize you were going to tackle this so soon. I was just ironing out the wrinkles in 8214097 before sending it for review later today. :) > > It turned out to be pretty easy. And I thought there was some chance that it would help with 8214097. > > >> Everything else seems fine. (I'll be reworking the BarrierSet creation assertion as part of 8214097.) > > Thanks. > > Is there actually any further change to BarrierSet installation needed for 8214097? I guess I will be finding out shortly, when your RFR comes out. I should have skimmed ahead in my mailbox; your RFR is already available in preliminary form. I?d forgotten how much bootstrapping clunkiness was needed for 8214097 without this change. So yeah, looks like doing mine first does help yours. Looking at yours, I kind of want the remaining bootstrapping in mine to use your change to set_as_starting_thread, (e.g. use your new _starting_thread to recognize it), except I think that function is called too late to use for that purpose. From david.holmes at oracle.com Mon Dec 10 23:00:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 09:00:49 +1000 Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <1d3d2ab8-7e90-4247-9b0f-885e2f49bffc@default> References: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> <4a3d5c3e-3dfd-4ba8-b929-374e971565aa@default> <1d3d2ab8-7e90-4247-9b0f-885e2f49bffc@default> Message-ID: <45d16d43-8b7b-573e-168f-ae21c6947666@oracle.com> On 11/12/2018 7:41 am, Markus Gronlund wrote: > Understood. > > The reason for suggesting that move is so that "shared post-run" can work with proper JavaThreads (not having oops for threadObj and threadGroup disappearing from underneath). Sure -if we need some additional guarantees for the "shared post-run" then we can move things to meet those guarantees. BTW I got tripped up by the gtest threads initially. AFAICS these "JavaThreads" don't have a Java threadObj! So care needs to be taken about assumptions during post-run. Cheers, David > We can take a closer look later. > > Cheers > Markus > > -----Original Message----- > From: David Holmes > Sent: den 10 december 2018 22:32 > To: Markus Gronlund > Cc: Kim Barrett ; hotspot-dev developers > Subject: Re: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic > > Hi Markus, > > Glad to hear this may be of use - thanks for taking a look. > > With regard to thread->exit(false) placement, it seems somewhat arbitrary as to whether something goes at the end of run() or the beginning of post_run(). So unless there was a strong reason to move something (like no delete before post_run()) I left things as they are. > > Cheers, > David > > On 11/12/2018 12:49 am, Markus Gronlund wrote: >> Hi David, >> >> Thank you for looking into this. >> >> I think this approach is in the right direction. Specially as it gives a form and a vocabulary to reason about thread initialization and teardown states. >> >> I like the fact you provide both shared as well as thread-specific locations for transitions, like: >> >> Thread::call_run() >> >> // shared pre-run >> // specific pre-run >> // shared run >> // specific run >> // shared post-run >> // specific post-run >> >> This means we can talk about this better now. >> >> I can for example say, "the hook is thread generic so it can go into "shared post-run"". >> >> In addition, we can start to bind properties to them, much like you have done with the assertions for thread locals. >> It will help a lot to have the guarantee using pre-and post-conditions that a thread will not be deleted somewhere deeper in the call stack. >> >> I could easily consolidate JFR_ONLY(Jfr::on_thread_exit();), originally located in JavaThread::exit() and NonJavaThread::post_run() into "shared post-run". >> >> I also elaborated with moving "this->exit(false);" from JavaThread::thread_main_inner() to JavaThread::post_run(), and that seemed to work quite well, although I understand that we are not moving all components at once. >> >> The most important thing is that we get a protocol and locations in place for where things could be moved (which is what you are addressing). >> >> Thanks again for doing this >> Markus >> >> >> -----Original Message----- >> From: David Holmes >> Sent: den 10 december 2018 08:08 >> To: hotspot-dev developers >> Cc: Kim Barrett ; Markus Gr?nlund >> >> Subject: Prelim RFR: JDK-8214097: Rework thread initialization and >> teardown logic >> >> This is a preliminary RFR to get feedback. I won't be pushing until 13 and the final RFR will have to sync with other changes that may impact what I'm doing here - e.g. Shenandoah GC. >> >> @Markus: does this help with your JFR event hooks? >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ >> >> The primary change is that we introduce initialization and tear-down >> hooks for the different thread types so that we can consistently apply >> common actions - like adding/removing NJTs to the NJT::list - and >> ensure consistent lifecycle management. This is done by introducing >> virtual >> pre_run() and post_run() hooks so that we arrange the non-virtual >> Thread::call_run() so that it does: >> >> void Thread::call_run() { >> >> // Perform common initialization actions >> >> register_thread_stack_with_NMT(); >> ... >> >> // Perform initialization actions >> this->pre_run(); >> >> // Invoke ::run() >> this->run(); >> >> // Perform common tear-down actions >> >> // Perform tear-down actions >> this->post_run(); >> >> } >> >> And establish some basic lifecycle rules, such as: >> - no deletion of the thread is allowed prior to post_run() >> - after post_run() 'this' should not be referenced >> - Thread::current must be cleared before Thread deletion >> >> I refactored a little bit of common setup by getting rid of >> NamedThread::initialize_named_thread(), and having NJT::pre_run() set >> the name.** >> >> Added some comments about how the different thread entry points and hooks connect together. >> >> Clarified some initialization related interactions between thread creation and BarrierSet creation (which Kim is trying to clean up under 8215097). >> >> Reverted the change in management.cpp put in under JDK-8212207 as it is no longer needed now that NJTs only get added to the list in pre_run() and so must be valid. And somewhat related I added an assert on Linux that osthread->pthread_id() has been set before the call to call_run(). >> >> Simplified gtest/threadHelper.inline.hpp as the hooks introduced are now supplied by Thread. >> >> Thanks to Kim Barrett for the updated NJT list management code and discussion on different approaches. >> >> Thanks, >> David >> >> ** The role of NamedThread seems quite tenous now, but it's still being used to store "stuff" that seems unrelated to being a NamedThread. >> Anyway that would be a future cleanup RFE. >> From jcbeyler at google.com Mon Dec 10 23:10:48 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 15:10:48 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> References: <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> Message-ID: Hi Serguei, Yes basically it is equivalent :) I can put them in but they are not required. The norm actually wanted to deprecate it but then remembered that C compatibility would require the static key-word for this case [1] So, really, they are not required here and will amount to the same thing: only that file can refer to them and you cannot get to them without a globally available method to return a pointer to them (ie same as a static variable in C). I can put static if it makes it easier to see but, by being in an anonymous namespace they are only available for the file's translation unit. For example: $ cat main.cpp int totally_global; static int explictly_static; namespace { int implicitly_static; } void foo(); int main() { foo(); } $ g++ -O3 main.cpp -c $ nm main.o U _GLOBAL_OFFSET_TABLE_ 0000000000000000 T main 0000000000000000 B totally_global U _Z3foov As you can see, the static and anonymous namespace variables are not in the file due to not being used. If you were to use them, you'd see them show up as something like: 0000000000000008 b _ZL17explicitly_static 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE Where again, it shows that it is mangling the names so that no external usage can happen without tinkering. Hopefully that helps :-), Jc [1] http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > Hi Jc, > > I had little experience with the C++ namespaces. > My understanding is that static in this context should mean internal > linkage. > > Thanks, > Serguei > > > On 12/10/18 13:57, JC Beyler wrote: > > Hi Serguei, > > The variables and functions are in a anonymous namespace; my understanding > of C++ is that this is equivalent to putting it as static.Hence, I didn't > add them there. Does that make sense? > > Thanks! > Jc > > On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com < > serguei.spitsyn at oracle.com> wrote: > >> Hi Jc, >> >> It looks good in general. >> One question though. >> >> >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >> >> I wonder if the variables and functions have to be static. >> >> Thanks, >> Serguei >> >> >> On 12/5/18 11:36, JC Beyler wrote: >> >> Hi all, >> >> My apologies to having to come back for another review for this change: I >> ran into a snag when trying to pull the latest changes compared to the base >> I was working on. I basically forgot that there was an issue with snprintf >> and that I had solved it via JDK-8213622. >> >> Could I have a new review of this webrev: >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> Incremental from the port of webrev.03 that got LGTMs: >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ >> >> A few comments on this because it took me a while to get things in a >> state I thought was good: >> - I had to implement an itoa method, do we have something like that in >> the test base (remember that JDK-8213622 could not use sprintf due to being >> in the test code)? >> >> - The differences here compared to the one you all reviewed are: >> - I found that adding to the strlen/memcpy error prone and thought >> that I would try to make it less so. If you want to compare, I extended the >> strlen/memcpy with the new format to show you if you prefer [1] >> - Note that the diff between the "old extended way from [1]" >> to the webrev.04 can be found in [2] >> >> - I added a test to test the exception wrapper in tests :); I'm not >> sure it is deemed useful or not but helped me assure myself that I was not >> doing things wrong; you can find the base test file here [3]; should we >> have this or not? (I know that normally we don't add tests to vmTestbase >> but thought this might be an exception) >> >> Thanks for your help and my apologies for the snag, >> Jc >> >> [1]: >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >> [2]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 >> [3] >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >> >> On Mon, Dec 3, 2018 at 11:29 PM David Holmes >> wrote: >> >>> Looks fine to me. >>> >>> Thanks, >>> David >>> >>> On 4/12/2018 4:04 pm, JC Beyler wrote: >>> > Hi both, >>> > >>> > Thanks for the reviews! Since Serguei did not insist on get_basename, >>> I >>> > went for get_dirname since the method is a local static method and >>> won't >>> > have its name start spreading, I think it's ok too. >>> > >>> > For the naming of the local variable, the idea initially was to use >>> the >>> > same name as the local variable for JNIEnv already used to reduce the >>> > code change. Since I'm now adding the line macro at the end anyway, >>> this >>> > does not matter anymore so I converged all local variables to "jni". >>> > >>> > So, without further ado, here is the new version: >>> > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >>> > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> > >>> > This passes the various tests changed by the webrev on my dev machine. >>> > >>> > Let me know what you think, >>> > Jc >>> > >>> > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com >>> > >> > > wrote: >>> > >>> > On 12/3/18 20:15, Chris Plummer wrote: >>> > > Hi JC, >>> > > >>> > > Overall it looks good. A few naming nits thought: >>> > > >>> > > In bi01t001.cpp, why have you declared the >>> > ExceptionCheckingJniEnvPtr >>> > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename >>> the >>> > > method argument passed in from jni to jni_env. >>> > > >>> > > Related to this, I also noticed in some files that already are >>> using >>> > > ExceptionCheckingJniEnvPtr, such as >>> CharArrayCriticalLocker.cpp, you >>> > > delcared it as env(jni_env). So that means there are 3 different >>> > names >>> > > you have used for the ExceptionCheckingJniEnvPtr local variable. >>> > They >>> > > should be consistent. >>> > > >>> > > Also, can you rename get_basename() to get_dirname()? I know >>> Serguei >>> > > suggested get_basename() a while back, but unless "basename" is >>> > > commonly used for this purpose, I think "dirname" is more self >>> > > explanatory. >>> > >>> > In general, I'm Okay with get_dirname(). >>> > Just to mention dirname can be both short or full, so it is a >>> little >>> > confusing as well. >>> > It is the reason why the get_basename() was suggested. >>> > However, I do not insist on get_basename() nor get_full_dirname(). >>> :) >>> > >>> > Thanks, >>> > Serguei >>> > >>> > >>> > > thanks, >>> > > >>> > > Chris >>> > > >>> > > On 12/2/18 10:29 PM, David Holmes wrote: >>> > >> Hi Jc, >>> > >> >>> > >> I've been lurking on this one and have had a look through. I'm >>> okay >>> > >> with the FatalError approach for the tests - we don't expect >>> > anything >>> > >> to go wrong in a well written test in a correctly functioning >>> VM. >>> > >> >>> > >> Thanks, >>> > >> David >>> > >> >>> > >> >>> > >> >>> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: >>> > >>> Hi all, >>> > >>> >>> > >>> Would someone on the GC or runtime team be motivated to give >>> > this a >>> > >>> review? :) >>> > >>> >>> > >>> It would be much appreciated! >>> > >>> >>> > >>> Webrev: >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> > >>> >>> > >>> Thanks for your help, >>> > >>> Jc >>> > >>> >>> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler < >>> jcbeyler at google.com >>> > >>> > >>> >> >>> wrote: >>> > >>> >>> > >>> Hi Chris, >>> > >>> >>> > >>> Yes I was waiting for another review since you had >>> explicitly >>> > >>> asked :) >>> > >>> >>> > >>> And sounds good that when someone from GC or runtime >>> gives a >>> > >>> review, >>> > >>> I'll wait for your full review on the webrev.02! >>> > >>> >>> > >>> Thanks again for your help, >>> > >>> Jc >>> > >>> >>> > >>> >>> > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >>> > >>> >> chris.plummer at oracle.com> >>> > >> >>> >>> > wrote: >>> > >>> >>> > >>> Hi JC, >>> > >>> >>> > >>> I think it would be good to get a review from the gc >>> or >>> > runtime >>> > >>> teams, since this also affects their tests. >>> > >>> >>> > >>> Also, once we are settled on this FatalError approach, >>> > I still >>> > >>> need to give your webrev-02 a full review. I only >>> > skimmed over >>> > >>> parts of it (I did look at all the changes in >>> webrevo-01). >>> > >>> >>> > >>> thanks, >>> > >>> >>> > >>> Chris >>> > >>> >>> > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >>> > >>> > >>> >> > > wrote: >>> > >>>> Hi Jc, >>> > >>>> >>> > >>>> I've already reviewed this too. >>> > >>>> >>> > >>>> Thanks, >>> > >>>> Serguei >>> > >>>> >>> > >>>> >>> > >>>> On 11/27/18 06:56, JC Beyler wrote: >>> > >>>>> Thanks Chris, >>> > >>>>> >>> > >>>>> Anybody else motivated to look at this and review >>> it? :) >>> > >>>>> Jc >>> > >>>>> >>> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >>> > >>>>> >> > >> > >> >>> > >>>>> wrote: >>> > >>>>> >>> > >>>>> Hi JC, >>> > >>>>> >>> > >>>>> I'm ok with the FatalError approach, but would >>> > like to >>> > >>>>> hear opinions from others also. >>> > >>>>> >>> > >>>>> thanks, >>> > >>>>> >>> > >>>>> Chris >>> > >>>>> >>> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >>> > >>>>>> Hi Chris, >>> > >>>>>> >>> > >>>>>> Thanks for taking the time to look at it and >>> yes you >>> > >>>>>> have raised exactly why the webrev is between >>> two >>> > >>>>>> worlds: in cases where a fatal error on >>> failure is >>> > >>>>>> wanted, should we simplify the code to remove >>> > the return >>> > >>>>>> tests since we do them internally? Now that >>> I've >>> > looked >>> > >>>>>> around for non-fatal cases, I think the answer >>> > is yes, >>> > >>>>>> it simplifies the code while maintaining the >>> checks. >>> > >>>>>> >>> > >>>>>> I looked a bit and it seems that I can't find >>> > easily a >>> > >>>>>> case where the test accepts a JNI failure to >>> > then move >>> > >>>>>> on. Therefore, perhaps, for now, the fail with >>> a >>> > Fatal >>> > >>>>>> is enough and we can work on the tests to clean >>> > them up? >>> > >>>>>> >>> > >>>>>> That means that this is the new webrev with >>> only >>> > Fatal >>> > >>>>>> and cleans up the tests so that it is no >>> longer in >>> > >>>>>> between two worlds: >>> > >>>>>> >>> > >>>>>> Webrev: >>> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>> > >>>>>> >> > >>> > >>>>>> Bug: >>> > https://bugs.openjdk.java.net/browse/JDK-8213501 >>> > >>>>>> >>> > >>>>>> (This passes testing on my dev machine for all >>> the >>> > >>>>>> modified tests) >>> > >>>>>> >>> > >>>>>> with the example you provided, it now looks >>> like: >>> > >>>>>> >>> > >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>> > >>> > >>>>>> >>> > >>>>>> >>> > < >>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>> > >>> > >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> Where it does, to me at least, seem cleaner >>> and less >>> > >>>>>> "noisy". >>> > >>>>>> >>> > >>>>>> Let me know what you think, >>> > >>>>>> Jc >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >>> > >>>>>> >> > >>> > >>>>>> >> > >> wrote: >>> > >>>>>> >>> > >>>>>> Hi JC, >>> > >>>>>> >>> > >>>>>> Sorry about the delay. I had to go back an >>> > look at >>> > >>>>>> the initial 8210842 webrev and RFR thread >>> to see >>> > >>>>>> what this was initially all about. >>> > >>>>>> >>> > >>>>>> In general the changes look good. >>> > >>>>>> >>> > >>>>>> I don't have a good answer to your >>> > >>>>>> FatalError/NonFatalError question. It makes >>> > the code >>> > >>>>>> a lot cleaner to use FatalError, but then >>> it >>> > is a >>> > >>>>>> behavior change, and you also need to deal >>> with >>> > >>>>>> tests that intentionally induce errors (do >>> > you have >>> > >>>>>> an example of that). >>> > >>>>>> >>> > >>>>>> In any case, right now your webrev seems >>> to be >>> > >>>>>> between two worlds. You are producing >>> > FatalError, >>> > >>>>>> but still checking results. Here's a good >>> > example: >>> > >>>>>> >>> > >>>>>> >>> > >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>> > >>> > >>>>>> >>> > >>>>>> >>> > < >>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>> > >>> > >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> I'm not sure if this is just a temporary >>> > state until >>> > >>>>>> it was decided which approach to take. >>> > >>>>>> >>> > >>>>>> thanks, >>> > >>>>>> >>> > >>>>>> Chris >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: >>> > >>>>>>> Hi all, >>> > >>>>>>> >>> > >>>>>>> Chris thought it made sense to have more >>> > eyes on >>> > >>>>>>> this change than just serviceability as >>> it will >>> > >>>>>>> modify to tests that are not only >>> > serviceability >>> > >>>>>>> tests so I've moved this to conversation >>> > here :) >>> > >>>>>>> >>> > >>>>>>> For convenience, I've copy-pasted the >>> > initial RFR: >>> > >>>>>>> >>> > >>>>>>> Could I have a review for the extension >>> and >>> > usage >>> > >>>>>>> of the ExceptionJniWrapper. This adds >>> lines and >>> > >>>>>>> filenames to the end of the wrapper JNI >>> > methods, >>> > >>>>>>> adds tracing, and throws an error if need >>> > be. I've >>> > >>>>>>> ported the gc/lock files to use the new >>> > >>>>>>> TRACE_JNI_CALL add-on and I've ported a >>> few >>> > of the >>> > >>>>>>> tests that were already changed for the >>> > assignment >>> > >>>>>>> webrev for JDK-8212884. >>> > >>>>>>> >>> > >>>>>>> Webrev: >>> > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>> > >>>>>>> >> > >>> > >>>>>>> Bug: >>> > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>> > >>>>>>> >>> > >>>>>>> For illustration, if I force an error to >>> the >>> > >>>>>>> AP04/ap04t03 test and set the verbosity >>> on, >>> > I get >>> > >>>>>>> something like: >>> > >>>>>>> >>> > >>>>>>> >> Calling JNI method FindClass from >>> > >>>>>>> ap04t003.cpp:343 >>> > >>>>>>> >> Calling with these parameter(s): >>> > >>>>>>> java/lang/Threadd >>> > >>>>>>> Wait for thread to finish >>> > >>>>>>> << Called JNI method FindClass from >>> > >>>>>>> ap04t003.cpp:343 >>> > >>>>>>> Exception in thread "Thread-0" >>> > >>>>>>> java.lang.NoClassDefFoundError: >>> > java/lang/Threadd >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> > >>>>>>> >>> > >>>>>>> Method) >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> > >>> > >>>>>>> >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> > >>> > >>>>>>> >>> > >>>>>>> Caused by: >>> java.lang.ClassNotFoundException: >>> > >>>>>>> java.lang.Threadd >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>> > >>> > >>>>>>> >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>> > >>> > >>>>>>> >>> > >>>>>>> at >>> > >>>>>>> >>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>> > >>>>>>> ... 3 more >>> > >>>>>>> FATAL ERROR in native method: JNI method >>> > FindClass >>> > >>>>>>> : internal error from ap04t003.cpp:343 >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> > >>>>>>> >>> > >>>>>>> Method) >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> > >>> > >>>>>>> >>> > >>>>>>> at >>> > >>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> > >>> > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> Questions/comments I have about this are: >>> > >>>>>>> - Do we want to force fatal errors when >>> a JNI >>> > >>>>>>> call fails in general? Most of these tests >>> > do the >>> > >>>>>>> right thing and test the return of the JNI >>> > calls, >>> > >>>>>>> for example: >>> > >>>>>>> thrClass = >>> > jni->FindClass("java/lang/Threadd", >>> > >>>>>>> TRACE_JNI_CALL); >>> > >>>>>>> if (thrClass == NULL) { >>> > >>>>>>> >>> > >>>>>>> but now the wrapper actually would do a >>> > fatal if >>> > >>>>>>> the FindClass call would return a nullptr, >>> > so we >>> > >>>>>>> could remove that test altogether. What >>> do you >>> > >>>>>>> think? >>> > >>>>>>> - I prefer to leave them as the tests >>> then >>> > >>>>>>> become closer to what real users would >>> have in >>> > >>>>>>> their code and is the "recommended" way of >>> > doing it >>> > >>>>>>> >>> > >>>>>>> - The alternative is to use the >>> > NonFatalError I >>> > >>>>>>> added which then just prints out that >>> something >>> > >>>>>>> went wrong, letting the test continue. >>> Question >>> > >>>>>>> will be what should be the default? The >>> > fatal or >>> > >>>>>>> the non-fatal error handling? >>> > >>>>>>> >>> > >>>>>>> On a different subject: >>> > >>>>>>> - On the new tests, I've removed the >>> > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper >>> > handles the >>> > >>>>>>> tracing and the verify in almost the same >>> > way; only >>> > >>>>>>> difference I can really tell is that the >>> > complain >>> > >>>>>>> method from NSK has a max complain before >>> > stopping >>> > >>>>>>> to "complain"; I have not added that part >>> > of the >>> > >>>>>>> code in this webrev >>> > >>>>>>> >>> > >>>>>>> Once we decide on these, I can continue >>> on the >>> > >>>>>>> files from JDK-8212884 and then do both >>> the >>> > >>>>>>> assignment in an if extraction >>> followed-by this >>> > >>>>>>> type of webrev in an easier fashion. >>> > Depending on >>> > >>>>>>> decisions here, NSK*VERIFY can be >>> deprecated as >>> > >>>>>>> well as we go forward. >>> > >>>>>>> >>> > >>>>>>> Thanks! >>> > >>>>>>> Jc >>> > >>>>>>> >>> > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris >>> Plummer >>> > >>>>>>> >> > >>> > >>>>>>> >> > >> wrote: >>> > >>>>>>> >>> > >>>>>>> On 11/19/18 10:07 AM, JC Beyler wrote: >>> > >>>>>>>> Hi all, >>> > >>>>>>>> >>> > >>>>>>>> @David/Chris: should I then push this >>> > RFR to >>> > >>>>>>>> the hotspot mailing or the runtime >>> > one? For >>> > >>>>>>>> what it's worth, a lot of the tests >>> > under the >>> > >>>>>>>> vmTestbase are jvmti so the review >>> also >>> > >>>>>>>> affects serviceability; it just turns >>> > out I >>> > >>>>>>>> started with the GC originally and >>> > then hit >>> > >>>>>>>> some other tests I had touched via >>> the >>> > >>>>>>>> assignment extraction. >>> > >>>>>>> I think hotspot would be best. >>> > >>>>>>> >>> > >>>>>>> Chris >>> > >>>>>>>> >>> > >>>>>>>> @Serguei: Done for the method >>> > renaming, for >>> > >>>>>>>> the indent, are you talking about >>> > going from >>> > >>>>>>>> the 8-indent to 4-indent? If so, >>> would >>> > it not >>> > >>>>>>>> just be better to do a new JBS bug >>> and >>> > do the >>> > >>>>>>>> whole files in one go? I ask because >>> > >>>>>>>> otherwise, it will look a bit weird >>> to >>> > have >>> > >>>>>>>> parts of the file as 8-indent and >>> others >>> > >>>>>>>> 4-indent? >>> > >>>>>>>> >>> > >>>>>>>> Thanks for looking at it! >>> > >>>>>>>> Jc >>> > >>>>>>>> >>> > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM >>> > >>>>>>>> serguei.spitsyn at oracle.com >> serguei.spitsyn at oracle.com> >>> > >>>>>>>> >> > > >>> > >>>>>>>> >> > >>> > >>>>>>>> >> > >> wrote: >>> > >>>>>>>> >>> > >>>>>>>> Hi Jc, >>> > >>>>>>>> >>> > >>>>>>>> We have to start this review >>> > anyway. :) >>> > >>>>>>>> It looks good to me in general. >>> > >>>>>>>> Thank you for your consistency >>> in this >>> > >>>>>>>> refactoring! >>> > >>>>>>>> >>> > >>>>>>>> Some minor comments. >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>> > >>> > >>>>>>>> >>> > >>>>>>>> +static const char* >>> > remove_folders(const >>> > >>>>>>>> char* fullname) { I'd suggest to >>> > rename >>> > >>>>>>>> the function name to something >>> > traditional >>> > >>>>>>>> like get_basename. Otherwise, it >>> > sounds >>> > >>>>>>>> like this function has to really >>> > remove >>> > >>>>>>>> folders. :) Also, all >>> *Locker.cpp have >>> > >>>>>>>> wrong indent in the bodies of if >>> > and while >>> > >>>>>>>> statements. Could this be fixed >>> > with the >>> > >>>>>>>> refactoring? I did not look on >>> how >>> > this >>> > >>>>>>>> impacts the tests other than >>> > >>>>>>>> serviceability. Thanks, Serguei >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> On 11/16/18 19:43, JC Beyler >>> wrote: >>> > >>>>>>>>> Hi all, >>> > >>>>>>>>> >>> > >>>>>>>>> Anybody motivated to review >>> this? :) >>> > >>>>>>>>> Jc >>> > >>>>>>>>> >>> > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 PM >>> JC >>> > Beyler >>> > >>>>>>>>> >> > >>> > >>>>>>>>> >> > >> wrote: >>> > >>>>>>>>> >>> > >>>>>>>>> Hi all, >>> > >>>>>>>>> >>> > >>>>>>>>> Could I have a review for >>> the >>> > >>>>>>>>> extension and usage of the >>> > >>>>>>>>> ExceptionJniWrapper. This >>> > adds lines >>> > >>>>>>>>> and filenames to the end of >>> the >>> > >>>>>>>>> wrapper JNI methods, adds >>> > tracing, >>> > >>>>>>>>> and throws an error if need >>> > be. I've >>> > >>>>>>>>> ported the gc/lock files to >>> > use the >>> > >>>>>>>>> new TRACE_JNI_CALL add-on >>> and >>> > I've >>> > >>>>>>>>> ported a few of the tests >>> > that were >>> > >>>>>>>>> already changed for the >>> > assignment >>> > >>>>>>>>> webrev for JDK-8212884. >>> > >>>>>>>>> >>> > >>>>>>>>> Webrev: >>> > >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>> > >>>>>>>>> < >>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/> >>> > >>>>>>>>> Bug: >>> > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>> > >>>>>>>>> >>> > >>>>>>>>> For illustration, if I force >>> > an error >>> > >>>>>>>>> to the AP04/ap04t03 test and >>> > set the >>> > >>>>>>>>> verbosity on, I get >>> something >>> > like: >>> > >>>>>>>>> >>> > >>>>>>>>> >> Calling JNI method >>> > FindClass from >>> > >>>>>>>>> ap04t003.cpp:343 >>> > >>>>>>>>> >> Calling with these >>> > parameter(s): >>> > >>>>>>>>> java/lang/Threadd >>> > >>>>>>>>> Wait for thread to finish >>> > >>>>>>>>> << Called JNI method >>> > FindClass from >>> > >>>>>>>>> ap04t003.cpp:343 >>> > >>>>>>>>> Exception in thread >>> "Thread-0" >>> > >>>>>>>>> java.lang.NoClassDefFoundError: >>> > >>>>>>>>> java/lang/Threadd >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> > >>>>>>>>> >>> > >>>>>>>>> Method) >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> > >>> > >>>>>>>>> >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> > >>> > >>>>>>>>> >>> > >>>>>>>>> Caused by: >>> > >>>>>>>>> java.lang.ClassNotFoundException: >>> > >>>>>>>>> java.lang.Threadd >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>> > >>> > >>>>>>>>> >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>> > >>> > >>>>>>>>> >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>> > >>>>>>>>> ... 3 more >>> > >>>>>>>>> FATAL ERROR in native >>> method: JNI >>> > >>>>>>>>> method FindClass : internal >>> error >>> > >>>>>>>>> from ap04t003.cpp:343 >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> > >>>>>>>>> >>> > >>>>>>>>> Method) >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> > >>> > >>>>>>>>> >>> > >>>>>>>>> at >>> > >>>>>>>>> >>> > >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> > >>> > >>>>>>>>> >>> > >>>>>>>>> >>> > >>>>>>>>> Questions/comments I have >>> about >>> > >>>>>>>>> this are: >>> > >>>>>>>>> - Do we want to force >>> fatal >>> > errors >>> > >>>>>>>>> when a JNI call fails in >>> general? >>> > >>>>>>>>> Most of these tests do the >>> right >>> > >>>>>>>>> thing and test the return of >>> > the JNI >>> > >>>>>>>>> calls, for example: >>> > >>>>>>>>> thrClass = >>> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >>> > >>>>>>>>> TRACE_JNI_CALL); >>> > >>>>>>>>> if (thrClass == NULL) { >>> > >>>>>>>>> >>> > >>>>>>>>> but now the wrapper actually >>> > would do >>> > >>>>>>>>> a fatal if the FindClass >>> call >>> > would >>> > >>>>>>>>> return a nullptr, so we >>> could >>> > remove >>> > >>>>>>>>> that test altogether. What >>> do >>> > you >>> > >>>>>>>>> think? >>> > >>>>>>>>> - I prefer to leave them >>> > as the >>> > >>>>>>>>> tests then become closer to >>> > what real >>> > >>>>>>>>> users would have in their >>> > code and is >>> > >>>>>>>>> the "recommended" way of >>> doing it >>> > >>>>>>>>> >>> > >>>>>>>>> - The alternative is to >>> > use the >>> > >>>>>>>>> NonFatalError I added which >>> > then just >>> > >>>>>>>>> prints out that something >>> > went wrong, >>> > >>>>>>>>> letting the test continue. >>> > Question >>> > >>>>>>>>> will be what should be the >>> > default? >>> > >>>>>>>>> The fatal or the non-fatal >>> error >>> > >>>>>>>>> handling? >>> > >>>>>>>>> >>> > >>>>>>>>> On a different subject: >>> > >>>>>>>>> - On the new tests, I've >>> > removed >>> > >>>>>>>>> the NSK_JNI_VERIFY since >>> the JNI >>> > >>>>>>>>> wrapper handles the tracing >>> > and the >>> > >>>>>>>>> verify in almost the same >>> > way; only >>> > >>>>>>>>> difference I can really tell >>> > is that >>> > >>>>>>>>> the complain method from NSK >>> > has a >>> > >>>>>>>>> max complain before >>> stopping to >>> > >>>>>>>>> "complain"; I have not >>> added that >>> > >>>>>>>>> part of the code in this >>> webrev >>> > >>>>>>>>> >>> > >>>>>>>>> Once we decide on these, I >>> can >>> > >>>>>>>>> continue on the files from >>> > >>>>>>>>> JDK-8212884 and then do >>> both the >>> > >>>>>>>>> assignment in an if >>> extraction >>> > >>>>>>>>> followed-by this type of >>> > webrev in an >>> > >>>>>>>>> easier fashion. Depending on >>> > >>>>>>>>> decisions here, NSK*VERIFY >>> can be >>> > >>>>>>>>> deprecated as well as we go >>> > forward. >>> > >>>>>>>>> >>> > >>>>>>>>> Thank you for the >>> > reviews/comments :) >>> > >>>>>>>>> Jc >>> > >>>>>>>>> >>> > >>>>>>>>> >>> > >>>>>>>>> >>> > >>>>>>>>> -- >>> > >>>>>>>>> Thanks, >>> > >>>>>>>>> Jc >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> -- >>> > >>>>>>>> Thanks, >>> > >>>>>>>> Jc >>> > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> -- >>> > >>>>>>> Thanks, >>> > >>>>>>> Jc >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> -- >>> > >>>>>> Thanks, >>> > >>>>>> Jc >>> > >>>>> >>> > >>>>> >>> > >>>>> >>> > >>>>> >>> > >>>>> -- >>> > >>>>> Thanks, >>> > >>>>> Jc >>> > >>>> >>> > >>> >>> > >>> >>> > >>> >>> > >>> -- >>> > >>> Thanks, >>> > >>> Jc >>> > >>> >>> > >>> >>> > >>> >>> > >>> -- >>> > >>> >>> > >>> Thanks, >>> > >>> Jc >>> > > >>> > > >>> > >>> > >>> > >>> > -- >>> > >>> > Thanks, >>> > Jc >>> >> >> >> -- >> >> Thanks, >> Jc >> >> >> > > -- > > Thanks, > Jc > > > -- Thanks, Jc From kim.barrett at oracle.com Mon Dec 10 23:18:16 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 18:18:16 -0500 Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> References: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> Message-ID: <92A45D7A-0594-4D21-9EF3-2633CDB87FC9@oracle.com> > On Dec 10, 2018, at 2:07 AM, David Holmes wrote: > > This is a preliminary RFR to get feedback. I won't be pushing until 13 and the final RFR will have to sync with other changes that may impact what I'm doing here - e.g. Shenandoah GC. > > @Markus: does this help with your JFR event hooks? > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 > Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ Other than the bootstrapping code that should be simplified / removed in response to JDK-8215097, this looks good. I think it will address the problems I ran into with GC-related hook changes I want to make as part of JDK-8209974. From david.holmes at oracle.com Mon Dec 10 23:23:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 09:23:08 +1000 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: References: Message-ID: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> Hi Magnus, On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: > I propose that we introduce a new define, available to all JDK native > files (Hotspot included), called JDK_EXPORT. The behavior of this symbol > will be very similar (as of now, in fact identical) to JNIEXPORT; > however, the semantics will not. > > Currently, we "mis-use" the JNIEXPORT define to mark a function for > exporting from the library. The problem with this is that JNIEXPORT is > part of the JNI interface, and is supposed to be used when C programs > interact with Java. And, when doing this, the function should be fully > decorated like this: "JNIEXPORT foo JNICALL". I've seen a lot of the emails on this issue and I don't fully understand what has been going wrong. But the intent is obviously the JNIEXPORT represents what is needed to export this function for use by JNI, while JNICALL defines the calling convention. I agree there may be some mistmatch when functions are actually not intended for general export outside the JDK but are only for internal JDK use. > We do have many such JNI exports in our native libraries, but we also > have a lot of other, non-JNI exports, where one native library just > provides an interface to other libraries. In these cases, we have still > used JNIEXPORT for the functionality of getting the function exported, > but we have not been consistent in our use of JNICALL. This has caused > us way too much trouble for something that should Just Work. Are you suggesting that the interface between different libraries in the JDK should not be a JNI interface? Is this because you think the functions in these libraries are only for JDK internal use or ... ?? > I therefore propose that we define "JDK_EXPORT", with the same behavior > as JNIEXPORT (that is, flagging the function for external visibility in > the resulting native library), but which is *not* supposed to be > exported to Java code using JNI, nor supposed to be decorated with Just a clarification there. JNI functions are not exported to Java code, they are exported to native code. Java code can declare native methods and those native methods must be written as JNI functions, but that's not what we are discussing. Libraries expose a JNI interface (a set of functions in the library) that can be called by application native code, using JNI. > JNICALL. All current instances of JNIEXPORT which is not pure JNI native > functions should be changed to use JDK_EXPORT instead. > > I further propose that this macro should reside in a new file "jdk.h", > placed in the new directory src/java.base/share/native/include/internal. > This header file path will automatically be provided to all native > libraries, but not copied to the JDK being built. (The existence of a > "include/internal" directory with this behavior has been discussed > before. There are more files that ought to be moved there, if/when it is > created.) I believe in many cases the #include "jni.h" can be just > modified to #include "#jdk.h", since most native code will not require > "jni.h" unless actually doing JNI calls -- most have included this file > to get the JNIEXPORT macro, which would explain the pervasive use of > #include "jni.h" in our code base. jni.h also defines all of the types used by the JNI. Those types are pervsive to the native code used throughout the JDK. > Thoughts? I think we need to understand the problems on Windows that prompted all this. Then I think we need to look at exactly how jni.h and JNIEXPORT etc are being used and understand whether this is truly an exported interface or not. Cheers, David > /Magnus > From david.holmes at oracle.com Mon Dec 10 23:54:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 09:54:34 +1000 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <989AD83F-F734-4DD8-B160-D614ECB0DB11@oracle.com> Message-ID: <9236193d-90d3-4462-1e88-a819f9ac0c74@oracle.com> On 11/12/2018 8:59 am, Kim Barrett wrote: >> On Dec 10, 2018, at 4:38 PM, Kim Barrett wrote: >> >>> On Dec 9, 2018, at 5:05 PM, David Holmes wrote: >>> >>> Didn't realize you were going to tackle this so soon. I was just ironing out the wrinkles in 8214097 before sending it for review later today. :) >> >> It turned out to be pretty easy. And I thought there was some chance that it would help with 8214097. >> >> >>> Everything else seems fine. (I'll be reworking the BarrierSet creation assertion as part of 8214097.) >> >> Thanks. >> >> Is there actually any further change to BarrierSet installation needed for 8214097? I guess I will be finding out shortly, when your RFR comes out. > > I should have skimmed ahead in my mailbox; your RFR is already available in preliminary form. > > I?d forgotten how much bootstrapping clunkiness was needed for 8214097 without this change. > So yeah, looks like doing mine first does help yours. > > Looking at yours, I kind of want the remaining bootstrapping in mine to use your change to set_as_starting_thread, > (e.g. use your new _starting_thread to recognize it), except I think that function is called too late to use for that purpose. Within the Thread::Thread constructor: _starting_thread == NULL && Thread::current_or_null() == NULL => Creation of the main thread itself _starting_thread == NULL && Thread::current_or_null() != NULL => The main thread has created other threads before completing it's own initialization - this should be an error! _starting_thread != NULL && Thread::current() == _starting_thread => Safe initialisation by the main thread else => error Once yours is pushed I'll see how to integrate things. I have to admit I didn't think of something as simple as adding a static flag :) Cheers, David From kim.barrett at oracle.com Tue Dec 11 00:21:02 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 19:21:02 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <9236193d-90d3-4462-1e88-a819f9ac0c74@oracle.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <989AD83F-F734-4DD8-B160-D614ECB0DB11@oracle.com> <9236193d-90d3-4462-1e88-a819f9ac0c74@oracle.com> Message-ID: <06C14DF9-93BB-49FD-89F9-60D2C5444464@oracle.com> > On Dec 10, 2018, at 6:54 PM, David Holmes wrote: > > On 11/12/2018 8:59 am, Kim Barrett wrote: >> Looking at yours, I kind of want the remaining bootstrapping in mine to use your change to set_as_starting_thread, >> (e.g. use your new _starting_thread to recognize it), except I think that function is called too late to use for that purpose. > > Within the Thread::Thread constructor: > > _starting_thread == NULL && Thread::current_or_null() == NULL => Creation of the main thread itself > > _starting_thread == NULL && Thread::current_or_null() != NULL => The main thread has created other threads before completing it's own initialization - this should be an error! > > _starting_thread != NULL && Thread::current() == _starting_thread => Safe initialisation by the main thread > > else => error > > Once yours is pushed I'll see how to integrate things. I have to admit I didn't think of something as simple as adding a static flag :) I was thinking about stealing your set_as_starting_thread change and adding it to mine for use in my new bootstrapping code. But I think I will leave something like that to you, if it still seems appropriate. From kim.barrett at oracle.com Tue Dec 11 00:23:48 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Dec 2018 19:23:48 -0500 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <9F9C227D-3437-4B3C-B308-971C9CAC8846@oracle.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <1c4c5dcd-cf0e-faf7-c207-e3a630fb4c7c@redhat.com> <9F9C227D-3437-4B3C-B308-971C9CAC8846@oracle.com> Message-ID: > On Dec 10, 2018, at 4:59 PM, Kim Barrett wrote: > >> On Dec 9, 2018, at 5:37 PM, Roman Kennke wrote: >> Shenandoah also creates workers in constructors. May I request to hold >> this off a little bit to avoid making a mess out of the very soon >> upstream push? I'm open to moving this into initialize() but I cannot >> tell real quick if it has or hasn't any funny side-effects. > > Looks like Shenandoah has been pushed. I?m planning to push tomorrow, unless there are objections. That will give > 24 hours for all the world?s overnight testing to pick up the Shenandoah integration. From david.holmes at oracle.com Tue Dec 11 01:42:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 11:42:39 +1000 Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <92A45D7A-0594-4D21-9EF3-2633CDB87FC9@oracle.com> References: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> <92A45D7A-0594-4D21-9EF3-2633CDB87FC9@oracle.com> Message-ID: Thanks Kim! Once 8209974 is pushed I'll re-work this. David On 11/12/2018 9:18 am, Kim Barrett wrote: >> On Dec 10, 2018, at 2:07 AM, David Holmes wrote: >> >> This is a preliminary RFR to get feedback. I won't be pushing until 13 and the final RFR will have to sync with other changes that may impact what I'm doing here - e.g. Shenandoah GC. >> >> @Markus: does this help with your JFR event hooks? >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ > > Other than the bootstrapping code that should be simplified / removed in response to JDK-8215097, > this looks good. I think it will address the problems I ran into with GC-related hook changes I want > to make as part of JDK-8209974. > From david.holmes at oracle.com Tue Dec 11 02:12:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 12:12:50 +1000 Subject: Prelim RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: References: <9fc60771-7ea8-3138-d3f8-703a5f4e6220@oracle.com> <92A45D7A-0594-4D21-9EF3-2633CDB87FC9@oracle.com> Message-ID: On 11/12/2018 11:42 am, David Holmes wrote: > Thanks Kim! > > Once 8209974 is pushed I'll re-work this. I meant 8215097 David > David > > On 11/12/2018 9:18 am, Kim Barrett wrote: >>> On Dec 10, 2018, at 2:07 AM, David Holmes >>> wrote: >>> >>> This is a preliminary RFR to get feedback. I won't be pushing until >>> 13 and the final RFR will have to sync with other changes that may >>> impact what I'm doing here - e.g. Shenandoah GC. >>> >>> @Markus: does this help with your JFR event hooks? >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >>> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev/ >> >> Other than the bootstrapping code that should be simplified / removed >> in response to JDK-8215097, >> this looks good.? I think it will address the problems I ran into with >> GC-related hook changes I want >> to make as part of JDK-8209974. >> From per.liden at oracle.com Tue Dec 11 09:14:39 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 11 Dec 2018 10:14:39 +0100 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: <27985338-635C-4331-9010-E65CB44FA6EB@oracle.com> References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <27985338-635C-4331-9010-E65CB44FA6EB@oracle.com> Message-ID: <97660a4b-11e2-1b0e-17da-3026e4e46017@oracle.com> Hi Kim, On 12/10/18 10:47 PM, Kim Barrett wrote: [...] >> But I should also add that ZGC registers its barrier set in the constructor too, before workers are created, so I don't think Kim's patch will cause any problems really. Other collectors typically registers their barrier set in initialize(), which I guess is the root of the bootstrapping issue. > > As noted in the CR, ZGC already does the BarrierSet and thread > creation in the desired order. (Observed both by inspection and > testing.) It was pretty obvious in ZGCs case; the relevant code is > much nicer than for some of the other GCs. > Ah, sorry, I didn't read the CR before replying. I now see that you had checked that ZGC wasn't a problem. So, no objections from me. Thanks! cheers, Per From rkennke at redhat.com Tue Dec 11 09:50:41 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 11 Dec 2018 10:50:41 +0100 Subject: RFR: 8215097: Do not create NonJavaThreads before BarrierSet In-Reply-To: References: <37F359D3-888C-4B9C-8B0A-C651170BA1B8@oracle.com> <1c4c5dcd-cf0e-faf7-c207-e3a630fb4c7c@redhat.com> <9F9C227D-3437-4B3C-B308-971C9CAC8846@oracle.com> Message-ID: <59256856-b16d-9b28-f3cf-7d7bd6d5f75d@redhat.com> >>> On Dec 9, 2018, at 5:37 PM, Roman Kennke wrote: >>> Shenandoah also creates workers in constructors. May I request to hold >>> this off a little bit to avoid making a mess out of the very soon >>> upstream push? I'm open to moving this into initialize() but I cannot >>> tell real quick if it has or hasn't any funny side-effects. >> >> Looks like Shenandoah has been pushed. > > I?m planning to push tomorrow, unless there are objections. That will give > 24 hours > for all the world?s overnight testing to pick up the Shenandoah integration. I just checked Shenandoah. It doesn't seem affected by it. Roman From Yang.Zhang at arm.com Tue Dec 11 09:59:46 2018 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Tue, 11 Dec 2018 09:59:46 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: <754b4943-559f-31d7-c564-a1103ac6d50d@oracle.com> References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> <754b4943-559f-31d7-c564-a1103ac6d50d@oracle.com> Message-ID: Hi Vladimir Andrew Can this patch be merged in jdk12? Because jdk 12 ramp down phase is coming. Regards Yang -----Original Message----- From: Vladimir Kozlov Sent: Friday, November 30, 2018 2:03 AM To: Yang Zhang (Arm Technology China) ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 Thank you, Yang, for running testing. No more comments from me. Vladimir On 11/28/18 10:39 PM, Yang Zhang (Arm Technology China) wrote: > Hi Vladimir > > I run all the jtreg test(hotspot/langtools/jdk, compiler/c2/cr634086 is included). They are passed. > > Regards > Yang > > -----Original Message----- > From: Vladimir Kozlov > Sent: Thursday, November 29, 2018 2:13 AM > To: Yang Zhang (Arm Technology China) ; Andrew > Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with > MaxVectorSize=8 > > Looks good. > > What testing you did? > Unfortunately compiler/c2/cr634086 test is not part of tier1 testing and not executed by 'submit' repo testing. You need to run it. > > Thanks, > Vladimir > > On 11/28/18 12:32 AM, Yang Zhang (Arm Technology China) wrote: >> Hi Andrew, Vladimir >> >> I update the reviewers and remove the new assert in Matcher::vector_shift_count_ideal_reg(). Could you please help review again? >> >> http://cr.openjdk.java.net/~yzhang/8213134/webrev.02/ >> >> Regards >> Yang >> >> -----Original Message----- >> From: Vladimir Kozlov >> Sent: Thursday, November 22, 2018 1:56 AM >> To: Yang Zhang (Arm Technology China) ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with >> MaxVectorSize=8 >> >> You are right. I think I was confused that you are not limiting MaxVectorSize but simple use min(16, MaxVectorSize). >> >> The assert is fine. >> >> Thanks, >> Vladimir >> >> On 11/20/18 10:28 PM, Yang Zhang (Arm Technology China) wrote: >>> >>>> I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. >>>> In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). >>>> aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. >>> >>> Matcher::vector_width_in_bytes() limits vector size to min(16, MaxVectorSize). >>> Matcher::vector_shift_count_ideal_reg() assert(MaxVectorSize >= >>> size, "Length isn't supported"); When MaxVectorSize is 8, size is 8. >>> When MaxVectorSize is greater than or equal 16, size is 16. >>> So (MaxVectorSize >= size) is always true. >>> >>> Test cases of cr6340864 are passed with or without this new assert. >>> >>> Do I need to remove this new assert? >>> >>> Regards >>> Yang >>> >>> >>> On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: >>>> Hi, >>>> >>>> When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? >>>> >>>> Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 >>>> >>>> This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. >>>> Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. >>>> compiler/c2/cr6340864/TestByteVect.java >>>> compiler/c2/cr6340864/TestIntVect.java >>>> compiler/c2/cr6340864/TestShortVect.java >>>> compiler/codegen/TestCharVect2.java >>>> >>>> Regards >>>> Yang >>>> >>>> >>> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. >>> >> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. >> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > From shade at redhat.com Tue Dec 11 12:05:33 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 11 Dec 2018 13:05:33 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: Hi, Getting back to this. Replying to all three reviews in the same mail, because they share the answers! New patch: http://cr.openjdk.java.net/~shade/8181143/webrev.04/ Testing: x86_64 build, hotspot tier1, new test On 11/20/18 8:07 AM, Thomas St?fe wrote: > I don't know whether I need 10ms resolution though - if we limit the > goal to catching just hanging VMOps - which would still pretty useful > - 1s would even be enough. Right. New patch implements variable delay, depending on what actual delay had been requested. New test makes sure it works reliably. > I think we can do with just two flags, since VMOperationTimeout and > VMOperationTimeoutDelay are redundant. Just go with the delay, make -1 > the default that does nothing. Same could go for SafepointTimeout vs > SafepointTimeoutDelay. Redid like this: diagnostic(bool, AbortVMOnVMOperationTimeout, false, \ "Abort upon failure to complete VM operation promptly") \ \ diagnostic(intx, AbortVMOnVMOperationTimeoutDelay, 1000, \ "Delay in milliseconds for option AbortVMOnVMOperationTimeout") \ range(0, max_intx) \ \ > I also agree with others that it would make more sense were we to kill > the VMThread instead (e.g. just with pthread_kill(SIGILL) or similar) > - we do something like that in error handling (see ErrorLogTimeout). I looked into that, and it seems too much hassle without significant benefit. For example, it would require drilling new platform-specific holes to support it, and it also likely to be supported on Linux (pthreads) only anyway? Seems to me piggybacking on periodic tasks is much simpler. In development use, you'd have a core file to see what is going on in VM. In "production" use you'd care that circuit breaker that did its job of terminating the VM. On 11/19/18 9:35 AM, Robbin Ehn wrote: > You patch seems not to be against jdk/jdk (jdk12). Really? The new patch is definitely against jdk/jdk. > Without the actual core file, I don't see the hs file being very useful > containing the watcherthread stacktrace. It would be good if the 'killer' > signaled the VM thread and we do the error reporting from the signal handler > instead in VM thread context. Right. Looked into that too, decided it is too complicated without significant benefit. See above. On 11/19/18 7:13 AM, David Holmes wrote: > You're not just introducing one diagnostic flag, your introducing the entire VM operation > timeout mechanism, including two product flags and one diagnostic. So the CR needs to reflect > that clearly and you will need a CSR request to add the two product flags. And they will need to > be documented. This was "solved" by doing two diagnostic flags, see above. I think the symmetry against other AbortVM* flags is important, and having the boolean flag for AbortVM* is good. > It's not safe to access vm_safepoint_description() outside the VMThread as the _cur_vm_operation > could be deleted while you're trying to access its name. Noted, removed that part. > And as we don't have a general timer mechanism this has to use polling so you pay for a 10ms > task wakeup regardless of how long the timeout is. Redone that part, see above. > Given the limitations of the global timeout I'm not sure I see a use for the non-aborting form. > This could just reduce down to: > > -XX:AbortVMOpsAfter:NN > > otherwise I don't really think this carries its weight. Of course that's just my opinion. > Interested to hear others. I prefer not to introduce another shape for AbortVM* options. Two diagnostic options seems to have much less weight on the whole thing. -Aleksey From magnus.ihse.bursie at oracle.com Tue Dec 11 13:16:31 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 11 Dec 2018 14:16:31 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> Message-ID: On 2018-12-10 19:02, Matthias Klose wrote: > On 10.12.18 11:19, Magnus Ihse Bursie wrote: >> >> On 2018-12-09 11:58, Matthias Klose wrote: >>> On 06.12.18 22:03, Erik Joelsson wrote: >>>> Nothing strange in there. Is it only printed once? I wouldn't be surprised if >>>> this got printed more than one time during a normal make execution (due to >>>> reloads caused by -include). If it is, then perhaps there is something different >>>> in a later print? >>> no, it's only printed once. And it seems to be independent of the test-image >>> target. just the bootcycle-images target is enough to trigger that [1]. Also not >>> architecture specific for the hotspot targets. Builds without the bootcycle >>> target succeed [2]. >>> >>> Anything else wrong with the configury in [1]? >> It's a bit hard to say. I'm reacting to "make -C build", maybe the -C does not >> play well with bootcycle builds? I don't think that's a very well tested >> combination, and bootcycle builds is really like running the build twice in >> different directories. But I don't know, it shouldn't affect things... >> You are also running with a very detailed log level, it hardly makes it easier >> to read the log. I recommend using "info,cmdlines" instead of "debug" unless >> actively debugging a hard-to-find issue. > I'm using the separate build dir to build the server and zero jvm variants. In > the past --with-jvm-variants=server,zero was not supported, and you can't do > bootcycle builds for both variants. > > Replacing LOG=debug with LOG=info,cmdlines still lets the build fail, or was > that expected? No, that was just a friendly hint on how to produce more usable logs. > >> ... Found it! Erik's intuition was correct: >> >> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 [...snip...] gtest micro >> make-make-base make-java-compilation make-copy-files make-idea >> make-compile-commands make-make[5]: make-Leaving make-directory >> make-'/<>' failure-handler make< >> make[5]: Leaving directory '/<>' >> make/Main.gmk:1088: *** target pattern contains no '%'. Stop. >> >> So once again we're being bit by the make changedir message. And maybe that's >> triggered in a new way due to the -C? Try adding --no-print-directory to your >> top-level make invocation, as a workaround. > Yes, this lets the bootcycle build succeed. Thanks for the pointer! Good that it helped. I opened https://bugs.openjdk.java.net/browse/JDK-8215212 to track that we resolve this properly. /Magnus > > Matthias > > >> We should probably send that as a flag to make, always. >> >> /Magnus >> >>> [1] >>> https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=amd64&ver=12~23-2&stamp=1544145960&raw=0 >>> >>> [2] https://buildd.debian.org/status/package.php?p=openjdk-12 >>> >>> >>>> /Erik >>>> >>>> On 2018-12-06 11:23, Matthias Klose wrote: >>>>> On 06.12.18 20:04, Erik Joelsson wrote: >>>>>> Could you insert this before line 1087 in make/Main.gmk and post the output? >>>>>> >>>>>> $(call PrintVar, ALL_NAMED_TESTS) >>>>>> >>>>>> /Erik >>>>>> >>>>>> On 2018-12-06 10:31, Matthias Klose wrote: >>>>>>> On 06.12.18 17:41, Aleksey Shipilev wrote: >>>>>>>> On 12/6/18 4:34 PM, Matthias Klose wrote: >>>>>>>>> my bad, that happens in the test-image target: >>>>>>>>> >>>>>>>>> In the build log I see: >>>>>>>>> >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn >>>>>>>>> FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp >>>>>>>>> >>>>>>>>> >>>>>>>>> make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>>>> make/Main.gmk:1087: *** target pattern contains no '%'. Stop. >>>>>>>>> make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' >>>>>>>>> make[5]: warning: -j1 forced in submake: resetting jobserver mode. >>>>>>>> I wonder if "~" in your path screws things up. >>>>>>> that worked at least for the openjdk-11 development cycle, and replacing the >>>>>>> tilde with a dash makes no difference. >>>>> ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all >>>>> hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co >>>>> mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc >>>>> hotspot_handshake hotspot_misc hotspot_native_sanity ho >>>>> tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime >>>>> hotspot_runtime hotspot_runtime_minimalvm ho >>>>> tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime >>>>> hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru >>>>> ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt >>>>> jdk_beans jdk_client_sanity jdk_collections jd >>>>> k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core >>>>> jdk_imageio jdk_instrument jdk_instrument_sanity >>>>> jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity >>>>> jdk_lang jdk_launcher jdk_management jdk_mana >>>>> gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi >>>>> jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 >>>>> jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity >>>>> jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other >>>>> jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common >>>>> tier1_compiler >>>>> tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms >>>>> tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher >>>>> tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime >>>>> tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 >>>>> tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages >>>>> vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi >>>>> vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick >>>>> vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring >>>>> vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict >>>>> vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth >>>>> vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact >>>>> vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle >>>>> vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick >>>>> vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro >>>>> make-make-base make-java-compilation make-copy-files make-idea >>>>> make-compile-commands failure-handler make< From sgehwolf at redhat.com Tue Dec 11 13:22:01 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 11 Dec 2018 14:22:01 +0100 Subject: bootcycle builds x86_64-linux-gnu? In-Reply-To: References: <802b6fab-30db-7db7-5a48-d78ee1c6a7aa@ubuntu.com> <4c491e45-c5e3-32a3-93ee-7d7041fe1200@redhat.com> <8f878053-b023-bc59-7eb8-825996065d0e@ubuntu.com> <709222a5-f687-d768-5b3e-2d99833c692c@oracle.com> Message-ID: On Mon, 2018-12-10 at 19:02 +0100, Matthias Klose wrote: > On 10.12.18 11:19, Magnus Ihse Bursie wrote: > > > > > > On 2018-12-09 11:58, Matthias Klose wrote: > > > On 06.12.18 22:03, Erik Joelsson wrote: > > > > Nothing strange in there. Is it only printed once? I wouldn't be surprised if > > > > this got printed more than one time during a normal make execution (due to > > > > reloads caused by -include). If it is, then perhaps there is something different > > > > in a later print? > > > > > > no, it's only printed once. And it seems to be independent of the test-image > > > target. just the bootcycle-images target is enough to trigger that [1]. Also not > > > architecture specific for the hotspot targets. Builds without the bootcycle > > > target succeed [2]. > > > > > > Anything else wrong with the configury in [1]? > > > > It's a bit hard to say. I'm reacting to "make -C build", maybe the -C does not > > play well with bootcycle builds? I don't think that's a very well tested > > combination, and bootcycle builds is really like running the build twice in > > different directories. But I don't know, it shouldn't affect things... > > You are also running with a very detailed log level, it hardly makes it easier > > to read the log. I recommend using "info,cmdlines" instead of "debug" unless > > actively debugging a hard-to-find issue. > > I'm using the separate build dir to build the server and zero jvm variants. In > the past --with-jvm-variants=server,zero was not supported, and you can't do > bootcycle builds for both variants. > > Replacing LOG=debug with LOG=info,cmdlines still lets the build fail, or was > that expected? > > > ... Found it! Erik's intuition was correct: > > > > ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 [...snip...] gtest micro > > make-make-base make-java-compilation make-copy-files make-idea > > make-compile-commands make-make[5]: make-Leaving make-directory > > make-'/<>' failure-handler make< > > make[5]: Leaving directory '/<>' > > make/Main.gmk:1088: *** target pattern contains no '%'. Stop. Reproduced. I've filed https://bugs.openjdk.java.net/browse/JDK-8215213 Thanks, Severin > > So once again we're being bit by the make changedir message. And maybe that's > > triggered in a new way due to the -C? Try adding --no-print-directory to your > > top-level make invocation, as a workaround. > > Yes, this lets the bootcycle build succeed. Thanks for the pointer! > > Matthias > > > > > > We should probably send that as a flag to make, always. > > > > /Magnus > > > > > > > > [1] > > > https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=amd64&ver=12~23-2&stamp=1544145960&raw=0 > > > > > > [2] https://buildd.debian.org/status/package.php?p=openjdk-12 > > > > > > > > > > /Erik > > > > > > > > On 2018-12-06 11:23, Matthias Klose wrote: > > > > > On 06.12.18 20:04, Erik Joelsson wrote: > > > > > > Could you insert this before line 1087 in make/Main.gmk and post the output? > > > > > > > > > > > > $(call PrintVar, ALL_NAMED_TESTS) > > > > > > > > > > > > /Erik > > > > > > > > > > > > On 2018-12-06 10:31, Matthias Klose wrote: > > > > > > > On 06.12.18 17:41, Aleksey Shipilev wrote: > > > > > > > > On 12/6/18 4:34 PM, Matthias Klose wrote: > > > > > > > > > my bad, that happens in the test-image target: > > > > > > > > > > > > > > > > > > In the build log I see: > > > > > > > > > > > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jdk > > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/langtools > > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/nashorn > > > > > > > > > FindJtregGroups /home/packages/openjdk/12/openjdk-12-12~23/test/jaxp > > > > > > > > > > > > > > > > > > > > > > > > > > > make[5]: Leaving directory '/home/packages/openjdk/12/openjdk-12-12~23' > > > > > > > > > make/Main.gmk:1087: *** target pattern contains no '%'. Stop. > > > > > > > > > make[5]: Entering directory '/home/packages/openjdk/12/openjdk-12-12~23' > > > > > > > > > make[5]: warning: -j1 forced in submake: resetting jobserver mode. > > > > > > > > > > > > > > > > I wonder if "~" in your path screws things up. > > > > > > > > > > > > > > that worked at least for the openjdk-11 development cycle, and replacing the > > > > > > > tilde with a dash makes no difference. > > > > > > > > > > ALL_NAMED_TESTS >build core_tools ctw_1 ctw_2 ctw_3 hotspot_all > > > > > hotspot_all_no_apps hotspot_appcds hotspot_cds hotspot_co > > > > > mpiler hotspot_compiler_all_gcs hotspot_compiler_xcomp hotspot_gc > > > > > hotspot_handshake hotspot_misc hotspot_native_sanity ho > > > > > tspot_nmt hotspot_not_fast_compiler hotspot_not_fast_gc hotspot_rest_runtime > > > > > hotspot_runtime hotspot_runtime_minimalvm ho > > > > > tspot_serviceability hotspot_slow_compiler hotspot_tier2_runtime > > > > > hotspot_tier2_runtime_platform_agnostic hotspot_tier3_ru > > > > > ntime jaxp_all jcstress_part1 jcstress_part2 jcstress_part3 jdk_2d jdk_awt > > > > > jdk_beans jdk_client_sanity jdk_collections jd > > > > > k_collections_core jdk_concurrent jdk_core jdk_desktop jdk_desktop_core > > > > > jdk_imageio jdk_instrument jdk_instrument_sanity > > > > > jdk_io jdk_jdi jdk_jdi_sanity jdk_jfr jdk_jfr_sanity jdk_jmx jdk_jmx_sanity > > > > > jdk_lang jdk_launcher jdk_management jdk_mana > > > > > gement_sanity jdk_math jdk_native_sanity jdk_net jdk_nio jdk_other jdk_rmi > > > > > jdk_sctp jdk_security jdk_security1 jdk_security2 jdk_security3 jdk_security4 > > > > > jdk_security_infra jdk_sound jdk_stable jdk_stream jdk_svc jdk_svc_sanity > > > > > jdk_swing jdk_swing_core jdk_text jdk_time jdk_tools jdk_util jdk_util_other > > > > > jfc_demo needs_g1gc svc_tools svc_tools_sanity tier1 tier1_common > > > > > tier1_compiler > > > > > tier1_compiler_1 tier1_compiler_2 tier1_compiler_3 tier1_compiler_not_cms > > > > > tier1_compiler_not_xcomp tier1_gc tier1_gc_1 tier1_gc_2 tier1_gc_gcbasher > > > > > tier1_gc_gcold tier1_part1 tier1_part2 tier1_part3 tier1_runtime > > > > > tier1_runtime_appcds tier1_runtime_appcds_exclude tier1_serviceability tier2 > > > > > tier2_part1 tier2_part2 tier2_part3 tier3 vmTestbase_largepages > > > > > vmTestbase_nsk_aod vmTestbase_nsk_jdb vmTestbase_nsk_jdi > > > > > vmTestbase_nsk_jdi_quick vmTestbase_nsk_jdwp vmTestbase_nsk_jdwp_quick > > > > > vmTestbase_nsk_jvmti vmTestbase_nsk_jvmti_quick vmTestbase_nsk_monitoring > > > > > vmTestbase_nsk_monitoring_quick vmTestbase_nsk_stress vmTestbase_nsk_sysdict > > > > > vmTestbase_vm_compiler vmTestbase_vm_compiler_quick vmTestbase_vm_defmeth > > > > > vmTestbase_vm_g1classunloading vmTestbase_vm_gc vmTestbase_vm_gc_compact > > > > > vmTestbase_vm_gc_concurrent vmTestbase_vm_gc_container vmTestbase_vm_gc_juggle > > > > > vmTestbase_vm_gc_locker vmTestbase_vm_gc_misc vmTestbase_vm_gc_quick > > > > > vmTestbase_vm_gc_ref vmTestbase_vm_metaspace vmTestbase_vm_mlvm gtest micro > > > > > make-make-base make-java-compilation make-copy-files make-idea > > > > > make-compile-commands failure-handler make< > > From magnus.ihse.bursie at oracle.com Tue Dec 11 14:34:51 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 11 Dec 2018 15:34:51 +0100 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> Message-ID: <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> On 2018-12-11 00:23, David Holmes wrote: > Hi Magnus, > > On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >> I propose that we introduce a new define, available to all JDK native >> files (Hotspot included), called JDK_EXPORT. The behavior of this >> symbol will be very similar (as of now, in fact identical) to >> JNIEXPORT; however, the semantics will not. >> >> Currently, we "mis-use" the JNIEXPORT define to mark a function for >> exporting from the library. The problem with this is that JNIEXPORT >> is part of the JNI interface, and is supposed to be used when C >> programs interact with Java. And, when doing this, the function >> should be fully decorated like this: "JNIEXPORT foo JNICALL". > > I've seen a lot of the emails on this issue and I don't fully > understand what has been going wrong. But the intent is obviously the > JNIEXPORT represents what is needed to export this function for use by > JNI, while JNICALL defines the calling convention. I agree there may > be some mistmatch when functions are actually not intended for general > export outside the JDK but are only for internal JDK use. > >> We do have many such JNI exports in our native libraries, but we also >> have a lot of other, non-JNI exports, where one native library just >> provides an interface to other libraries. In these cases, we have >> still used JNIEXPORT for the functionality of getting the function >> exported, but we have not been consistent in our use of JNICALL. This >> has caused us way too much trouble for something that should Just >> Work. > > Are you suggesting that the interface between different libraries in > the JDK should not be a JNI interface? Is this because you think the > functions in these libraries are only for JDK internal use or ... ?? > >> I therefore propose that we define "JDK_EXPORT", with the same >> behavior as JNIEXPORT (that is, flagging the function for external >> visibility in the resulting native library), but which is *not* >> supposed to be exported to Java code using JNI, nor supposed to be >> decorated with > > Just a clarification there. JNI functions are not exported to Java > code, they are exported to native code. Java code can declare native > methods and those native methods must be written as JNI functions, but > that's not what we are discussing. Libraries expose a JNI interface (a > set of functions in the library) that can be called by application > native code, using JNI. We're apparently looking at "JNI" and "exporting" from two opposite sides here. :-) Just to make everything clear: If I have a Java class class MyClass { public static void native myNativeFunc(); } then I have one half of the JNI function, the Java half. This must be matched by a corresponding implementation in native, like this: JNIEXPORT void JNICALL Java_MyClass_myNativeFunc(void) { // ... do stuff } And this is the native half of the JNI function. Right? Let's leave aside which side is "exporting" to the other for now. :-) This way of setting up native functions that can be called from Java is what I refer to as JNI. And when you declare a native JNI function, you *must* use both JNIEXPORT and JNICALL. Alright? We do have a lot of those functions in our native libraries. And they are correct just the way they are. However, we also have *other* native functions, that are exported, not as JNI functions that should be called from Java, but as normal native library functions that should be called by other native code. Okay so far? And *those* functions have been problematic in how we decorate them. My proposal is that we *refrain* from using JNIEXPORT for those functions, and instead use JDK_EXPORT as name for the macro that decorates them as exported. That way, we can clearly see that a function like this: JDK_EXPORT void JLI_ReadEnv(char* env); is correctly declared, and will be exported to other native libraries, but not to Java. Just to clarify, this has nothing to do with if this is a officially supported API or not. In general though, I assume that most (if not all?) of our exported functions (apart from the JNI_* stuff) is supposed to be consumed by other libraries in the JDK, and is not a public API. /Magnus > >> JNICALL. All current instances of JNIEXPORT which is not pure JNI >> native functions should be changed to use JDK_EXPORT instead. >> >> I further propose that this macro should reside in a new file >> "jdk.h", placed in the new directory >> src/java.base/share/native/include/internal. This header file path >> will automatically be provided to all native libraries, but not >> copied to the JDK being built. (The existence of a "include/internal" >> directory with this behavior has been discussed before. There are >> more files that ought to be moved there, if/when it is created.) I >> believe in many cases the #include "jni.h" can be just modified to >> #include "#jdk.h", since most native code will not require "jni.h" >> unless actually doing JNI calls -- most have included this file to >> get the JNIEXPORT macro, which would explain the pervasive use of >> #include "jni.h" in our code base. > > jni.h also defines all of the types used by the JNI. Those types are > pervsive to the native code used throughout the JDK. > >> Thoughts? > > I think we need to understand the problems on Windows that prompted > all this. Then I think we need to look at exactly how jni.h and > JNIEXPORT etc are being used and understand whether this is truly an > exported interface or not. > > Cheers, > David > >> /Magnus >> From rkennke at redhat.com Tue Dec 11 15:18:40 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 11 Dec 2018 16:18:40 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: <5c21c712-4292-e704-f162-cd0ff8b5228a@redhat.com> Hi Aleksey, this looks good to me. Thanks! Roman > Hi, > > Getting back to this. Replying to all three reviews in the same mail, because they share the > answers! New patch: > http://cr.openjdk.java.net/~shade/8181143/webrev.04/ > > Testing: x86_64 build, hotspot tier1, new test > > On 11/20/18 8:07 AM, Thomas St?fe wrote: >> I don't know whether I need 10ms resolution though - if we limit the >> goal to catching just hanging VMOps - which would still pretty useful >> - 1s would even be enough. > > Right. New patch implements variable delay, depending on what actual delay had been requested. New > test makes sure it works reliably. > >> I think we can do with just two flags, since VMOperationTimeout and >> VMOperationTimeoutDelay are redundant. Just go with the delay, make -1 >> the default that does nothing. Same could go for SafepointTimeout vs >> SafepointTimeoutDelay. > > Redid like this: > > diagnostic(bool, AbortVMOnVMOperationTimeout, false, \ > "Abort upon failure to complete VM operation promptly") \ > \ > diagnostic(intx, AbortVMOnVMOperationTimeoutDelay, 1000, \ > "Delay in milliseconds for option AbortVMOnVMOperationTimeout") \ > range(0, max_intx) \ > \ > >> I also agree with others that it would make more sense were we to kill >> the VMThread instead (e.g. just with pthread_kill(SIGILL) or similar) >> - we do something like that in error handling (see ErrorLogTimeout). > > I looked into that, and it seems too much hassle without significant benefit. For example, it would > require drilling new platform-specific holes to support it, and it also likely to be supported on > Linux (pthreads) only anyway? Seems to me piggybacking on periodic tasks is much simpler. In > development use, you'd have a core file to see what is going on in VM. In "production" use you'd > care that circuit breaker that did its job of terminating the VM. > > On 11/19/18 9:35 AM, Robbin Ehn wrote: >> You patch seems not to be against jdk/jdk (jdk12). > > Really? The new patch is definitely against jdk/jdk. > >> Without the actual core file, I don't see the hs file being very useful >> containing the watcherthread stacktrace. It would be good if the 'killer' >> signaled the VM thread and we do the error reporting from the signal handler >> instead in VM thread context. > > Right. Looked into that too, decided it is too complicated without significant benefit. See above. > > On 11/19/18 7:13 AM, David Holmes wrote: >> You're not just introducing one diagnostic flag, your introducing the entire VM operation >> timeout mechanism, including two product flags and one diagnostic. So the CR needs to reflect >> that clearly and you will need a CSR request to add the two product flags. And they will need to >> be documented. > > This was "solved" by doing two diagnostic flags, see above. I think the symmetry against other > AbortVM* flags is important, and having the boolean flag for AbortVM* is good. > >> It's not safe to access vm_safepoint_description() outside the VMThread as the _cur_vm_operation >> could be deleted while you're trying to access its name. > > Noted, removed that part. > >> And as we don't have a general timer mechanism this has to use polling so you pay for a 10ms >> task wakeup regardless of how long the timeout is. > > Redone that part, see above. > >> Given the limitations of the global timeout I'm not sure I see a use for the non-aborting form. >> This could just reduce down to: >> >> -XX:AbortVMOpsAfter:NN >> >> otherwise I don't really think this carries its weight. Of course that's just my opinion. >> Interested to hear others. > > I prefer not to introduce another shape for AbortVM* options. Two diagnostic options seems to have > much less weight on the whole thing. > > -Aleksey > From volker.simonis at gmail.com Tue Dec 11 15:22:11 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Dec 2018 16:22:11 +0100 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> Message-ID: On Tue, Dec 11, 2018 at 3:35 PM Magnus Ihse Bursie wrote: > > > > On 2018-12-11 00:23, David Holmes wrote: > > Hi Magnus, > > > > On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: > >> I propose that we introduce a new define, available to all JDK native > >> files (Hotspot included), called JDK_EXPORT. The behavior of this > >> symbol will be very similar (as of now, in fact identical) to > >> JNIEXPORT; however, the semantics will not. > >> > >> Currently, we "mis-use" the JNIEXPORT define to mark a function for > >> exporting from the library. The problem with this is that JNIEXPORT > >> is part of the JNI interface, and is supposed to be used when C > >> programs interact with Java. And, when doing this, the function > >> should be fully decorated like this: "JNIEXPORT foo JNICALL". > > > > I've seen a lot of the emails on this issue and I don't fully > > understand what has been going wrong. But the intent is obviously the > > JNIEXPORT represents what is needed to export this function for use by > > JNI, while JNICALL defines the calling convention. I agree there may > > be some mistmatch when functions are actually not intended for general > > export outside the JDK but are only for internal JDK use. > > > >> We do have many such JNI exports in our native libraries, but we also > >> have a lot of other, non-JNI exports, where one native library just > >> provides an interface to other libraries. In these cases, we have > >> still used JNIEXPORT for the functionality of getting the function > >> exported, but we have not been consistent in our use of JNICALL. This > >> has caused us way too much trouble for something that should Just > >> Work. > > > > Are you suggesting that the interface between different libraries in > > the JDK should not be a JNI interface? Is this because you think the > > functions in these libraries are only for JDK internal use or ... ?? > > > >> I therefore propose that we define "JDK_EXPORT", with the same > >> behavior as JNIEXPORT (that is, flagging the function for external > >> visibility in the resulting native library), but which is *not* > >> supposed to be exported to Java code using JNI, nor supposed to be > >> decorated with > > > > Just a clarification there. JNI functions are not exported to Java > > code, they are exported to native code. Java code can declare native > > methods and those native methods must be written as JNI functions, but > > that's not what we are discussing. Libraries expose a JNI interface (a > > set of functions in the library) that can be called by application > > native code, using JNI. > We're apparently looking at "JNI" and "exporting" from two opposite > sides here. :-) Just to make everything clear: If I have a Java class > class MyClass { > public static void native myNativeFunc(); > } > > then I have one half of the JNI function, the Java half. This must be > matched by a corresponding implementation in native, like this: > JNIEXPORT void JNICALL > Java_MyClass_myNativeFunc(void) { > // ... do stuff > } > > And this is the native half of the JNI function. Right? Let's leave > aside which side is "exporting" to the other for now. :-) > > This way of setting up native functions that can be called from Java is > what I refer to as JNI. And when you declare a native JNI function, you > *must* use both JNIEXPORT and JNICALL. Alright? > > We do have a lot of those functions in our native libraries. And they > are correct just the way they are. > > However, we also have *other* native functions, that are exported, not > as JNI functions that should be called from Java, but as normal native > library functions that should be called by other native code. Okay so > far? And *those* functions have been problematic in how we decorate > them. My proposal is that we *refrain* from using JNIEXPORT for those > functions, and instead use JDK_EXPORT as name for the macro that > decorates them as exported. That way, we can clearly see that a function > like this: > > JDK_EXPORT void > JLI_ReadEnv(char* env); > > is correctly declared, and will be exported to other native libraries, > but not to Java. > Hi Magnus, I agree with your explanation and I think your proposal is sound. Have you checked how many of the occurrences of "#include "jni.h"" can really be replaced by "#include "jdk.h""? I think native code also "misuses" jni.h in order to agree on common types like jint, jlong, etc.. across different native libraries. We could of course also define such common types in "jdk.h", but I'm not sure if it's worth the effort? Regards, Volker > Just to clarify, this has nothing to do with if this is a officially > supported API or not. In general though, I assume that most (if not > all?) of our exported functions (apart from the JNI_* stuff) is supposed > to be consumed by other libraries in the JDK, and is not a public API. > > /Magnus > > > > > > >> JNICALL. All current instances of JNIEXPORT which is not pure JNI > >> native functions should be changed to use JDK_EXPORT instead. > >> > >> I further propose that this macro should reside in a new file > >> "jdk.h", placed in the new directory > >> src/java.base/share/native/include/internal. This header file path > >> will automatically be provided to all native libraries, but not > >> copied to the JDK being built. (The existence of a "include/internal" > >> directory with this behavior has been discussed before. There are > >> more files that ought to be moved there, if/when it is created.) I > >> believe in many cases the #include "jni.h" can be just modified to > >> #include "#jdk.h", since most native code will not require "jni.h" > >> unless actually doing JNI calls -- most have included this file to > >> get the JNIEXPORT macro, which would explain the pervasive use of > >> #include "jni.h" in our code base. > > > > jni.h also defines all of the types used by the JNI. Those types are > > pervsive to the native code used throughout the JDK. > > > >> Thoughts? > > > > I think we need to understand the problems on Windows that prompted > > all this. Then I think we need to look at exactly how jni.h and > > JNIEXPORT etc are being used and understand whether this is truly an > > exported interface or not. > > > > Cheers, > > David > > > >> /Magnus > >> > From zgu at redhat.com Tue Dec 11 17:23:55 2018 From: zgu at redhat.com (zgu at redhat.com) Date: Tue, 11 Dec 2018 12:23:55 -0500 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: <1544549035.4592.90.camel@redhat.com> Looks good to me. -Zhengyu On Tue, 2018-12-11 at 13:05 +0100, Aleksey Shipilev wrote: > Hi, > > Getting back to this. Replying to all three reviews in the same mail, > because they share the > answers! New patch: > http://cr.openjdk.java.net/~shade/8181143/webrev.04/ > > Testing: x86_64 build, hotspot tier1, new test > > On 11/20/18 8:07 AM, Thomas St?fe wrote: > > I don't know whether I need 10ms resolution though - if we limit > > the > > goal to catching just hanging VMOps - which would still pretty > > useful > > - 1s would even be enough. > > Right. New patch implements variable delay, depending on what actual > delay had been requested. New > test makes sure it works reliably. > > > I think we can do with just two flags, since VMOperationTimeout and > > VMOperationTimeoutDelay are redundant. Just go with the delay, make > > -1 > > the default that does nothing. Same could go for SafepointTimeout > > vs > > SafepointTimeoutDelay. > > Redid like this: > > diagnostic(bool, AbortVMOnVMOperationTimeout, > false, \ > "Abort upon failure to complete VM operation > promptly") \ > > \ > diagnostic(intx, AbortVMOnVMOperationTimeoutDelay, > 1000, \ > "Delay in milliseconds for option > AbortVMOnVMOperationTimeout") \ > range(0, > max_intx) \ > > \ > > > I also agree with others that it would make more sense were we to > > kill > > the VMThread instead (e.g. just with pthread_kill(SIGILL) or > > similar) > > - we do something like that in error handling (see > > ErrorLogTimeout). > > I looked into that, and it seems too much hassle without significant > benefit. For example, it would > require drilling new platform-specific holes to support it, and it > also likely to be supported on > Linux (pthreads) only anyway? Seems to me piggybacking on periodic > tasks is much simpler. In > development use, you'd have a core file to see what is going on in > VM. In "production" use you'd > care that circuit breaker that did its job of terminating the VM. > > On 11/19/18 9:35 AM, Robbin Ehn wrote: > > You patch seems not to be against jdk/jdk (jdk12). > > Really? The new patch is definitely against jdk/jdk. > > > Without the actual core file, I don't see the hs file being very > > useful > > containing the watcherthread stacktrace. It would be good if the > > 'killer' > > signaled the VM thread and we do the error reporting from the > > signal handler > > instead in VM thread context. > > Right. Looked into that too, decided it is too complicated without > significant benefit. See above. > > On 11/19/18 7:13 AM, David Holmes wrote: > > You're not just introducing one diagnostic flag, your introducing > > the entire VM operation > > timeout mechanism, including two product flags and one diagnostic. > > So the CR needs to reflect > > that clearly and you will need a CSR request to add the two product > > flags. And they will need to > > be documented. > > This was "solved" by doing two diagnostic flags, see above. I think > the symmetry against other > AbortVM* flags is important, and having the boolean flag for AbortVM* > is good. > > > It's not safe to access vm_safepoint_description() outside the > > VMThread as the _cur_vm_operation > > could be deleted while you're trying to access its name. > > Noted, removed that part. > > > And as we don't have a general timer mechanism this has to use > > polling so you pay for a 10ms > > task wakeup regardless of how long the timeout is. > > Redone that part, see above. > > > Given the limitations of the global timeout I'm not sure I see a > > use for the non-aborting form. > > This could just reduce down to: > > > > -XX:AbortVMOpsAfter:NN > > > > otherwise I don't really think this carries its weight. Of course > > that's just my opinion. > > Interested to hear others. > > I prefer not to introduce another shape for AbortVM* options. Two > diagnostic options seems to have > much less weight on the whole thing. > > -Aleksey > From dean.long at oracle.com Tue Dec 11 22:05:22 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 11 Dec 2018 14:05:22 -0800 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: Message-ID: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> Hi Nick.? This change looks good to me. dl On 12/2/18 10:06 PM, Nick Gasson wrote: > Hi, > > Could someone please help me review this fix for a test failure on ARM32: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 > Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ > > This fixes two assertion failures related to biased locking when C2's > bias inlining feature is disabled (-XX:-OptoBiasInlining): > > MacroAssembler::atomic_cas_bool takes three register arguments plus a > temporary register for use in the CAS loop. If the caller passes > `noreg' as this temporary register (which happens in the > !OptoBiasInlining call path from MacroAssembler::fast_lock) then > MacroAssembler::atomic_cas_bool will default to using LR as a > temporary. But it's possible with C2 that LR is one of the other three > arguments which then triggers an assert_different_registers assertion > failure. Fix this by supplying an additional scratch register to > MacroAssembler::fast_lock if !OptoBiasInlining. > > In the !OptoBiasInlining case MacroAssembler::fast_lock calls > MacroAssembler::biased_locking_enter to handle the case where the mark > word contains the biased lock pattern which fast_lock wouldn't > otherwise see if OptoBiasInlining was true. However in the case that > we fail to acquire the biased lock this falls through to label > `failed' and runs the normal fast_lock code that implicitly assumes > the mark word does not have the bias pattern. This means we can end up > with a BasicLock where _displaced_header contains the biased lock > pattern which is an illegal state and subsequently triggers an > assertion failure in ObjectSynchronizer::fast_exit. > > The right thing to do here is branch to `done' whenever the bias lock > pattern is present and let the runtime handle the failure case. Also > edited the comment describing MacroAssembler::biased_locking_enter to > more accurately describe what it does (the comment about `slow_case' > being optional is wrong). > > Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this > patch. > > Thanks, > Nick From david.holmes at oracle.com Tue Dec 11 22:47:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Dec 2018 08:47:31 +1000 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> Message-ID: <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: > > > On 2018-12-11 00:23, David Holmes wrote: >> Hi Magnus, >> >> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>> I propose that we introduce a new define, available to all JDK native >>> files (Hotspot included), called JDK_EXPORT. The behavior of this >>> symbol will be very similar (as of now, in fact identical) to >>> JNIEXPORT; however, the semantics will not. >>> >>> Currently, we "mis-use" the JNIEXPORT define to mark a function for >>> exporting from the library. The problem with this is that JNIEXPORT >>> is part of the JNI interface, and is supposed to be used when C >>> programs interact with Java. And, when doing this, the function >>> should be fully decorated like this: "JNIEXPORT foo JNICALL". >> >> I've seen a lot of the emails on this issue and I don't fully >> understand what has been going wrong. But the intent is obviously the >> JNIEXPORT represents what is needed to export this function for use by >> JNI, while JNICALL defines the calling convention. I agree there may >> be some mistmatch when functions are actually not intended for general >> export outside the JDK but are only for internal JDK use. >> >>> We do have many such JNI exports in our native libraries, but we also >>> have a lot of other, non-JNI exports, where one native library just >>> provides an interface to other libraries. In these cases, we have >>> still used JNIEXPORT for the functionality of getting the function >>> exported, but we have not been consistent in our use of JNICALL. This >>> has caused us way too much trouble for something that should Just >>> Work. >> >> Are you suggesting that the interface between different libraries in >> the JDK should not be a JNI interface? Is this because you think the >> functions in these libraries are only for JDK internal use or ... ?? >> >>> I therefore propose that we define "JDK_EXPORT", with the same >>> behavior as JNIEXPORT (that is, flagging the function for external >>> visibility in the resulting native library), but which is *not* >>> supposed to be exported to Java code using JNI, nor supposed to be >>> decorated with >> >> Just a clarification there. JNI functions are not exported to Java >> code, they are exported to native code. Java code can declare native >> methods and those native methods must be written as JNI functions, but >> that's not what we are discussing. Libraries expose a JNI interface (a >> set of functions in the library) that can be called by application >> native code, using JNI. > We're apparently looking at "JNI" and "exporting" from two opposite > sides here. :-) Just to make everything clear: If I have a Java class > class MyClass { > ? public static void native myNativeFunc(); > } > > then I have one half of the JNI function, the Java half. This must be > matched by a corresponding implementation in native, like this: > JNIEXPORT void JNICALL > Java_MyClass_myNativeFunc(void) { > // ... do stuff > } > > And this is the native half of the JNI function. Right? Let's leave > aside which side is "exporting" to the other for now. :-) > > This way of setting up native functions that can be called from Java is > what I refer to as JNI. And when you declare a native JNI function, you > *must* use both JNIEXPORT and JNICALL. Alright? > > We do have a lot of those functions in our native libraries. And they > are correct just the way they are. Yep all well and good. A function declared native in Java must have an implementation as you describe. But not all native functions exist to provide the native-half of a Java native function! > However, we also have *other* native functions, that are exported, not > as JNI functions that should be called from Java, but as normal native > library functions that should be called by other native code. Okay so > far? And *those* functions have been problematic in how we decorate But there are again two cases. Those functions exported from a library that are expected to be called from external code using the JNI interface mechanism - such as all the JNI functions and JVM TI functions we export from the JVM - and those "exported" for access between libraries within the JDK (such as all the JVM_* functions in libjvm). I think it is only the second group that should be addressed by your JDK_EXPORT proposal - though I'm not completely clear exactly how to identify them. > them. My proposal is that we *refrain* from using JNIEXPORT for those > functions, and instead use JDK_EXPORT as name for the macro that > decorates them as exported. That way, we can clearly see that a function > like this: > > JDK_EXPORT void > JLI_ReadEnv(char* env); > > is correctly declared, and will be exported to other native libraries, > but not to Java. The issue is not whether it is "exported to Java"** but whether it is exported using the JNI mechanism such that other native code calls it using the JNI mechanism. ** There is no way to write a native method declaration in Java such that it would be linked to the JLI_ReadEnv function. The naming is all wrong, as is the signature. > Just to clarify, this has nothing to do with if this is a officially > supported API or not. In general though, I assume that most (if not > all?) of our exported functions (apart from the JNI_* stuff) is supposed > to be consumed by other libraries in the JDK, and is not a public API. I think it varies library by library. You may need native application code that can call directly into native JDK libraries. JLI is the Java Launcher Interface - I think it was introduced to make it easier for other launchers to be created. Native agents may need access to libmanagement or libjdwp functions. Native graphics code may need access to the JDK graphics library. Some of these accesses may be unsupported and undocumented, but I don't think you can just cut them all off. David > > /Magnus > > > >> >>> JNICALL. All current instances of JNIEXPORT which is not pure JNI >>> native functions should be changed to use JDK_EXPORT instead. >>> >>> I further propose that this macro should reside in a new file >>> "jdk.h", placed in the new directory >>> src/java.base/share/native/include/internal. This header file path >>> will automatically be provided to all native libraries, but not >>> copied to the JDK being built. (The existence of a "include/internal" >>> directory with this behavior has been discussed before. There are >>> more files that ought to be moved there, if/when it is created.) I >>> believe in many cases the #include "jni.h" can be just modified to >>> #include "#jdk.h", since most native code will not require "jni.h" >>> unless actually doing JNI calls -- most have included this file to >>> get the JNIEXPORT macro, which would explain the pervasive use of >>> #include "jni.h" in our code base. >> >> jni.h also defines all of the types used by the JNI. Those types are >> pervsive to the native code used throughout the JDK. >> >>> Thoughts? >> >> I think we need to understand the problems on Windows that prompted >> all this. Then I think we need to look at exactly how jni.h and >> JNIEXPORT etc are being used and understand whether this is truly an >> exported interface or not. >> >> Cheers, >> David >> >>> /Magnus >>> > From Nick.Gasson at arm.com Wed Dec 12 01:15:22 2018 From: Nick.Gasson at arm.com (Nick Gasson (Arm Technology China)) Date: Wed, 12 Dec 2018 01:15:22 +0000 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> Message-ID: > Hi Nick. This change looks good to me. Thanks Dean. Could you help me to commit this patch if it doesn't need other reviews? Nick > -----Original Message----- > From: dean.long at oracle.com > Sent: 12 December 2018 06:05 > To: Nick Gasson (Arm Technology China) ; hotspot- > dev at openjdk.java.net; Boris Ulasevich > Cc: nd ; hotspot-compiler-dev at openjdk.java.net compiler > ; aarch32-port- > dev at openjdk.java.net > Subject: Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on > ARM32 > > Hi Nick.? This change looks good to me. > > dl > > On 12/2/18 10:06 PM, Nick Gasson wrote: > > Hi, > > > > Could someone please help me review this fix for a test failure on ARM32: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 > > Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ > > > > This fixes two assertion failures related to biased locking when C2's > > bias inlining feature is disabled (-XX:-OptoBiasInlining): > > > > MacroAssembler::atomic_cas_bool takes three register arguments plus a > > temporary register for use in the CAS loop. If the caller passes > > `noreg' as this temporary register (which happens in the > > !OptoBiasInlining call path from MacroAssembler::fast_lock) then > > MacroAssembler::atomic_cas_bool will default to using LR as a > > temporary. But it's possible with C2 that LR is one of the other three > > arguments which then triggers an assert_different_registers assertion > > failure. Fix this by supplying an additional scratch register to > > MacroAssembler::fast_lock if !OptoBiasInlining. > > > > In the !OptoBiasInlining case MacroAssembler::fast_lock calls > > MacroAssembler::biased_locking_enter to handle the case where the mark > > word contains the biased lock pattern which fast_lock wouldn't > > otherwise see if OptoBiasInlining was true. However in the case that > > we fail to acquire the biased lock this falls through to label > > `failed' and runs the normal fast_lock code that implicitly assumes > > the mark word does not have the bias pattern. This means we can end up > > with a BasicLock where _displaced_header contains the biased lock > > pattern which is an illegal state and subsequently triggers an > > assertion failure in ObjectSynchronizer::fast_exit. > > > > The right thing to do here is branch to `done' whenever the bias lock > > pattern is present and let the runtime handle the failure case. Also > > edited the comment describing MacroAssembler::biased_locking_enter to > > more accurately describe what it does (the comment about `slow_case' > > being optional is wrong). > > > > Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this > > patch. > > > > Thanks, > > Nick From dean.long at oracle.com Wed Dec 12 01:30:29 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 11 Dec 2018 17:30:29 -0800 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> Message-ID: I can help you commit it, but first I think another review would be good. dl On 12/11/18 5:15 PM, Nick Gasson (Arm Technology China) wrote: >> Hi Nick. This change looks good to me. > Thanks Dean. Could you help me to commit this patch if it doesn't need other reviews? > > Nick > > >> -----Original Message----- >> From: dean.long at oracle.com >> Sent: 12 December 2018 06:05 >> To: Nick Gasson (Arm Technology China) ; hotspot- >> dev at openjdk.java.net; Boris Ulasevich >> Cc: nd ; hotspot-compiler-dev at openjdk.java.net compiler >> ; aarch32-port- >> dev at openjdk.java.net >> Subject: Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on >> ARM32 >> >> Hi Nick.? This change looks good to me. >> >> dl >> >> On 12/2/18 10:06 PM, Nick Gasson wrote: >>> Hi, >>> >>> Could someone please help me review this fix for a test failure on ARM32: >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 >>> Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ >>> >>> This fixes two assertion failures related to biased locking when C2's >>> bias inlining feature is disabled (-XX:-OptoBiasInlining): >>> >>> MacroAssembler::atomic_cas_bool takes three register arguments plus a >>> temporary register for use in the CAS loop. If the caller passes >>> `noreg' as this temporary register (which happens in the >>> !OptoBiasInlining call path from MacroAssembler::fast_lock) then >>> MacroAssembler::atomic_cas_bool will default to using LR as a >>> temporary. But it's possible with C2 that LR is one of the other three >>> arguments which then triggers an assert_different_registers assertion >>> failure. Fix this by supplying an additional scratch register to >>> MacroAssembler::fast_lock if !OptoBiasInlining. >>> >>> In the !OptoBiasInlining case MacroAssembler::fast_lock calls >>> MacroAssembler::biased_locking_enter to handle the case where the mark >>> word contains the biased lock pattern which fast_lock wouldn't >>> otherwise see if OptoBiasInlining was true. However in the case that >>> we fail to acquire the biased lock this falls through to label >>> `failed' and runs the normal fast_lock code that implicitly assumes >>> the mark word does not have the bias pattern. This means we can end up >>> with a BasicLock where _displaced_header contains the biased lock >>> pattern which is an illegal state and subsequently triggers an >>> assertion failure in ObjectSynchronizer::fast_exit. >>> >>> The right thing to do here is branch to `done' whenever the bias lock >>> pattern is present and let the runtime handle the failure case. Also >>> edited the comment describing MacroAssembler::biased_locking_enter to >>> more accurately describe what it does (the comment about `slow_case' >>> being optional is wrong). >>> >>> Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this >>> patch. >>> >>> Thanks, >>> Nick From jcbeyler at google.com Wed Dec 12 05:16:59 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 11 Dec 2018 21:16:59 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> Message-ID: Hi all, Here is the new webrev with the TEST.groups change. Serguei, let me know if I convinced you with the static vs anonymous namespaces or if you'd still rather have a "static" for now :-) Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 Thanks again for the reviews! Jc On Mon, Dec 10, 2018 at 3:10 PM JC Beyler wrote: > Hi Serguei, > > Yes basically it is equivalent :) I can put them in but they are not > required. The norm actually wanted to deprecate it but then remembered that > C compatibility would require the static key-word for this case [1] > > So, really, they are not required here and will amount to the same thing: > only that file can refer to them and you cannot get to them without a > globally available method to return a pointer to them (ie same as a static > variable in C). > > I can put static if it makes it easier to see but, by being in an > anonymous namespace they are only available for the file's translation > unit. For example: > > $ cat main.cpp > > int totally_global; > static int explictly_static; > > namespace { > int implicitly_static; > } > > void foo(); > int main() { > foo(); > } > > $ g++ -O3 main.cpp -c > $ nm main.o > U _GLOBAL_OFFSET_TABLE_ > 0000000000000000 T main > 0000000000000000 B totally_global > U _Z3foov > > As you can see, the static and anonymous namespace variables are not in > the file due to not being used. If you were to use them, you'd see them > show up as something like: > 0000000000000008 b _ZL17explicitly_static > 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE > > Where again, it shows that it is mangling the names so that no external > usage can happen without tinkering. > > Hopefully that helps :-), > Jc > > [1] http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 > > > On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com < > serguei.spitsyn at oracle.com> wrote: > >> Hi Jc, >> >> I had little experience with the C++ namespaces. >> My understanding is that static in this context should mean internal >> linkage. >> >> Thanks, >> Serguei >> >> >> On 12/10/18 13:57, JC Beyler wrote: >> >> Hi Serguei, >> >> The variables and functions are in a anonymous namespace; my >> understanding of C++ is that this is equivalent to putting it as >> static.Hence, I didn't add them there. Does that make sense? >> >> Thanks! >> Jc >> >> On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com < >> serguei.spitsyn at oracle.com> wrote: >> >>> Hi Jc, >>> >>> It looks good in general. >>> One question though. >>> >>> >>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>> >>> I wonder if the variables and functions have to be static. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 12/5/18 11:36, JC Beyler wrote: >>> >>> Hi all, >>> >>> My apologies to having to come back for another review for this change: >>> I ran into a snag when trying to pull the latest changes compared to the >>> base I was working on. I basically forgot that there was an issue with >>> snprintf and that I had solved it via JDK-8213622. >>> >>> Could I have a new review of this webrev: >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> Incremental from the port of webrev.03 that got LGTMs: >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ >>> >>> A few comments on this because it took me a while to get things in a >>> state I thought was good: >>> - I had to implement an itoa method, do we have something like that in >>> the test base (remember that JDK-8213622 could not use sprintf due to being >>> in the test code)? >>> >>> - The differences here compared to the one you all reviewed are: >>> - I found that adding to the strlen/memcpy error prone and thought >>> that I would try to make it less so. If you want to compare, I extended the >>> strlen/memcpy with the new format to show you if you prefer [1] >>> - Note that the diff between the "old extended way from [1]" >>> to the webrev.04 can be found in [2] >>> >>> - I added a test to test the exception wrapper in tests :); I'm not >>> sure it is deemed useful or not but helped me assure myself that I was not >>> doing things wrong; you can find the base test file here [3]; should we >>> have this or not? (I know that normally we don't add tests to vmTestbase >>> but thought this might be an exception) >>> >>> Thanks for your help and my apologies for the snag, >>> Jc >>> >>> [1]: >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>> [2]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 >>> [3] >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>> >>> On Mon, Dec 3, 2018 at 11:29 PM David Holmes >>> wrote: >>> >>>> Looks fine to me. >>>> >>>> Thanks, >>>> David >>>> >>>> On 4/12/2018 4:04 pm, JC Beyler wrote: >>>> > Hi both, >>>> > >>>> > Thanks for the reviews! Since Serguei did not insist on get_basename, >>>> I >>>> > went for get_dirname since the method is a local static method and >>>> won't >>>> > have its name start spreading, I think it's ok too. >>>> > >>>> > For the naming of the local variable, the idea initially was to use >>>> the >>>> > same name as the local variable for JNIEnv already used to reduce the >>>> > code change. Since I'm now adding the line macro at the end anyway, >>>> this >>>> > does not matter anymore so I converged all local variables to "jni". >>>> > >>>> > So, without further ado, here is the new version: >>>> > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >>>> > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> > >>>> > This passes the various tests changed by the webrev on my dev machine. >>>> > >>>> > Let me know what you think, >>>> > Jc >>>> > >>>> > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com >>>> > >>> > > wrote: >>>> > >>>> > On 12/3/18 20:15, Chris Plummer wrote: >>>> > > Hi JC, >>>> > > >>>> > > Overall it looks good. A few naming nits thought: >>>> > > >>>> > > In bi01t001.cpp, why have you declared the >>>> > ExceptionCheckingJniEnvPtr >>>> > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename >>>> the >>>> > > method argument passed in from jni to jni_env. >>>> > > >>>> > > Related to this, I also noticed in some files that already are >>>> using >>>> > > ExceptionCheckingJniEnvPtr, such as >>>> CharArrayCriticalLocker.cpp, you >>>> > > delcared it as env(jni_env). So that means there are 3 >>>> different >>>> > names >>>> > > you have used for the ExceptionCheckingJniEnvPtr local >>>> variable. >>>> > They >>>> > > should be consistent. >>>> > > >>>> > > Also, can you rename get_basename() to get_dirname()? I know >>>> Serguei >>>> > > suggested get_basename() a while back, but unless "basename" is >>>> > > commonly used for this purpose, I think "dirname" is more self >>>> > > explanatory. >>>> > >>>> > In general, I'm Okay with get_dirname(). >>>> > Just to mention dirname can be both short or full, so it is a >>>> little >>>> > confusing as well. >>>> > It is the reason why the get_basename() was suggested. >>>> > However, I do not insist on get_basename() nor >>>> get_full_dirname(). :) >>>> > >>>> > Thanks, >>>> > Serguei >>>> > >>>> > >>>> > > thanks, >>>> > > >>>> > > Chris >>>> > > >>>> > > On 12/2/18 10:29 PM, David Holmes wrote: >>>> > >> Hi Jc, >>>> > >> >>>> > >> I've been lurking on this one and have had a look through. >>>> I'm okay >>>> > >> with the FatalError approach for the tests - we don't expect >>>> > anything >>>> > >> to go wrong in a well written test in a correctly functioning >>>> VM. >>>> > >> >>>> > >> Thanks, >>>> > >> David >>>> > >> >>>> > >> >>>> > >> >>>> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: >>>> > >>> Hi all, >>>> > >>> >>>> > >>> Would someone on the GC or runtime team be motivated to give >>>> > this a >>>> > >>> review? :) >>>> > >>> >>>> > >>> It would be much appreciated! >>>> > >>> >>>> > >>> Webrev: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> > >>> >>>> > >>> Thanks for your help, >>>> > >>> Jc >>>> > >>> >>>> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler < >>>> jcbeyler at google.com >>>> > >>>> > >>> >> >>>> wrote: >>>> > >>> >>>> > >>> Hi Chris, >>>> > >>> >>>> > >>> Yes I was waiting for another review since you had >>>> explicitly >>>> > >>> asked :) >>>> > >>> >>>> > >>> And sounds good that when someone from GC or runtime >>>> gives a >>>> > >>> review, >>>> > >>> I'll wait for your full review on the webrev.02! >>>> > >>> >>>> > >>> Thanks again for your help, >>>> > >>> Jc >>>> > >>> >>>> > >>> >>>> > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >>>> > >>> >>> chris.plummer at oracle.com> >>>> > >>> >>> >>>> > wrote: >>>> > >>> >>>> > >>> Hi JC, >>>> > >>> >>>> > >>> I think it would be good to get a review from the gc >>>> or >>>> > runtime >>>> > >>> teams, since this also affects their tests. >>>> > >>> >>>> > >>> Also, once we are settled on this FatalError >>>> approach, >>>> > I still >>>> > >>> need to give your webrev-02 a full review. I only >>>> > skimmed over >>>> > >>> parts of it (I did look at all the changes in >>>> webrevo-01). >>>> > >>> >>>> > >>> thanks, >>>> > >>> >>>> > >>> Chris >>>> > >>> >>>> > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >>>> > >>>> > >>> >>> > > wrote: >>>> > >>>> Hi Jc, >>>> > >>>> >>>> > >>>> I've already reviewed this too. >>>> > >>>> >>>> > >>>> Thanks, >>>> > >>>> Serguei >>>> > >>>> >>>> > >>>> >>>> > >>>> On 11/27/18 06:56, JC Beyler wrote: >>>> > >>>>> Thanks Chris, >>>> > >>>>> >>>> > >>>>> Anybody else motivated to look at this and review >>>> it? :) >>>> > >>>>> Jc >>>> > >>>>> >>>> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >>>> > >>>>> >>> > >>> chris.plummer at oracle.com >>>> > >> >>>> > >>>>> wrote: >>>> > >>>>> >>>> > >>>>> Hi JC, >>>> > >>>>> >>>> > >>>>> I'm ok with the FatalError approach, but would >>>> > like to >>>> > >>>>> hear opinions from others also. >>>> > >>>>> >>>> > >>>>> thanks, >>>> > >>>>> >>>> > >>>>> Chris >>>> > >>>>> >>>> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >>>> > >>>>>> Hi Chris, >>>> > >>>>>> >>>> > >>>>>> Thanks for taking the time to look at it and >>>> yes you >>>> > >>>>>> have raised exactly why the webrev is between >>>> two >>>> > >>>>>> worlds: in cases where a fatal error on >>>> failure is >>>> > >>>>>> wanted, should we simplify the code to remove >>>> > the return >>>> > >>>>>> tests since we do them internally? Now that >>>> I've >>>> > looked >>>> > >>>>>> around for non-fatal cases, I think the answer >>>> > is yes, >>>> > >>>>>> it simplifies the code while maintaining the >>>> checks. >>>> > >>>>>> >>>> > >>>>>> I looked a bit and it seems that I can't find >>>> > easily a >>>> > >>>>>> case where the test accepts a JNI failure to >>>> > then move >>>> > >>>>>> on. Therefore, perhaps, for now, the fail >>>> with a >>>> > Fatal >>>> > >>>>>> is enough and we can work on the tests to >>>> clean >>>> > them up? >>>> > >>>>>> >>>> > >>>>>> That means that this is the new webrev with >>>> only >>>> > Fatal >>>> > >>>>>> and cleans up the tests so that it is no >>>> longer in >>>> > >>>>>> between two worlds: >>>> > >>>>>> >>>> > >>>>>> Webrev: >>>> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>> > >>>>>> < >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/> >>>> > >>>>>> Bug: >>>> > https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> > >>>>>> >>>> > >>>>>> (This passes testing on my dev machine for >>>> all the >>>> > >>>>>> modified tests) >>>> > >>>>>> >>>> > >>>>>> with the example you provided, it now looks >>>> like: >>>> > >>>>>> >>>> > >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> > >>>> > >>>>>> >>>> > >>>>>> >>>> > < >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> > >>>> > >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>>>> Where it does, to me at least, seem cleaner >>>> and less >>>> > >>>>>> "noisy". >>>> > >>>>>> >>>> > >>>>>> Let me know what you think, >>>> > >>>>>> Jc >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >>>> > >>>>>> >>> > >>>> > >>>>>> >>> > >> wrote: >>>> > >>>>>> >>>> > >>>>>> Hi JC, >>>> > >>>>>> >>>> > >>>>>> Sorry about the delay. I had to go back an >>>> > look at >>>> > >>>>>> the initial 8210842 webrev and RFR thread >>>> to see >>>> > >>>>>> what this was initially all about. >>>> > >>>>>> >>>> > >>>>>> In general the changes look good. >>>> > >>>>>> >>>> > >>>>>> I don't have a good answer to your >>>> > >>>>>> FatalError/NonFatalError question. It >>>> makes >>>> > the code >>>> > >>>>>> a lot cleaner to use FatalError, but then >>>> it >>>> > is a >>>> > >>>>>> behavior change, and you also need to >>>> deal with >>>> > >>>>>> tests that intentionally induce errors (do >>>> > you have >>>> > >>>>>> an example of that). >>>> > >>>>>> >>>> > >>>>>> In any case, right now your webrev seems >>>> to be >>>> > >>>>>> between two worlds. You are producing >>>> > FatalError, >>>> > >>>>>> but still checking results. Here's a good >>>> > example: >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> > >>>> > >>>>>> >>>> > >>>>>> >>>> > < >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> > >>>> > >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>>>> I'm not sure if this is just a temporary >>>> > state until >>>> > >>>>>> it was decided which approach to take. >>>> > >>>>>> >>>> > >>>>>> thanks, >>>> > >>>>>> >>>> > >>>>>> Chris >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: >>>> > >>>>>>> Hi all, >>>> > >>>>>>> >>>> > >>>>>>> Chris thought it made sense to have more >>>> > eyes on >>>> > >>>>>>> this change than just serviceability as >>>> it will >>>> > >>>>>>> modify to tests that are not only >>>> > serviceability >>>> > >>>>>>> tests so I've moved this to conversation >>>> > here :) >>>> > >>>>>>> >>>> > >>>>>>> For convenience, I've copy-pasted the >>>> > initial RFR: >>>> > >>>>>>> >>>> > >>>>>>> Could I have a review for the extension >>>> and >>>> > usage >>>> > >>>>>>> of the ExceptionJniWrapper. This adds >>>> lines and >>>> > >>>>>>> filenames to the end of the wrapper JNI >>>> > methods, >>>> > >>>>>>> adds tracing, and throws an error if need >>>> > be. I've >>>> > >>>>>>> ported the gc/lock files to use the new >>>> > >>>>>>> TRACE_JNI_CALL add-on and I've ported a >>>> few >>>> > of the >>>> > >>>>>>> tests that were already changed for the >>>> > assignment >>>> > >>>>>>> webrev for JDK-8212884. >>>> > >>>>>>> >>>> > >>>>>>> Webrev: >>>> > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>> > >>>>>>> < >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01> >>>> > >>>>>>> Bug: >>>> > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> > >>>>>>> >>>> > >>>>>>> For illustration, if I force an error to >>>> the >>>> > >>>>>>> AP04/ap04t03 test and set the verbosity >>>> on, >>>> > I get >>>> > >>>>>>> something like: >>>> > >>>>>>> >>>> > >>>>>>> >> Calling JNI method FindClass from >>>> > >>>>>>> ap04t003.cpp:343 >>>> > >>>>>>> >> Calling with these parameter(s): >>>> > >>>>>>> java/lang/Threadd >>>> > >>>>>>> Wait for thread to finish >>>> > >>>>>>> << Called JNI method FindClass from >>>> > >>>>>>> ap04t003.cpp:343 >>>> > >>>>>>> Exception in thread "Thread-0" >>>> > >>>>>>> java.lang.NoClassDefFoundError: >>>> > java/lang/Threadd >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> > >>>>>>> >>>> > >>>>>>> Method) >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> > >>>>>>> >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> > >>>>>>> >>>> > >>>>>>> Caused by: >>>> java.lang.ClassNotFoundException: >>>> > >>>>>>> java.lang.Threadd >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>> > >>>> > >>>>>>> >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>> > >>>> > >>>>>>> >>>> > >>>>>>> at >>>> > >>>>>>> >>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>> > >>>>>>> ... 3 more >>>> > >>>>>>> FATAL ERROR in native method: JNI method >>>> > FindClass >>>> > >>>>>>> : internal error from ap04t003.cpp:343 >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> > >>>>>>> >>>> > >>>>>>> Method) >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> > >>>>>>> >>>> > >>>>>>> at >>>> > >>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> > >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> Questions/comments I have about this are: >>>> > >>>>>>> - Do we want to force fatal errors >>>> when a JNI >>>> > >>>>>>> call fails in general? Most of these >>>> tests >>>> > do the >>>> > >>>>>>> right thing and test the return of the >>>> JNI >>>> > calls, >>>> > >>>>>>> for example: >>>> > >>>>>>> thrClass = >>>> > jni->FindClass("java/lang/Threadd", >>>> > >>>>>>> TRACE_JNI_CALL); >>>> > >>>>>>> if (thrClass == NULL) { >>>> > >>>>>>> >>>> > >>>>>>> but now the wrapper actually would do a >>>> > fatal if >>>> > >>>>>>> the FindClass call would return a >>>> nullptr, >>>> > so we >>>> > >>>>>>> could remove that test altogether. What >>>> do you >>>> > >>>>>>> think? >>>> > >>>>>>> - I prefer to leave them as the >>>> tests then >>>> > >>>>>>> become closer to what real users would >>>> have in >>>> > >>>>>>> their code and is the "recommended" way >>>> of >>>> > doing it >>>> > >>>>>>> >>>> > >>>>>>> - The alternative is to use the >>>> > NonFatalError I >>>> > >>>>>>> added which then just prints out that >>>> something >>>> > >>>>>>> went wrong, letting the test continue. >>>> Question >>>> > >>>>>>> will be what should be the default? The >>>> > fatal or >>>> > >>>>>>> the non-fatal error handling? >>>> > >>>>>>> >>>> > >>>>>>> On a different subject: >>>> > >>>>>>> - On the new tests, I've removed the >>>> > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper >>>> > handles the >>>> > >>>>>>> tracing and the verify in almost the same >>>> > way; only >>>> > >>>>>>> difference I can really tell is that the >>>> > complain >>>> > >>>>>>> method from NSK has a max complain before >>>> > stopping >>>> > >>>>>>> to "complain"; I have not added that part >>>> > of the >>>> > >>>>>>> code in this webrev >>>> > >>>>>>> >>>> > >>>>>>> Once we decide on these, I can continue >>>> on the >>>> > >>>>>>> files from JDK-8212884 and then do both >>>> the >>>> > >>>>>>> assignment in an if extraction >>>> followed-by this >>>> > >>>>>>> type of webrev in an easier fashion. >>>> > Depending on >>>> > >>>>>>> decisions here, NSK*VERIFY can be >>>> deprecated as >>>> > >>>>>>> well as we go forward. >>>> > >>>>>>> >>>> > >>>>>>> Thanks! >>>> > >>>>>>> Jc >>>> > >>>>>>> >>>> > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris >>>> Plummer >>>> > >>>>>>> >>> > >>>> > >>>>>>> >>> > >> wrote: >>>> > >>>>>>> >>>> > >>>>>>> On 11/19/18 10:07 AM, JC Beyler >>>> wrote: >>>> > >>>>>>>> Hi all, >>>> > >>>>>>>> >>>> > >>>>>>>> @David/Chris: should I then push >>>> this >>>> > RFR to >>>> > >>>>>>>> the hotspot mailing or the runtime >>>> > one? For >>>> > >>>>>>>> what it's worth, a lot of the tests >>>> > under the >>>> > >>>>>>>> vmTestbase are jvmti so the review >>>> also >>>> > >>>>>>>> affects serviceability; it just >>>> turns >>>> > out I >>>> > >>>>>>>> started with the GC originally and >>>> > then hit >>>> > >>>>>>>> some other tests I had touched via >>>> the >>>> > >>>>>>>> assignment extraction. >>>> > >>>>>>> I think hotspot would be best. >>>> > >>>>>>> >>>> > >>>>>>> Chris >>>> > >>>>>>>> >>>> > >>>>>>>> @Serguei: Done for the method >>>> > renaming, for >>>> > >>>>>>>> the indent, are you talking about >>>> > going from >>>> > >>>>>>>> the 8-indent to 4-indent? If so, >>>> would >>>> > it not >>>> > >>>>>>>> just be better to do a new JBS bug >>>> and >>>> > do the >>>> > >>>>>>>> whole files in one go? I ask because >>>> > >>>>>>>> otherwise, it will look a bit weird >>>> to >>>> > have >>>> > >>>>>>>> parts of the file as 8-indent and >>>> others >>>> > >>>>>>>> 4-indent? >>>> > >>>>>>>> >>>> > >>>>>>>> Thanks for looking at it! >>>> > >>>>>>>> Jc >>>> > >>>>>>>> >>>> > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM >>>> > >>>>>>>> serguei.spitsyn at oracle.com >>> serguei.spitsyn at oracle.com> >>>> > >>>>>>>> >>> > > >>>> > >>>>>>>> >>> > >>>> > >>>>>>>> >>> > >> wrote: >>>> > >>>>>>>> >>>> > >>>>>>>> Hi Jc, >>>> > >>>>>>>> >>>> > >>>>>>>> We have to start this review >>>> > anyway. :) >>>> > >>>>>>>> It looks good to me in general. >>>> > >>>>>>>> Thank you for your consistency >>>> in this >>>> > >>>>>>>> refactoring! >>>> > >>>>>>>> >>>> > >>>>>>>> Some minor comments. >>>> > >>>>>>>> >>>> > >>>>>>>> >>>> > >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>> > >>>> > >>>>>>>> >>>> > >>>>>>>> +static const char* >>>> > remove_folders(const >>>> > >>>>>>>> char* fullname) { I'd suggest to >>>> > rename >>>> > >>>>>>>> the function name to something >>>> > traditional >>>> > >>>>>>>> like get_basename. Otherwise, it >>>> > sounds >>>> > >>>>>>>> like this function has to really >>>> > remove >>>> > >>>>>>>> folders. :) Also, all >>>> *Locker.cpp have >>>> > >>>>>>>> wrong indent in the bodies of if >>>> > and while >>>> > >>>>>>>> statements. Could this be fixed >>>> > with the >>>> > >>>>>>>> refactoring? I did not look on >>>> how >>>> > this >>>> > >>>>>>>> impacts the tests other than >>>> > >>>>>>>> serviceability. Thanks, Serguei >>>> > >>>>>>>> >>>> > >>>>>>>> >>>> > >>>>>>>> On 11/16/18 19:43, JC Beyler >>>> wrote: >>>> > >>>>>>>>> Hi all, >>>> > >>>>>>>>> >>>> > >>>>>>>>> Anybody motivated to review >>>> this? :) >>>> > >>>>>>>>> Jc >>>> > >>>>>>>>> >>>> > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 PM >>>> JC >>>> > Beyler >>>> > >>>>>>>>> >>> > >>>> > >>>>>>>>> >>> > >> wrote: >>>> > >>>>>>>>> >>>> > >>>>>>>>> Hi all, >>>> > >>>>>>>>> >>>> > >>>>>>>>> Could I have a review for >>>> the >>>> > >>>>>>>>> extension and usage of the >>>> > >>>>>>>>> ExceptionJniWrapper. This >>>> > adds lines >>>> > >>>>>>>>> and filenames to the end >>>> of the >>>> > >>>>>>>>> wrapper JNI methods, adds >>>> > tracing, >>>> > >>>>>>>>> and throws an error if need >>>> > be. I've >>>> > >>>>>>>>> ported the gc/lock files to >>>> > use the >>>> > >>>>>>>>> new TRACE_JNI_CALL add-on >>>> and >>>> > I've >>>> > >>>>>>>>> ported a few of the tests >>>> > that were >>>> > >>>>>>>>> already changed for the >>>> > assignment >>>> > >>>>>>>>> webrev for JDK-8212884. >>>> > >>>>>>>>> >>>> > >>>>>>>>> Webrev: >>>> > >>>>>>>>> >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>> > >>>>>>>>> < >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/> >>>> > >>>>>>>>> Bug: >>>> > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> > >>>>>>>>> >>>> > >>>>>>>>> For illustration, if I >>>> force >>>> > an error >>>> > >>>>>>>>> to the AP04/ap04t03 test >>>> and >>>> > set the >>>> > >>>>>>>>> verbosity on, I get >>>> something >>>> > like: >>>> > >>>>>>>>> >>>> > >>>>>>>>> >> Calling JNI method >>>> > FindClass from >>>> > >>>>>>>>> ap04t003.cpp:343 >>>> > >>>>>>>>> >> Calling with these >>>> > parameter(s): >>>> > >>>>>>>>> java/lang/Threadd >>>> > >>>>>>>>> Wait for thread to finish >>>> > >>>>>>>>> << Called JNI method >>>> > FindClass from >>>> > >>>>>>>>> ap04t003.cpp:343 >>>> > >>>>>>>>> Exception in thread >>>> "Thread-0" >>>> > >>>>>>>>> java.lang.NoClassDefFoundError: >>>> > >>>>>>>>> java/lang/Threadd >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> > >>>>>>>>> >>>> > >>>>>>>>> Method) >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> Caused by: >>>> > >>>>>>>>> java.lang.ClassNotFoundException: >>>> > >>>>>>>>> java.lang.Threadd >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>> > >>>>>>>>> ... 3 more >>>> > >>>>>>>>> FATAL ERROR in native >>>> method: JNI >>>> > >>>>>>>>> method FindClass : >>>> internal error >>>> > >>>>>>>>> from ap04t003.cpp:343 >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> > >>>>>>>>> >>>> > >>>>>>>>> Method) >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> at >>>> > >>>>>>>>> >>>> > >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> >>>> > >>>>>>>>> Questions/comments I have >>>> about >>>> > >>>>>>>>> this are: >>>> > >>>>>>>>> - Do we want to force >>>> fatal >>>> > errors >>>> > >>>>>>>>> when a JNI call fails in >>>> general? >>>> > >>>>>>>>> Most of these tests do the >>>> right >>>> > >>>>>>>>> thing and test the return >>>> of >>>> > the JNI >>>> > >>>>>>>>> calls, for example: >>>> > >>>>>>>>> thrClass = >>>> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >>>> > >>>>>>>>> TRACE_JNI_CALL); >>>> > >>>>>>>>> if (thrClass == NULL) { >>>> > >>>>>>>>> >>>> > >>>>>>>>> but now the wrapper >>>> actually >>>> > would do >>>> > >>>>>>>>> a fatal if the FindClass >>>> call >>>> > would >>>> > >>>>>>>>> return a nullptr, so we >>>> could >>>> > remove >>>> > >>>>>>>>> that test altogether. What >>>> do >>>> > you >>>> > >>>>>>>>> think? >>>> > >>>>>>>>> - I prefer to leave >>>> them >>>> > as the >>>> > >>>>>>>>> tests then become closer to >>>> > what real >>>> > >>>>>>>>> users would have in their >>>> > code and is >>>> > >>>>>>>>> the "recommended" way of >>>> doing it >>>> > >>>>>>>>> >>>> > >>>>>>>>> - The alternative is to >>>> > use the >>>> > >>>>>>>>> NonFatalError I added which >>>> > then just >>>> > >>>>>>>>> prints out that something >>>> > went wrong, >>>> > >>>>>>>>> letting the test continue. >>>> > Question >>>> > >>>>>>>>> will be what should be the >>>> > default? >>>> > >>>>>>>>> The fatal or the non-fatal >>>> error >>>> > >>>>>>>>> handling? >>>> > >>>>>>>>> >>>> > >>>>>>>>> On a different subject: >>>> > >>>>>>>>> - On the new tests, I've >>>> > removed >>>> > >>>>>>>>> the NSK_JNI_VERIFY since >>>> the JNI >>>> > >>>>>>>>> wrapper handles the tracing >>>> > and the >>>> > >>>>>>>>> verify in almost the same >>>> > way; only >>>> > >>>>>>>>> difference I can really >>>> tell >>>> > is that >>>> > >>>>>>>>> the complain method from >>>> NSK >>>> > has a >>>> > >>>>>>>>> max complain before >>>> stopping to >>>> > >>>>>>>>> "complain"; I have not >>>> added that >>>> > >>>>>>>>> part of the code in this >>>> webrev >>>> > >>>>>>>>> >>>> > >>>>>>>>> Once we decide on these, I >>>> can >>>> > >>>>>>>>> continue on the files from >>>> > >>>>>>>>> JDK-8212884 and then do >>>> both the >>>> > >>>>>>>>> assignment in an if >>>> extraction >>>> > >>>>>>>>> followed-by this type of >>>> > webrev in an >>>> > >>>>>>>>> easier fashion. Depending >>>> on >>>> > >>>>>>>>> decisions here, NSK*VERIFY >>>> can be >>>> > >>>>>>>>> deprecated as well as we go >>>> > forward. >>>> > >>>>>>>>> >>>> > >>>>>>>>> Thank you for the >>>> > reviews/comments :) >>>> > >>>>>>>>> Jc >>>> > >>>>>>>>> >>>> > >>>>>>>>> >>>> > >>>>>>>>> >>>> > >>>>>>>>> -- >>>> > >>>>>>>>> Thanks, >>>> > >>>>>>>>> Jc >>>> > >>>>>>>> >>>> > >>>>>>>> >>>> > >>>>>>>> >>>> > >>>>>>>> -- >>>> > >>>>>>>> Thanks, >>>> > >>>>>>>> Jc >>>> > >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> -- >>>> > >>>>>>> Thanks, >>>> > >>>>>>> Jc >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>>>> >>>> > >>>>>> -- >>>> > >>>>>> Thanks, >>>> > >>>>>> Jc >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> >>>> > >>>>> -- >>>> > >>>>> Thanks, >>>> > >>>>> Jc >>>> > >>>> >>>> > >>> >>>> > >>> >>>> > >>> >>>> > >>> -- >>>> > >>> Thanks, >>>> > >>> Jc >>>> > >>> >>>> > >>> >>>> > >>> >>>> > >>> -- >>>> > >>> >>>> > >>> Thanks, >>>> > >>> Jc >>>> > > >>>> > > >>>> > >>>> > >>>> > >>>> > -- >>>> > >>>> > Thanks, >>>> > Jc >>>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >>> >>> >>> >> >> -- >> >> Thanks, >> Jc >> >> >> > > -- > > Thanks, > Jc > -- Thanks, Jc From volker.simonis at gmail.com Wed Dec 12 07:44:56 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 12 Dec 2018 08:44:56 +0100 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> Message-ID: On Tue, Dec 11, 2018 at 11:47 PM David Holmes wrote: > > On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: > > > > > > On 2018-12-11 00:23, David Holmes wrote: > >> Hi Magnus, > >> > >> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: > >>> I propose that we introduce a new define, available to all JDK native > >>> files (Hotspot included), called JDK_EXPORT. The behavior of this > >>> symbol will be very similar (as of now, in fact identical) to > >>> JNIEXPORT; however, the semantics will not. > >>> > >>> Currently, we "mis-use" the JNIEXPORT define to mark a function for > >>> exporting from the library. The problem with this is that JNIEXPORT > >>> is part of the JNI interface, and is supposed to be used when C > >>> programs interact with Java. And, when doing this, the function > >>> should be fully decorated like this: "JNIEXPORT foo JNICALL". > >> > >> I've seen a lot of the emails on this issue and I don't fully > >> understand what has been going wrong. But the intent is obviously the > >> JNIEXPORT represents what is needed to export this function for use by > >> JNI, while JNICALL defines the calling convention. I agree there may > >> be some mistmatch when functions are actually not intended for general > >> export outside the JDK but are only for internal JDK use. > >> > >>> We do have many such JNI exports in our native libraries, but we also > >>> have a lot of other, non-JNI exports, where one native library just > >>> provides an interface to other libraries. In these cases, we have > >>> still used JNIEXPORT for the functionality of getting the function > >>> exported, but we have not been consistent in our use of JNICALL. This > >>> has caused us way too much trouble for something that should Just > >>> Work. > >> > >> Are you suggesting that the interface between different libraries in > >> the JDK should not be a JNI interface? Is this because you think the > >> functions in these libraries are only for JDK internal use or ... ?? > >> > >>> I therefore propose that we define "JDK_EXPORT", with the same > >>> behavior as JNIEXPORT (that is, flagging the function for external > >>> visibility in the resulting native library), but which is *not* > >>> supposed to be exported to Java code using JNI, nor supposed to be > >>> decorated with > >> > >> Just a clarification there. JNI functions are not exported to Java > >> code, they are exported to native code. Java code can declare native > >> methods and those native methods must be written as JNI functions, but > >> that's not what we are discussing. Libraries expose a JNI interface (a > >> set of functions in the library) that can be called by application > >> native code, using JNI. > > We're apparently looking at "JNI" and "exporting" from two opposite > > sides here. :-) Just to make everything clear: If I have a Java class > > class MyClass { > > public static void native myNativeFunc(); > > } > > > > then I have one half of the JNI function, the Java half. This must be > > matched by a corresponding implementation in native, like this: > > JNIEXPORT void JNICALL > > Java_MyClass_myNativeFunc(void) { > > // ... do stuff > > } > > > > And this is the native half of the JNI function. Right? Let's leave > > aside which side is "exporting" to the other for now. :-) > > > > This way of setting up native functions that can be called from Java is > > what I refer to as JNI. And when you declare a native JNI function, you > > *must* use both JNIEXPORT and JNICALL. Alright? > > > > We do have a lot of those functions in our native libraries. And they > > are correct just the way they are. > > Yep all well and good. A function declared native in Java must have an > implementation as you describe. But not all native functions exist to > provide the native-half of a Java native function! > > > However, we also have *other* native functions, that are exported, not > > as JNI functions that should be called from Java, but as normal native > > library functions that should be called by other native code. Okay so > > far? And *those* functions have been problematic in how we decorate > > But there are again two cases. Those functions exported from a library > that are expected to be called from external code using the JNI > interface mechanism - such as all the JNI functions and JVM TI functions > we export from the JVM - and those "exported" for access between > libraries within the JDK (such as all the JVM_* functions in libjvm). > > I think it is only the second group that should be addressed by your > JDK_EXPORT proposal - though I'm not completely clear exactly how to > identify them. > > > them. My proposal is that we *refrain* from using JNIEXPORT for those > > functions, and instead use JDK_EXPORT as name for the macro that > > decorates them as exported. That way, we can clearly see that a function > > like this: > > > > JDK_EXPORT void > > JLI_ReadEnv(char* env); > > > > is correctly declared, and will be exported to other native libraries, > > but not to Java. > > The issue is not whether it is "exported to Java"** but whether it is > exported using the JNI mechanism such that other native code calls it > using the JNI mechanism. > > ** There is no way to write a native method declaration in Java such > that it would be linked to the JLI_ReadEnv function. The naming is all > wrong, as is the signature. > But that's exactly what this change is about! Remove the usage of JNIEXPORT from functions which are NOT exported using the JNI mechanism. What don't you like about it ? > > Just to clarify, this has nothing to do with if this is a officially > > supported API or not. In general though, I assume that most (if not > > all?) of our exported functions (apart from the JNI_* stuff) is supposed > > to be consumed by other libraries in the JDK, and is not a public API. > > I think it varies library by library. You may need native application > code that can call directly into native JDK libraries. JLI is the Java > Launcher Interface - I think it was introduced to make it easier for > other launchers to be created. Native agents may need access to > libmanagement or libjdwp functions. Native graphics code may need access > to the JDK graphics library. Some of these accesses may be unsupported > and undocumented, but I don't think you can just cut them all off. > Nobody wants to cut off anything. Magnus only proposes to decorate these required functions with the new JDK_EXPORT macro (instead of JNIEXPORT) in order to make it clear that they are not exported by using the JNI mechanism (but they will still be exported, technically speaking, JDK_EXPORT will even resolve to the exact same function modifiers!). > David > > > > > /Magnus > > > > > > > >> > >>> JNICALL. All current instances of JNIEXPORT which is not pure JNI > >>> native functions should be changed to use JDK_EXPORT instead. > >>> > >>> I further propose that this macro should reside in a new file > >>> "jdk.h", placed in the new directory > >>> src/java.base/share/native/include/internal. This header file path > >>> will automatically be provided to all native libraries, but not > >>> copied to the JDK being built. (The existence of a "include/internal" > >>> directory with this behavior has been discussed before. There are > >>> more files that ought to be moved there, if/when it is created.) I > >>> believe in many cases the #include "jni.h" can be just modified to > >>> #include "#jdk.h", since most native code will not require "jni.h" > >>> unless actually doing JNI calls -- most have included this file to > >>> get the JNIEXPORT macro, which would explain the pervasive use of > >>> #include "jni.h" in our code base. > >> > >> jni.h also defines all of the types used by the JNI. Those types are > >> pervsive to the native code used throughout the JDK. > >> > >>> Thoughts? > >> > >> I think we need to understand the problems on Windows that prompted > >> all this. Then I think we need to look at exactly how jni.h and > >> JNIEXPORT etc are being used and understand whether this is truly an > >> exported interface or not. > >> > >> Cheers, > >> David > >> > >>> /Magnus > >>> > > From david.holmes at oracle.com Wed Dec 12 08:40:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Dec 2018 18:40:02 +1000 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> Message-ID: <1d74b9d1-e467-674f-7170-48b48c943a4e@oracle.com> On 12/12/2018 5:44 pm, Volker Simonis wrote: > On Tue, Dec 11, 2018 at 11:47 PM David Holmes wrote: >> >> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >>> >>> >>> On 2018-12-11 00:23, David Holmes wrote: >>>> Hi Magnus, >>>> >>>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>>> I propose that we introduce a new define, available to all JDK native >>>>> files (Hotspot included), called JDK_EXPORT. The behavior of this >>>>> symbol will be very similar (as of now, in fact identical) to >>>>> JNIEXPORT; however, the semantics will not. >>>>> >>>>> Currently, we "mis-use" the JNIEXPORT define to mark a function for >>>>> exporting from the library. The problem with this is that JNIEXPORT >>>>> is part of the JNI interface, and is supposed to be used when C >>>>> programs interact with Java. And, when doing this, the function >>>>> should be fully decorated like this: "JNIEXPORT foo JNICALL". >>>> >>>> I've seen a lot of the emails on this issue and I don't fully >>>> understand what has been going wrong. But the intent is obviously the >>>> JNIEXPORT represents what is needed to export this function for use by >>>> JNI, while JNICALL defines the calling convention. I agree there may >>>> be some mistmatch when functions are actually not intended for general >>>> export outside the JDK but are only for internal JDK use. >>>> >>>>> We do have many such JNI exports in our native libraries, but we also >>>>> have a lot of other, non-JNI exports, where one native library just >>>>> provides an interface to other libraries. In these cases, we have >>>>> still used JNIEXPORT for the functionality of getting the function >>>>> exported, but we have not been consistent in our use of JNICALL. This >>>>> has caused us way too much trouble for something that should Just >>>>> Work. >>>> >>>> Are you suggesting that the interface between different libraries in >>>> the JDK should not be a JNI interface? Is this because you think the >>>> functions in these libraries are only for JDK internal use or ... ?? >>>> >>>>> I therefore propose that we define "JDK_EXPORT", with the same >>>>> behavior as JNIEXPORT (that is, flagging the function for external >>>>> visibility in the resulting native library), but which is *not* >>>>> supposed to be exported to Java code using JNI, nor supposed to be >>>>> decorated with >>>> >>>> Just a clarification there. JNI functions are not exported to Java >>>> code, they are exported to native code. Java code can declare native >>>> methods and those native methods must be written as JNI functions, but >>>> that's not what we are discussing. Libraries expose a JNI interface (a >>>> set of functions in the library) that can be called by application >>>> native code, using JNI. >>> We're apparently looking at "JNI" and "exporting" from two opposite >>> sides here. :-) Just to make everything clear: If I have a Java class >>> class MyClass { >>> public static void native myNativeFunc(); >>> } >>> >>> then I have one half of the JNI function, the Java half. This must be >>> matched by a corresponding implementation in native, like this: >>> JNIEXPORT void JNICALL >>> Java_MyClass_myNativeFunc(void) { >>> // ... do stuff >>> } >>> >>> And this is the native half of the JNI function. Right? Let's leave >>> aside which side is "exporting" to the other for now. :-) >>> >>> This way of setting up native functions that can be called from Java is >>> what I refer to as JNI. And when you declare a native JNI function, you >>> *must* use both JNIEXPORT and JNICALL. Alright? >>> >>> We do have a lot of those functions in our native libraries. And they >>> are correct just the way they are. >> >> Yep all well and good. A function declared native in Java must have an >> implementation as you describe. But not all native functions exist to >> provide the native-half of a Java native function! >> >>> However, we also have *other* native functions, that are exported, not >>> as JNI functions that should be called from Java, but as normal native >>> library functions that should be called by other native code. Okay so >>> far? And *those* functions have been problematic in how we decorate >> >> But there are again two cases. Those functions exported from a library >> that are expected to be called from external code using the JNI >> interface mechanism - such as all the JNI functions and JVM TI functions >> we export from the JVM - and those "exported" for access between >> libraries within the JDK (such as all the JVM_* functions in libjvm). >> >> I think it is only the second group that should be addressed by your >> JDK_EXPORT proposal - though I'm not completely clear exactly how to >> identify them. >> >>> them. My proposal is that we *refrain* from using JNIEXPORT for those >>> functions, and instead use JDK_EXPORT as name for the macro that >>> decorates them as exported. That way, we can clearly see that a function >>> like this: >>> >>> JDK_EXPORT void >>> JLI_ReadEnv(char* env); >>> >>> is correctly declared, and will be exported to other native libraries, >>> but not to Java. >> >> The issue is not whether it is "exported to Java"** but whether it is >> exported using the JNI mechanism such that other native code calls it >> using the JNI mechanism. >> >> ** There is no way to write a native method declaration in Java such >> that it would be linked to the JLI_ReadEnv function. The naming is all >> wrong, as is the signature. >> > > But that's exactly what this change is about! Remove the usage of > JNIEXPORT from functions which are NOT exported using the JNI > mechanism. What don't you like about it ? I'm just saying we need to be clear about what functions we plan on changing. Taking concrete examples as I don't see JLI_Read anywhere, we have: JNIEXPORT void JNICALL JLI_ReportErrorMessage(const char* fmt, ...) { JNIEXPORT void JNICALL JLI_ReportErrorMessageSys(const char* fmt, ...) { JNIEXPORT void JNICALL JLI_ReportExceptionDescription(JNIEnv * env) { JNIEXPORT StdArg JNICALL *JLI_GetStdArgs() { JNIEXPORT int JNICALL JLI_GetStdArgc() { return 0; } which seems to define the exported interface for (part of) libJLI and establishes both the export status and the expected calling convention. Would these be changed? >>> Just to clarify, this has nothing to do with if this is a officially >>> supported API or not. In general though, I assume that most (if not >>> all?) of our exported functions (apart from the JNI_* stuff) is supposed >>> to be consumed by other libraries in the JDK, and is not a public API. >> >> I think it varies library by library. You may need native application >> code that can call directly into native JDK libraries. JLI is the Java >> Launcher Interface - I think it was introduced to make it easier for >> other launchers to be created. Native agents may need access to >> libmanagement or libjdwp functions. Native graphics code may need access >> to the JDK graphics library. Some of these accesses may be unsupported >> and undocumented, but I don't think you can just cut them all off. >> > > Nobody wants to cut off anything. Magnus only proposes to decorate > these required functions with the new JDK_EXPORT macro (instead of > JNIEXPORT) in order to make it clear that they are not exported by > using the JNI mechanism (but they will still be exported, technically > speaking, JDK_EXPORT will even resolve to the exact same function > modifiers!). Maybe I've misunderstood the proposal. I thought some functions presently JNIEXPORT and JNICALL would be changed to just JDK_EXPORT as they are deemed not to be "JNI exported functions" as they are not the native-half of any Java native method. If the proposal is only to use JDK_EXPORT where JNICALL is missing (and JDK_EXPORT is identical to JNIEXPORT) that seems fine as it just serves as "documentation". But if JDKEXPORT differs from JNI_EXPORT then you may change the ability of external code to link to libraries it presently does link to. And the change to jdk.h versus jni.h is still unclear to me given jni.h defines the types that allow interaction with Java and which may still be needed in a JDK_EXPORT function implementation. I think there is overloaded use of the term "JNI" function in all this David ----- >> David >> >>> >>> /Magnus >>> >>> >>> >>>> >>>>> JNICALL. All current instances of JNIEXPORT which is not pure JNI >>>>> native functions should be changed to use JDK_EXPORT instead. >>>>> >>>>> I further propose that this macro should reside in a new file >>>>> "jdk.h", placed in the new directory >>>>> src/java.base/share/native/include/internal. This header file path >>>>> will automatically be provided to all native libraries, but not >>>>> copied to the JDK being built. (The existence of a "include/internal" >>>>> directory with this behavior has been discussed before. There are >>>>> more files that ought to be moved there, if/when it is created.) I >>>>> believe in many cases the #include "jni.h" can be just modified to >>>>> #include "#jdk.h", since most native code will not require "jni.h" >>>>> unless actually doing JNI calls -- most have included this file to >>>>> get the JNIEXPORT macro, which would explain the pervasive use of >>>>> #include "jni.h" in our code base. >>>> >>>> jni.h also defines all of the types used by the JNI. Those types are >>>> pervsive to the native code used throughout the JDK. >>>> >>>>> Thoughts? >>>> >>>> I think we need to understand the problems on Windows that prompted >>>> all this. Then I think we need to look at exactly how jni.h and >>>> JNIEXPORT etc are being used and understand whether this is truly an >>>> exported interface or not. >>>> >>>> Cheers, >>>> David >>>> >>>>> /Magnus >>>>> >>> From manc at google.com Wed Dec 12 10:30:22 2018 From: manc at google.com (Man Cao) Date: Wed, 12 Dec 2018 18:30:22 +0800 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> References: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> Message-ID: Hi, Addressed several comments. New webrev: https://cr.openjdk.java.net/~manc/8212206/webrev.01/ Diff from webrev.00: https://cr.openjdk.java.net/~manc/8212206/webrev.diff.00-01/ > > Assuming that all collectors want to implement this, and actually > > need to I am leaning towards doing so. However the ZGC/Shenandoah > > people might object to this. I haven't moved the GCOverheadChecker instance to CollectedHeap yet. Should I wait for ZGC/Shenandoah people to give some green light? > Creating a CSR and getting it approved is not a big deal; it may even > be useful as it clearly communicates the change to the users. > Additionally I think due to that translation table I mentioned, the old > name can still be used I think. As for the hsperfdata counter sun.gc.policy.gcTimeLimitExceeded, I found two issues: (a) The translation table in aliasmap seems to mainly target JDK-internal usage of the counters. Only the PerfDataBufferImpl.findByName() method makes use of the aliasmap. There are use cases that doesn't work with the aliasmap. E.g.: $ jstat -J-Djstat.showUnsupported=true -name java.ci.totalTime // This works $ jstat -J-Djstat.showUnsupported=true -name hotspot.ci.total.time // This doesn't work This is because the "jstat -name" would invoke the PerfDataBufferImpl.findByPattern() method, which does not take the aliasmap into account. In addition, there are independent implementations that read /tmp/hsperfdata_/ file directly, e.g.: https://github.com/twitter/commons/tree/master/src/python/twitter/common/java/perfdata And Google internally has a Java implementation that does the job (but uses Guava library). These tools do not support aliasmap. As for this counter, fortunately I found it hardly used anywhere in OpenJDK or across Google's depot. And its current implementation is not that useful, as described below. (b) This counter contains a boolean value that is set at every GC. This makes its usefulness limited, as it is very hard to catch the moment when it is set to 1. When a full GC sets it to 1 and throws an OOM exception due to GC overhead exceeded, the JVM could subsequently trigger another full GC and reset the counter to 0, then terminates due to the OOM exception. If -XX:PerfDataSaveFile=foo.hsperf is set, foo.hsperf would contain 0 for this counter in this case, which is quite unexpected. I'd propose to change this counter to a cumulative counter, i.e, the total number of GCs that trigger GC-overhead-limit-exceeded event, and rename this counter as the same time. I think it is cleaner to do this change in a separate RFE and CSR. What do you think? Thanks, Man From edward.nevill at gmail.com Wed Dec 12 10:55:48 2018 From: edward.nevill at gmail.com (Edward Nevill) Date: Wed, 12 Dec 2018 10:55:48 +0000 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> Message-ID: <5bd2e4c606f6ba53f924fb744dbce97e898932e1.camel@gmail.com> Hi, Change looks fine to me, but I am not a *Reviewer*. All the best, Ed. On Tue, 2018-12-11 at 17:30 -0800, dean.long at oracle.com wrote: > I can help you commit it, but first I think another review would be > good. > > dl > > On 12/11/18 5:15 PM, Nick Gasson (Arm Technology China) wrote: > > > Hi Nick. This change looks good to me. > > > > Thanks Dean. Could you help me to commit this patch if it doesn't > > need other reviews? > > > > Nick > > > > > > > -----Original Message----- > > > From: dean.long at oracle.com > > > Sent: 12 December 2018 06:05 > > > To: Nick Gasson (Arm Technology China) ; > > > hotspot- > > > dev at openjdk.java.net; Boris Ulasevich > > m> > > > Cc: nd ; hotspot-compiler-dev at openjdk.java.net > > > compiler > > > ; aarch32-port- > > > dev at openjdk.java.net > > > Subject: Re: RFR: 8214512: Jtreg test > > > compiler/c2/Test8062950.java fails on > > > ARM32 > > > > > > Hi Nick. This change looks good to me. > > > > > > dl > > > > > > On 12/2/18 10:06 PM, Nick Gasson wrote: > > > > Hi, > > > > > > > > Could someone please help me review this fix for a test failure > > > > on ARM32: > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 > > > > Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ > > > > From magnus.ihse.bursie at oracle.com Wed Dec 12 10:54:56 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 12 Dec 2018 11:54:56 +0100 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: <1d74b9d1-e467-674f-7170-48b48c943a4e@oracle.com> References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> <1d74b9d1-e467-674f-7170-48b48c943a4e@oracle.com> Message-ID: <0f80432d-fd9d-d510-e9b0-d2b7100e8a08@oracle.com> On 2018-12-12 09:40, David Holmes wrote: > On 12/12/2018 5:44 pm, Volker Simonis wrote: >> On Tue, Dec 11, 2018 at 11:47 PM David Holmes >> wrote: >>> >>> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >>>> >>>> >>>> On 2018-12-11 00:23, David Holmes wrote: >>>>> Hi Magnus, >>>>> >>>>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>>>> I propose that we introduce a new define, available to all JDK >>>>>> native >>>>>> files (Hotspot included), called JDK_EXPORT. The behavior of this >>>>>> symbol will be very similar (as of now, in fact identical) to >>>>>> JNIEXPORT; however, the semantics will not. >>>>>> >>>>>> Currently, we "mis-use" the JNIEXPORT define to mark a function for >>>>>> exporting from the library. The problem with this is that JNIEXPORT >>>>>> is part of the JNI interface, and is supposed to be used when C >>>>>> programs interact with Java. And, when doing this, the function >>>>>> should be fully decorated like this: "JNIEXPORT foo JNICALL". >>>>> >>>>> I've seen a lot of the emails on this issue and I don't fully >>>>> understand what has been going wrong. But the intent is obviously the >>>>> JNIEXPORT represents what is needed to export this function for >>>>> use by >>>>> JNI, while JNICALL defines the calling convention. I agree there may >>>>> be some mistmatch when functions are actually not intended for >>>>> general >>>>> export outside the JDK but are only for internal JDK use. >>>>> >>>>>> We do have many such JNI exports in our native libraries, but we >>>>>> also >>>>>> have a lot of other, non-JNI exports, where one native library just >>>>>> provides an interface to other libraries. In these cases, we have >>>>>> still used JNIEXPORT for the functionality of getting the function >>>>>> exported, but we have not been consistent in our use of JNICALL. >>>>>> This >>>>>> has caused us way too much trouble for something that should Just >>>>>> Work. >>>>> >>>>> Are you suggesting that the interface between different libraries in >>>>> the JDK should not be a JNI interface? Is this because you think the >>>>> functions in these libraries are only for JDK internal use or ... ?? >>>>> >>>>>> I therefore propose that we define "JDK_EXPORT", with the same >>>>>> behavior as JNIEXPORT (that is, flagging the function for external >>>>>> visibility in the resulting native library), but which is *not* >>>>>> supposed to be exported to Java code using JNI, nor supposed to be >>>>>> decorated with >>>>> >>>>> Just a clarification there. JNI functions are not exported to Java >>>>> code, they are exported to native code. Java code can declare native >>>>> methods and those native methods must be written as JNI functions, >>>>> but >>>>> that's not what we are discussing. Libraries expose a JNI >>>>> interface (a >>>>> set of functions in the library) that can be called by application >>>>> native code, using JNI. >>>> We're apparently looking at "JNI" and "exporting" from two opposite >>>> sides here. :-) Just to make everything clear: If I have a Java class >>>> class MyClass { >>>> public static void native myNativeFunc(); >>>> } >>>> >>>> then I have one half of the JNI function, the Java half. This must be >>>> matched by a corresponding implementation in native, like this: >>>> JNIEXPORT void JNICALL >>>> Java_MyClass_myNativeFunc(void) { >>>> // ... do stuff >>>> } >>>> >>>> And this is the native half of the JNI function. Right? Let's leave >>>> aside which side is "exporting" to the other for now. :-) >>>> >>>> This way of setting up native functions that can be called from >>>> Java is >>>> what I refer to as JNI. And when you declare a native JNI function, >>>> you >>>> *must* use both JNIEXPORT and JNICALL. Alright? >>>> >>>> We do have a lot of those functions in our native libraries. And they >>>> are correct just the way they are. >>> >>> Yep all well and good. A function declared native in Java must have an >>> implementation as you describe. But not all native functions exist to >>> provide the native-half of a Java native function! >>> >>>> However, we also have *other* native functions, that are exported, not >>>> as JNI functions that should be called from Java, but as normal native >>>> library functions that should be called by other native code. Okay so >>>> far? And *those* functions have been problematic in how we decorate >>> >>> But there are again two cases. Those functions exported from a library >>> that are expected to be called from external code using the JNI >>> interface mechanism - such as all the JNI functions and JVM TI >>> functions >>> we export from the JVM - and those "exported" for access between >>> libraries within the JDK (such as all the JVM_* functions in libjvm). >>> >>> I think it is only the second group that should be addressed by your >>> JDK_EXPORT proposal - though I'm not completely clear exactly how to >>> identify them. >>> >>>> them. My proposal is that we *refrain* from using JNIEXPORT for those >>>> functions, and instead use JDK_EXPORT as name for the macro that >>>> decorates them as exported. That way, we can clearly see that a >>>> function >>>> like this: >>>> >>>> JDK_EXPORT void >>>> JLI_ReadEnv(char* env); >>>> >>>> is correctly declared, and will be exported to other native libraries, >>>> but not to Java. >>> >>> The issue is not whether it is "exported to Java"** but whether it is >>> exported using the JNI mechanism such that other native code calls it >>> using the JNI mechanism. >>> >>> ** There is no way to write a native method declaration in Java such >>> that it would be linked to the JLI_ReadEnv function. The naming is all >>> wrong, as is the signature. >>> >> >> But that's exactly what this change is about! Remove the usage of >> JNIEXPORT from functions which are NOT exported using the JNI >> mechanism. What don't you like about it ? > > I'm just saying we need to be clear about what functions we plan on > changing. Taking concrete examples as I don't see JLI_Read anywhere, > we have: > > JNIEXPORT void JNICALL > JLI_ReportErrorMessage(const char* fmt, ...) { > > JNIEXPORT void JNICALL > JLI_ReportErrorMessageSys(const char* fmt, ...) { > > JNIEXPORT void JNICALL > JLI_ReportExceptionDescription(JNIEnv * env) { > > JNIEXPORT StdArg JNICALL > *JLI_GetStdArgs() > { > > JNIEXPORT int JNICALL > JLI_GetStdArgc() { > return 0; > } > > which seems to define the exported interface for (part of) libJLI and > establishes both the export status and the expected calling > convention. Would these be changed? Yes! Those are exactly the kind of functions that should change. At this point, I intended to go out and make a full list of all functions that I wanted to change, but a grep for JNIEXPORT resulted in over 3800 hits, so I'm not doing it now. (However, if my proposal is eventually accepted, I will need to go through these 3800 calls and check them up. Fortunately, with some scripting, the search can be drastically limited.) These functions you list *cannot* be called from Java. There is no corresponding Java native methods. All suchs methods need to follow the JNI calling convention, which does not *only* put requirements on the decoration, but also on the name ("Java_fully_qualified_class_name_and_method") and on the argument list (first argument must be a JNIEnv*). For the purpose of this discussion, I'm calling functions that fulfil these requirements *JNI functions*. The functions you listed above is *not* JNI function. OK? Other examples of non-JNI functions are e.g: JNIEXPORT void JNICALL SplashSetScaleFactor(float); (from splashscreen) or JNIEXPORT jboolean JNICALL doDrawPath(DrawHandler* hnd, ...) Contrast this with: JNIEXPORT void JNICALL Java_sun_java2d_loops_DrawPath_DrawPath(JNIEnv *env, jobject self, ...) The latter function is called, not from some other native code, but directly from java, since there is a sun.java2d.loops.DrawPath.DrawPath() method (it's apparently a native constructor, a bit odd, but hey... 2d...). The former function is called from *other native libraries in the JDK*, from native code, using #import "java2d/loops/ProcessPath.h" and then calling doDrawPath(). I think there's an obvious difference between these two, and I'm getting increasingly frustrated that I'm unable to communicate this difference to you. :-( Are you with me now, with this classification? My proposal is that we should *only* use JNIEXPORT and JNICALL in the latter case, the JNI case. The former case, the non-JNI, "native lib to native lib" function, should use JDK_EXPORT instead of JNIEXPORT, so we let JNIEXPORT correctly signify that the function marked is a JNI-call. *Also*, we should remove JNICALL as well. Since these functions are just ways for our native libraries to communicate with another, we can determine to change the calling convention at will, just as if we've added a new argument to a function signature. Also, for practical purposes, JNICALL is empty for all platforms except Windows 32. And, just to be extremely clear, I do *not* propose we change official APIs. Documented, public function calls will not be affected. This includes e.g. JNIEXPORT jint JNICALL JNI_CreateJavaVM(JavaVM **pvm, void **penv, void *args); which will continue to look like that, even though it can not be called from Java. It also applies to all other functions in jni.h, and other documented header files. > >>>> Just to clarify, this has nothing to do with if this is a officially >>>> supported API or not. In general though, I assume that most (if not >>>> all?) of our exported functions (apart from the JNI_* stuff) is >>>> supposed >>>> to be consumed by other libraries in the JDK, and is not a public API. >>> >>> I think it varies library by library. You may need native application >>> code that can call directly into native JDK libraries. JLI is the Java >>> Launcher Interface - I think it was introduced to make it easier for >>> other launchers to be created. Native agents may need access to >>> libmanagement or libjdwp functions. Native graphics code may need >>> access >>> to the JDK graphics library. Some of these accesses may be unsupported >>> and undocumented, but I don't think you can just cut them all off. >>> >> >> Nobody wants to cut off anything. Magnus only proposes to decorate >> these required functions with the new JDK_EXPORT macro (instead of >> JNIEXPORT) in order to make it clear that they are not exported by >> using the JNI mechanism (but they will still be exported, technically >> speaking, JDK_EXPORT will even resolve to the exact same function >> modifiers!). > > Maybe I've misunderstood the proposal. I thought some functions > presently JNIEXPORT and JNICALL would be changed to just JDK_EXPORT as > they are deemed not to be "JNI exported functions" as they are not the > native-half of any Java native method. If I understand you correctly, this is what I'm proposing. > If the proposal is only to use JDK_EXPORT where JNICALL is missing > (and JDK_EXPORT is identical to JNIEXPORT) that seems fine as it just > serves as "documentation". But if JDKEXPORT differs from JNI_EXPORT > then you may change the ability of external code to link to libraries > it presently does link to. JDK_EXPORT and JNIEXPORT will be identical. The difference is, we can in the future change JDK_EXPORT at will, without having to do a CCC or risk breaking tons of user code. For instance, in AIX, we build native libraries with all symbols for xlc version 12 or below, but for version 13 doing proper symbol visibility becomes doable. In that case, the AIX team might find that it would be a good idea to declare JNIEXPORT as a way to set exported visibility on functions, to help user's developing native code, but we might still keep JDK_EXPORT as a no-op (and continue exporting all symbols) for our internal need. (Not saying any of this is a good idea, but it's an example of the freedom it gives us.) I think you are trying to use "JNICALL is missing" as a way to measure either: 1) the function is a JNI-function, an implementation of a Java native method, or 2) the function is officially documented Unfortunately, neither of these are true. And the presence, or absence, of JNICALL is at this time mostly arbitrary, for non-JNI ("lib to lib") functions. Which is one thing that has caused us a lot of trouble! > > And the change to jdk.h versus jni.h is still unclear to me given > jni.h defines the types that allow interaction with Java and which may > still be needed in a JDK_EXPORT function implementation. > > I think there is overloaded use of the term "JNI" function in all this Yes, that is one part of the confusion I want to address. There is no need to bring in "JNI" in functions that has nothing to do with JNI. /Magnus > > David > ----- > >>> David >>> >>>> >>>> /Magnus >>>> >>>> >>>> >>>>> >>>>>> JNICALL. All current instances of JNIEXPORT which is not pure JNI >>>>>> native functions should be changed to use JDK_EXPORT instead. >>>>>> >>>>>> I further propose that this macro should reside in a new file >>>>>> "jdk.h", placed in the new directory >>>>>> src/java.base/share/native/include/internal. This header file path >>>>>> will automatically be provided to all native libraries, but not >>>>>> copied to the JDK being built. (The existence of a >>>>>> "include/internal" >>>>>> directory with this behavior has been discussed before. There are >>>>>> more files that ought to be moved there, if/when it is created.) I >>>>>> believe in many cases the #include "jni.h" can be just modified to >>>>>> #include "#jdk.h", since most native code will not require "jni.h" >>>>>> unless actually doing JNI calls -- most have included this file to >>>>>> get the JNIEXPORT macro, which would explain the pervasive use of >>>>>> #include "jni.h" in our code base. >>>>> >>>>> jni.h also defines all of the types used by the JNI. Those types are >>>>> pervsive to the native code used throughout the JDK. >>>>> >>>>>> Thoughts? >>>>> >>>>> I think we need to understand the problems on Windows that prompted >>>>> all this. Then I think we need to look at exactly how jni.h and >>>>> JNIEXPORT etc are being used and understand whether this is truly an >>>>> exported interface or not. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> /Magnus >>>>>> >>>> From magnus.ihse.bursie at oracle.com Wed Dec 12 11:03:20 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 12 Dec 2018 12:03:20 +0100 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> Message-ID: <236420f2-cd56-75a7-db52-59a7ef9f9be3@oracle.com> On 2018-12-11 23:47, David Holmes wrote: > On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >> >> >> On 2018-12-11 00:23, David Holmes wrote: >>> Hi Magnus, >>> >>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>> I propose that we introduce a new define, available to all JDK >>>> native files (Hotspot included), called JDK_EXPORT. The behavior of >>>> this symbol will be very similar (as of now, in fact identical) to >>>> JNIEXPORT; however, the semantics will not. >>>> >>>> Currently, we "mis-use" the JNIEXPORT define to mark a function for >>>> exporting from the library. The problem with this is that JNIEXPORT >>>> is part of the JNI interface, and is supposed to be used when C >>>> programs interact with Java. And, when doing this, the function >>>> should be fully decorated like this: "JNIEXPORT foo JNICALL". >>> >>> I've seen a lot of the emails on this issue and I don't fully >>> understand what has been going wrong. But the intent is obviously >>> the JNIEXPORT represents what is needed to export this function for >>> use by JNI, while JNICALL defines the calling convention. I agree >>> there may be some mistmatch when functions are actually not intended >>> for general export outside the JDK but are only for internal JDK use. >>> >>>> We do have many such JNI exports in our native libraries, but we >>>> also have a lot of other, non-JNI exports, where one native library >>>> just provides an interface to other libraries. In these cases, we >>>> have still used JNIEXPORT for the functionality of getting the >>>> function exported, but we have not been consistent in our use of >>>> JNICALL. This has caused us way too much trouble for something that >>>> should Just Work. >>> >>> Are you suggesting that the interface between different libraries in >>> the JDK should not be a JNI interface? Is this because you think the >>> functions in these libraries are only for JDK internal use or ... ?? >>> >>>> I therefore propose that we define "JDK_EXPORT", with the same >>>> behavior as JNIEXPORT (that is, flagging the function for external >>>> visibility in the resulting native library), but which is *not* >>>> supposed to be exported to Java code using JNI, nor supposed to be >>>> decorated with >>> >>> Just a clarification there. JNI functions are not exported to Java >>> code, they are exported to native code. Java code can declare native >>> methods and those native methods must be written as JNI functions, >>> but that's not what we are discussing. Libraries expose a JNI >>> interface (a set of functions in the library) that can be called by >>> application native code, using JNI. >> We're apparently looking at "JNI" and "exporting" from two opposite >> sides here. :-) Just to make everything clear: If I have a Java class >> class MyClass { >> public static void native myNativeFunc(); >> } >> >> then I have one half of the JNI function, the Java half. This must be >> matched by a corresponding implementation in native, like this: >> JNIEXPORT void JNICALL >> Java_MyClass_myNativeFunc(void) { >> // ... do stuff >> } >> >> And this is the native half of the JNI function. Right? Let's leave >> aside which side is "exporting" to the other for now. :-) >> >> This way of setting up native functions that can be called from Java >> is what I refer to as JNI. And when you declare a native JNI >> function, you *must* use both JNIEXPORT and JNICALL. Alright? >> >> We do have a lot of those functions in our native libraries. And they >> are correct just the way they are. > > Yep all well and good. A function declared native in Java must have an > implementation as you describe. But not all native functions exist to > provide the native-half of a Java native function! > >> However, we also have *other* native functions, that are exported, >> not as JNI functions that should be called from Java, but as normal >> native library functions that should be called by other native code. >> Okay so far? And *those* functions have been problematic in how we >> decorate > > But there are again two cases. Those functions exported from a library > that are expected to be called from external code using the JNI > interface mechanism - such as all the JNI functions and JVM TI > functions we export from the JVM - and those "exported" for access > between libraries within the JDK (such as all the JVM_* functions in > libjvm). > > I think it is only the second group that should be addressed by your > JDK_EXPORT proposal - though I'm not completely clear exactly how to > identify them. Alright, now I think we're on the same page again. :) Yes, this is what I'm saying. I'm not proposing to modify public APIs. You identify external APIs by the fact that they are documented. And if you are unsure, you ask Jon. ;-) > >> them. My proposal is that we *refrain* from using JNIEXPORT for those >> functions, and instead use JDK_EXPORT as name for the macro that >> decorates them as exported. That way, we can clearly see that a >> function like this: >> >> JDK_EXPORT void >> JLI_ReadEnv(char* env); >> >> is correctly declared, and will be exported to other native >> libraries, but not to Java. > > The issue is not whether it is "exported to Java"** but whether it is > exported using the JNI mechanism such that other native code calls it > using the JNI mechanism. > > ** There is no way to write a native method declaration in Java such > that it would be linked to the JLI_ReadEnv function. The naming is all > wrong, as is the signature. > >> Just to clarify, this has nothing to do with if this is a officially >> supported API or not. In general though, I assume that most (if not >> all?) of our exported functions (apart from the JNI_* stuff) is >> supposed to be consumed by other libraries in the JDK, and is not a >> public API. > > I think it varies library by library. You may need native application > code that can call directly into native JDK libraries. JLI is the Java > Launcher Interface - I think it was introduced to make it easier for > other launchers to be created. Native agents may need access to > libmanagement or libjdwp functions. Native graphics code may need > access to the JDK graphics library. Some of these accesses may be > unsupported and undocumented, but I don't think you can just cut them > all off. If they are unsupported and undocumented, I sure as h*ll can cut them all off! :-) Besides, and let me re-iterate this, the only change that will happen by removing JNICALL, is on Windows 32. No other platform will be affected. There is no official support for Windows 32 anymore. There's some low-effort community work done on keeping Windows 32 alive, but it's not backed by any major player. Right now, they are taking a lot of hits *due to the fact* that we have not sorted this out, and waste a lot of their precious effort trying to fix this piecemeal. If we do fix things according to this proposal, then it's at least clear how things *should* work. If it turns out that there's some code out there in the wild, running on Windows 32, that uses an undocumented and unsupported function call, and it breaks -- well, sucks to be them. They'll just have to do what everyone does who uses an undocumented interface that suddenly changes: update their code. /Magnus > > David > >> >> /Magnus >> >> >> >>> >>>> JNICALL. All current instances of JNIEXPORT which is not pure JNI >>>> native functions should be changed to use JDK_EXPORT instead. >>>> >>>> I further propose that this macro should reside in a new file >>>> "jdk.h", placed in the new directory >>>> src/java.base/share/native/include/internal. This header file path >>>> will automatically be provided to all native libraries, but not >>>> copied to the JDK being built. (The existence of a >>>> "include/internal" directory with this behavior has been discussed >>>> before. There are more files that ought to be moved there, if/when >>>> it is created.) I believe in many cases the #include "jni.h" can be >>>> just modified to #include "#jdk.h", since most native code will not >>>> require "jni.h" unless actually doing JNI calls -- most have >>>> included this file to get the JNIEXPORT macro, which would explain >>>> the pervasive use of #include "jni.h" in our code base. >>> >>> jni.h also defines all of the types used by the JNI. Those types are >>> pervsive to the native code used throughout the JDK. >>> >>>> Thoughts? >>> >>> I think we need to understand the problems on Windows that prompted >>> all this. Then I think we need to look at exactly how jni.h and >>> JNIEXPORT etc are being used and understand whether this is truly an >>> exported interface or not. >>> >>> Cheers, >>> David >>> >>>> /Magnus >>>> >> From shade at redhat.com Wed Dec 12 11:22:49 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Dec 2018 12:22:49 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: <5c21c712-4292-e704-f162-cd0ff8b5228a@redhat.com> References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> <5c21c712-4292-e704-f162-cd0ff8b5228a@redhat.com> Message-ID: On 12/11/18 4:18 PM, Roman Kennke wrote: > this looks good to me. Thanks for reviews! I fixed "jint" -> "int" in atomic flag to make Windows and Solaris builds happy, updated the webrev in-place: http://cr.openjdk.java.net/~shade/8181143/webrev.04/ Testing: x86_64 build, hotspot tier1, new test, jdk-submit (passed!) -Aleksey From rkennke at redhat.com Wed Dec 12 11:27:00 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 12 Dec 2018 12:27:00 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> <5c21c712-4292-e704-f162-cd0ff8b5228a@redhat.com> Message-ID: >> this looks good to me. > > Thanks for reviews! I fixed "jint" -> "int" in atomic flag to make Windows and Solaris builds happy, > updated the webrev in-place: > http://cr.openjdk.java.net/~shade/8181143/webrev.04/ > > Testing: x86_64 build, hotspot tier1, new test, jdk-submit (passed!) ok, still good :-) Roman From boris.ulasevich at bell-sw.com Wed Dec 12 11:41:55 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Wed, 12 Dec 2018 14:41:55 +0300 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: Message-ID: Hi, The change is Ok for me. > and let the runtime handle the failure case. One minor comment. It would be good to add inline comment to MacroAssembler::fast_lock bottom. Compare with other ports: aarch: __ bind(cont); aarch: // flag == EQ indicates success aarch: // flag == NE indicates failure x86: // At DONE_LABEL the icc ZFlag is set as follows ... x86: // Fast_Unlock uses the same protocol. x86: // ZFlag == 1 -> Success x86: // ZFlag == 0 -> Failure - force control through the slow-path best regards, Boris On 03.12.2018 9:06, Nick Gasson wrote: > Hi, > > Could someone please help me review this fix for a test failure on ARM32: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 > Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ > > This fixes two assertion failures related to biased locking when C2's > bias inlining feature is disabled (-XX:-OptoBiasInlining): > > MacroAssembler::atomic_cas_bool takes three register arguments plus a > temporary register for use in the CAS loop. If the caller passes > `noreg' as this temporary register (which happens in the > !OptoBiasInlining call path from MacroAssembler::fast_lock) then > MacroAssembler::atomic_cas_bool will default to using LR as a > temporary. But it's possible with C2 that LR is one of the other three > arguments which then triggers an assert_different_registers assertion > failure. Fix this by supplying an additional scratch register to > MacroAssembler::fast_lock if !OptoBiasInlining. > > In the !OptoBiasInlining case MacroAssembler::fast_lock calls > MacroAssembler::biased_locking_enter to handle the case where the mark > word contains the biased lock pattern which fast_lock wouldn't > otherwise see if OptoBiasInlining was true. However in the case that > we fail to acquire the biased lock this falls through to label > `failed' and runs the normal fast_lock code that implicitly assumes > the mark word does not have the bias pattern. This means we can end up > with a BasicLock where _displaced_header contains the biased lock > pattern which is an illegal state and subsequently triggers an > assertion failure in ObjectSynchronizer::fast_exit. > > The right thing to do here is branch to `done' whenever the bias lock > pattern is present and let the runtime handle the failure case. Also > edited the comment describing MacroAssembler::biased_locking_enter to > more accurately describe what it does (the comment about `slow_case' > being optional is wrong). > > Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this > patch. > > Thanks, > Nick > From david.holmes at oracle.com Wed Dec 12 12:17:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Dec 2018 22:17:15 +1000 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: <236420f2-cd56-75a7-db52-59a7ef9f9be3@oracle.com> References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> <236420f2-cd56-75a7-db52-59a7ef9f9be3@oracle.com> Message-ID: Okay I went away and did some homework ... Let me back up a bit and see if I have the evolution of this correctly. The native implementation of Java methods should be declared and defined with JNIEXPORT and JNICALL. JNIEXPORT controls the export visibility so they can be linked. JNICALL controls the calling convention and is needed so that the JVM's calling convention matches the native code. [This part was unclear to me.] Other native methods exported from the same (or different) native libraries may also be decorated with JNIEXPORT and JNICALL. But in places there is a mismatch between the declaration in the header and the definition leading to errors. There are two main types of such native functions: a) those publicly defined in the various native APIs: JNI itself (jni.h), JVM TI (jvmti.h), AWT (jawt.h) ... b) those intended only for use by other native code within the JDK libraries (JLI_* - though I note Alan's comment re javafxpackager, others??) and a possible third type are callback functions like the JPLISAgent event handlers (e.g. eventHandlerVMInit). There is a view that no native method that isn't the native-half of a Java method should use JNICALL. [Okay I can see how that makes sense - regardless of the actual calling convention used marking such methods as "must use the same calling convention as the VM native method call" is somewhat meaningless. Yet obviously the public native APIs do specify JNICALL so this is not a hard and fast rule. Further the callbacks must also follow a convention known to the VM doing the calling!] Where we have (introduced?) a discrepancy between the definition and declaration the approach has been (because of the previous view) to remove JNICALL. [This should only relate to methods of kind (b) above.] That leaves those methods with JNIEXPORT only. That seems wrong to you because they are not "JNI methods", so you want to replace with JDK_EXPORT to make it clear. [Ok - this seems reasonable.] If JNICALL is removed from those native methods and they are currently linked by external applications, those applications may stop working. But this only affects win32 (as its the only platform where JNICALL is different to the default C/C++ calling convention?) so your position is this is acceptable breakage - and would only affect unsupported/undocumented APIs. Does that sum it up? We still need to be sure that we're only changing functions of type (b) above. And I note that this has been driven by the choice to remove JNICALL where there was a discrepancy - that leads to the visibility of the two kinds of methods. If it had been chosen to add JNICALL where missing instead, then we may not have been having this conversation. This will also need a CSR request due to the change in linking behaviour. Cheers, David ----- On 12/12/2018 9:03 pm, Magnus Ihse Bursie wrote: > > > On 2018-12-11 23:47, David Holmes wrote: >> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >>> >>> >>> On 2018-12-11 00:23, David Holmes wrote: >>>> Hi Magnus, >>>> >>>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>>> I propose that we introduce a new define, available to all JDK >>>>> native files (Hotspot included), called JDK_EXPORT. The behavior of >>>>> this symbol will be very similar (as of now, in fact identical) to >>>>> JNIEXPORT; however, the semantics will not. >>>>> >>>>> Currently, we "mis-use" the JNIEXPORT define to mark a function for >>>>> exporting from the library. The problem with this is that JNIEXPORT >>>>> is part of the JNI interface, and is supposed to be used when C >>>>> programs interact with Java. And, when doing this, the function >>>>> should be fully decorated like this: "JNIEXPORT foo JNICALL". >>>> >>>> I've seen a lot of the emails on this issue and I don't fully >>>> understand what has been going wrong. But the intent is obviously >>>> the JNIEXPORT represents what is needed to export this function for >>>> use by JNI, while JNICALL defines the calling convention. I agree >>>> there may be some mistmatch when functions are actually not intended >>>> for general export outside the JDK but are only for internal JDK use. >>>> >>>>> We do have many such JNI exports in our native libraries, but we >>>>> also have a lot of other, non-JNI exports, where one native library >>>>> just provides an interface to other libraries. In these cases, we >>>>> have still used JNIEXPORT for the functionality of getting the >>>>> function exported, but we have not been consistent in our use of >>>>> JNICALL. This has caused us way too much trouble for something that >>>>> should Just Work. >>>> >>>> Are you suggesting that the interface between different libraries in >>>> the JDK should not be a JNI interface? Is this because you think the >>>> functions in these libraries are only for JDK internal use or ... ?? >>>> >>>>> I therefore propose that we define "JDK_EXPORT", with the same >>>>> behavior as JNIEXPORT (that is, flagging the function for external >>>>> visibility in the resulting native library), but which is *not* >>>>> supposed to be exported to Java code using JNI, nor supposed to be >>>>> decorated with >>>> >>>> Just a clarification there. JNI functions are not exported to Java >>>> code, they are exported to native code. Java code can declare native >>>> methods and those native methods must be written as JNI functions, >>>> but that's not what we are discussing. Libraries expose a JNI >>>> interface (a set of functions in the library) that can be called by >>>> application native code, using JNI. >>> We're apparently looking at "JNI" and "exporting" from two opposite >>> sides here. :-) Just to make everything clear: If I have a Java class >>> class MyClass { >>> ?? public static void native myNativeFunc(); >>> } >>> >>> then I have one half of the JNI function, the Java half. This must be >>> matched by a corresponding implementation in native, like this: >>> JNIEXPORT void JNICALL >>> Java_MyClass_myNativeFunc(void) { >>> // ... do stuff >>> } >>> >>> And this is the native half of the JNI function. Right? Let's leave >>> aside which side is "exporting" to the other for now. :-) >>> >>> This way of setting up native functions that can be called from Java >>> is what I refer to as JNI. And when you declare a native JNI >>> function, you *must* use both JNIEXPORT and JNICALL. Alright? >>> >>> We do have a lot of those functions in our native libraries. And they >>> are correct just the way they are. >> >> Yep all well and good. A function declared native in Java must have an >> implementation as you describe. But not all native functions exist to >> provide the native-half of a Java native function! >> >>> However, we also have *other* native functions, that are exported, >>> not as JNI functions that should be called from Java, but as normal >>> native library functions that should be called by other native code. >>> Okay so far? And *those* functions have been problematic in how we >>> decorate >> >> But there are again two cases. Those functions exported from a library >> that are expected to be called from external code using the JNI >> interface mechanism - such as all the JNI functions and JVM TI >> functions we export from the JVM - and those "exported" for access >> between libraries within the JDK (such as all the JVM_* functions in >> libjvm). >> >> I think it is only the second group that should be addressed by your >> JDK_EXPORT proposal - though I'm not completely clear exactly how to >> identify them. > Alright, now I think we're on the same page again. :) > > Yes, this is what I'm saying. I'm not proposing to modify public APIs. > > You identify external APIs by the fact that they are documented. And if > you are unsure, you ask Jon. ;-) > >> >>> them. My proposal is that we *refrain* from using JNIEXPORT for those >>> functions, and instead use JDK_EXPORT as name for the macro that >>> decorates them as exported. That way, we can clearly see that a >>> function like this: >>> >>> JDK_EXPORT void >>> JLI_ReadEnv(char* env); >>> >>> is correctly declared, and will be exported to other native >>> libraries, but not to Java. >> >> The issue is not whether it is "exported to Java"** but whether it is >> exported using the JNI mechanism such that other native code calls it >> using the JNI mechanism. >> >> ** There is no way to write a native method declaration in Java such >> that it would be linked to the JLI_ReadEnv function. The naming is all >> wrong, as is the signature. >> >>> Just to clarify, this has nothing to do with if this is a officially >>> supported API or not. In general though, I assume that most (if not >>> all?) of our exported functions (apart from the JNI_* stuff) is >>> supposed to be consumed by other libraries in the JDK, and is not a >>> public API. >> >> I think it varies library by library. You may need native application >> code that can call directly into native JDK libraries. JLI is the Java >> Launcher Interface - I think it was introduced to make it easier for >> other launchers to be created. Native agents may need access to >> libmanagement or libjdwp functions. Native graphics code may need >> access to the JDK graphics library. Some of these accesses may be >> unsupported and undocumented, but I don't think you can just cut them >> all off. > If they are unsupported and undocumented, I sure as h*ll can cut them > all off! :-) > > Besides, and let me re-iterate this, the only change that will happen by > removing JNICALL, is on Windows 32. No other platform will be affected. > There is no official support for Windows 32 anymore. There's some > low-effort community work done on keeping Windows 32 alive, but it's not > backed by any major player. Right now, they are taking a lot of hits > *due to the fact* that we have not sorted this out, and waste a lot of > their precious effort trying to fix this piecemeal. If we do fix things > according to this proposal, then it's at least clear how things *should* > work. If it turns out that there's some code out there in the wild, > running on Windows 32, that uses an undocumented and unsupported > function call, and it breaks -- well, sucks to be them. They'll just > have to do what everyone does who uses an undocumented interface that > suddenly changes: update their code. > > /Magnus > >> >> David >> >>> >>> /Magnus >>> >>> >>> >>>> >>>>> JNICALL. All current instances of JNIEXPORT which is not pure JNI >>>>> native functions should be changed to use JDK_EXPORT instead. >>>>> >>>>> I further propose that this macro should reside in a new file >>>>> "jdk.h", placed in the new directory >>>>> src/java.base/share/native/include/internal. This header file path >>>>> will automatically be provided to all native libraries, but not >>>>> copied to the JDK being built. (The existence of a >>>>> "include/internal" directory with this behavior has been discussed >>>>> before. There are more files that ought to be moved there, if/when >>>>> it is created.) I believe in many cases the #include "jni.h" can be >>>>> just modified to #include "#jdk.h", since most native code will not >>>>> require "jni.h" unless actually doing JNI calls -- most have >>>>> included this file to get the JNIEXPORT macro, which would explain >>>>> the pervasive use of #include "jni.h" in our code base. >>>> >>>> jni.h also defines all of the types used by the JNI. Those types are >>>> pervsive to the native code used throughout the JDK. >>>> >>>>> Thoughts? >>>> >>>> I think we need to understand the problems on Windows that prompted >>>> all this. Then I think we need to look at exactly how jni.h and >>>> JNIEXPORT etc are being used and understand whether this is truly an >>>> exported interface or not. >>>> >>>> Cheers, >>>> David >>>> >>>>> /Magnus >>>>> >>> > From david.holmes at oracle.com Wed Dec 12 12:28:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Dec 2018 22:28:18 +1000 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: Hi Aleksey, On 11/12/2018 10:05 pm, Aleksey Shipilev wrote: > Hi, > > Getting back to this. Replying to all three reviews in the same mail, because they share the > answers! New patch: > http://cr.openjdk.java.net/~shade/8181143/webrev.04/ Okay I can live with that. :) Three comments: 1. We don't use bug id's to name test directories any more (I see a couple have snuck in this past year! so please place the test in a suitable directory ... Safepoints? 2. -Xmx1g might be an issue on some test systems. Does it have to be that large? 3. Can you disable core dumping as part of the test so we don't get large core files generated. No need to see updated webrev. Thanks, David > Testing: x86_64 build, hotspot tier1, new test > > On 11/20/18 8:07 AM, Thomas St?fe wrote: >> I don't know whether I need 10ms resolution though - if we limit the >> goal to catching just hanging VMOps - which would still pretty useful >> - 1s would even be enough. > > Right. New patch implements variable delay, depending on what actual delay had been requested. New > test makes sure it works reliably. > >> I think we can do with just two flags, since VMOperationTimeout and >> VMOperationTimeoutDelay are redundant. Just go with the delay, make -1 >> the default that does nothing. Same could go for SafepointTimeout vs >> SafepointTimeoutDelay. > > Redid like this: > > diagnostic(bool, AbortVMOnVMOperationTimeout, false, \ > "Abort upon failure to complete VM operation promptly") \ > \ > diagnostic(intx, AbortVMOnVMOperationTimeoutDelay, 1000, \ > "Delay in milliseconds for option AbortVMOnVMOperationTimeout") \ > range(0, max_intx) \ > \ > >> I also agree with others that it would make more sense were we to kill >> the VMThread instead (e.g. just with pthread_kill(SIGILL) or similar) >> - we do something like that in error handling (see ErrorLogTimeout). > > I looked into that, and it seems too much hassle without significant benefit. For example, it would > require drilling new platform-specific holes to support it, and it also likely to be supported on > Linux (pthreads) only anyway? Seems to me piggybacking on periodic tasks is much simpler. In > development use, you'd have a core file to see what is going on in VM. In "production" use you'd > care that circuit breaker that did its job of terminating the VM. > > On 11/19/18 9:35 AM, Robbin Ehn wrote: >> You patch seems not to be against jdk/jdk (jdk12). > > Really? The new patch is definitely against jdk/jdk. > >> Without the actual core file, I don't see the hs file being very useful >> containing the watcherthread stacktrace. It would be good if the 'killer' >> signaled the VM thread and we do the error reporting from the signal handler >> instead in VM thread context. > > Right. Looked into that too, decided it is too complicated without significant benefit. See above. > > On 11/19/18 7:13 AM, David Holmes wrote: >> You're not just introducing one diagnostic flag, your introducing the entire VM operation >> timeout mechanism, including two product flags and one diagnostic. So the CR needs to reflect >> that clearly and you will need a CSR request to add the two product flags. And they will need to >> be documented. > > This was "solved" by doing two diagnostic flags, see above. I think the symmetry against other > AbortVM* flags is important, and having the boolean flag for AbortVM* is good. > >> It's not safe to access vm_safepoint_description() outside the VMThread as the _cur_vm_operation >> could be deleted while you're trying to access its name. > > Noted, removed that part. > >> And as we don't have a general timer mechanism this has to use polling so you pay for a 10ms >> task wakeup regardless of how long the timeout is. > > Redone that part, see above. > >> Given the limitations of the global timeout I'm not sure I see a use for the non-aborting form. >> This could just reduce down to: >> >> -XX:AbortVMOpsAfter:NN >> >> otherwise I don't really think this carries its weight. Of course that's just my opinion. >> Interested to hear others. > > I prefer not to introduce another shape for AbortVM* options. Two diagnostic options seems to have > much less weight on the whole thing. > > -Aleksey > From thomas.schatzl at oracle.com Wed Dec 12 13:06:07 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 12 Dec 2018 14:06:07 +0100 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> Message-ID: Hi Amit, On Sat, 2018-12-08 at 04:50 +0530, amith pawar wrote: > Hi Thomas, > > Please check my inline comments and attached patch is updated as per > your suggestion. thanks a lot! Looks very good now. Fixed issues: - you can use the LogStream directly for writing, you do not need to create an outputStream from it. - "isrunning_..." should be renamed "is_running_..." There are still some "is" prefixes glued to the predicate left. Here are webrevs: http://cr.openjdk.java.net/~tschatzl/8213827/webrev.3/ (latest webrev) http://cr.openjdk.java.net/~tschatzl/8213827/webrev.2_to_3/ (above suggested changes) http://cr.openjdk.java.net/~tschatzl/8213827/webrev.2/ (original changes) Some comments further down. > > On Mon, Dec 3, 2018 at 6:12 PM Thomas Schatzl < > thomas.schatzl at oracle.com> wrote: > > Hi, > > > > On Fri, 2018-11-30 at 11:11 +0100, Thomas Schatzl wrote: > > > Hi Amit, > > > > > > [...] > > > > > > > > Sorry. Please check the attached patch. > > > > > > I created a webrev at > > > http://cr.openjdk.java.net/~tschatzl/8213827/webrev/ . > > > > > > Let the reviews begin :) > > > > > > > style issues: > > > > - random whitespaces in many places. > > - all if's or if-then-else require braces > > - only class members have a leading underscore > > - do not declare local variables long before they are used. That > > makes the code hard to read. > > - do not introduce local variables for no reason. E.g. > > > > 2907 struct bitmask *bmp; > > > > [...unrelated code...] > > > > 2911 bmp = _numa_get_interleave_mask(); > > 2912 set_numa_interleave_ptr(&bmp); > > 2913 bmp = _numa_get_membind(); > > 2914 set_numa_membind_ptr(&bmp); > > > > is much less readable than > > > > set_numa_interleave_ptr(_numa_get_interleave_mask()); > > set_numa_membind_ptr(_numa_get_membind_mask()); > > > > - in the type declaration, please put the * next to the type. > > While > > this is kind-of wrong, the majority of existing code does so (not > > only > > in these files). > > > > code issues: > > > > - interleave/membind mode are mutually exclusive, which the code > > kind-of tries to establish (and check) using special values for eg. > > the > > value of Linux::_numa_interleave_ptr. > > > > I think the code gets a lot more readable if you introduce an enum > > and > > an explicit global variable holding the current NUMA node. If you > > do > > not want to do that, move the condition > > "Linux::_numa_interleave_ptr != > > 0" into a new method. > > > > - move the new NUMA initialization code in os::init_2() into a new > > method please. It's gotten very long now. > > > > - please avoid duplicating code; ie. > > > > 5039 // Check for membind mode. > > 5040 bmp = Linux::_numa_membind_ptr; > > 5041 for (int node = 0; node <= Linux::numa_max_node() ; > > node++) > > { > > 5042 if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > 5043 _is_membind = true; > > 5044 } > > 5045 } > > > > 5048 bmp = Linux::_numa_interleave_ptr; > > 5049 for (int node = 0; node <= Linux::numa_max_node() ; > > node++) > > { > > 5050 if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > 5051 _is_interleaved = true; > > 5052 // Set membind to false as interleave mode allows > > all > > nodes to be used. > > 5053 _is_membind = false; > > 5054 } > > 5055 } > > > > and right after: > > > > Something like: > > > > is_interleave = > > is_any_bit_in_bitmask_set(Linux::_numa_interleave_ptr); > > is_membind = !is_interleave && > > is_any_bit_in_bitmask_set(Linux::_numa_membind_ptr); > > > > with an appropriate is_any_bit_in_bitmask_set() helper method is > > much > > more clear. > > > > (Even better, as suggested above use a global "NUMA mode" enum that > > is > > filled in somewhere instead of multiple flags) > > > > - Use a LogStream to print the log messages instead of rolling > > your > > own code to create a string containing the node number. The code > > that > > is there exhibits a potential buffer overflow (with many NUMA > > nodes), > > and a potential out-of-bounds write. > > > > - the log message for selected NUMA mode and nodes should be a > > single > > log message. > > > > - partially pre-existing: the numa_set_xxx_ptr methods take a > > "struct bitmask**". Could you explain why "**"? All of those store > > the dereferenced result anyway, so why not pass the dereferenced > > pointer right away? > > Following bitmask pointers are defined in libnuma library. 'dlvsym' > will return the address of pointer which needs to be de-referenced. > Updated other methods as suggested. > struct bitmask *numa_all_nodes_ptr; > struct bitmask *numa_no_nodes_ptr; > struct bitmask *numa_all_cpus_ptr; > > This also simplifies the code at the callers. > > > > - partially pre-existing: I would prefer if the numa_set_xx_ptr > > methods were all renamed to numa_set_xx_bitmask. The "ptr" suffix > > seems redundant, and most importantly very unspecific. > > Newly introduced membind and interleave bitmask pointer names are > changed. Also above mentioned three pointers are imported. If you > suggest will rename these also. If you want we can do this in a separate changeset. Otherwise, please do right away. Please also fix the glued "is" in the isnode_in_bound_nodes, isbound_to_single_node, isnode_in_existing_nodes, isnode_in_configured_nodes methods. > > - in numa_interleave_memory(), why does the code not make the > > membind/interleave mode distinction when the v1 API is used? > > v2 API's works as expected for membind/interleave case but > numa_get_membind_v1 API's always returns null . v1 and v2 prototypes > are also not same. Please check below. > nodemask_t numa_get_membind_v1(void) > struct bitmask * numa_get_membind_v2(void) > > void numa_interleave_memory_v1(void *mem, size_t size, const > nodemask_t *mask) > void numa_interleave_memory_v2(void *mem, size_t size, struct bitmask > *bmp) Okay, understood. Thanks! Thanks, Thomas From shade at redhat.com Wed Dec 12 13:08:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Dec 2018 14:08:47 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: On 12/12/18 1:28 PM, David Holmes wrote: >> ?? http://cr.openjdk.java.net/~shade/8181143/webrev.04/ > > Okay I can live with that. :) Thanks! > Three comments: > > 1. We don't use bug id's to name test directories any more (I see a couple? have snuck in this past > year! so please place the test in a suitable directory ... Safepoints? Dropped. > 2. -Xmx1g might be an issue on some test systems. Does it have to be that large? Yeah, I was trying to make enough objects for Serial to definitely choke up. I think I can trim it to 256m, let me run it through jdk-submit. > 3. Can you disable core dumping as part of the test so we don't get large core files generated. Supplying -XX:-CreateCoredumpOnCrash is going to be enough, right? Did this: diff -r e4bf4aa1100c test/hotspot/jtreg/runtime/8181143/TestAbortOnVMOperationTimeout.java --- a/test/hotspot/jtreg/runtime/8181143/TestAbortOnVMOperationTimeout.java Tue Dec 11 22:45:38 2018 +0100 +++ b/test/hotspot/jtreg/runtime/8181143/TestAbortOnVMOperationTimeout.java Wed Dec 12 13:46:15 2018 +0100 @@ -27,5 +27,4 @@ /* * @test TestAbortOnVMOperationTimeout - * @bug 8181143 * @summary Check abort on VM timeout is working * @library /test/lib @@ -38,5 +37,5 @@ public static void main(String[] args) throws Exception { if (args.length > 0) { - Object[] arr = new Object[40_000_000]; + Object[] arr = new Object[10_000_000]; for (int i = 0; i < arr.length; i++) { arr[i] = new Object(); @@ -51,6 +50,6 @@ } - // These should fail: Serial is not very fast. Traversing 40M objects in 5 ms - // means less than 0.125 ns per object, which is not doable. + // These should fail: Serial is not very fast. Traversing 10M objects in 5 ms + // means less than 0.5 ns per object, which is not doable. for (int delay : new int[]{0, 1, 5}) { testWith(delay, false); @@ -63,6 +62,7 @@ "-XX:+AbortVMOnVMOperationTimeout", "-XX:AbortVMOnVMOperationTimeoutDelay=" + delay, - "-Xmx1g", + "-Xmx256m", "-XX:+UseSerialGC", + "-XX:-CreateCoredumpOnCrash", "TestAbortOnVMOperationTimeout", "foo" Full webrev: http://cr.openjdk.java.net/~shade/8181143/webrev.05/ -Aleksey From Pengfei.Li at arm.com Wed Dec 12 14:43:23 2018 From: Pengfei.Li at arm.com (Pengfei Li (Arm Technology China)) Date: Wed, 12 Dec 2018 14:43:23 +0000 Subject: 8212043: Request for help with Submit Failure In-Reply-To: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> References: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> Message-ID: Hi, My patch (JDK-8212043) modified hotspot shared code so it was pushed to the submit repo for test. http://hg.openjdk.java.net/jdk/submit/rev/b61efbb70833 Today I got below email describing that sparc build has 2 failures. As I don't have a sparc machine to reproduce, could any engineer from Oracle help look at this issue? -- Thanks, Pengfei ________________________________ Build Details: 2018-12-11-1622582.pengfei.li.source 0 Failed Tests Mach5 Tasks Results Summary * FAILED: 0 * EXECUTED_WITH_FAILURE: 2 * NA: 0 * KILLED: 0 * PASSED: 71 * UNABLE_TO_RUN: 3 Build 2 Executed with failure * solaris-sparcv9-solaris-sparcv9-build-9 error while building, return value: 2 * solaris-sparcv9-debug-solaris-sparcv9-build-10 error while building, return value: 2 1 Not run * solaris-sparcv9-install-solaris-sparcv9-build-17 Dependency task failed: mach5...14173-solaris-sparcv9-solaris-sparcv9-build-9 Test 2 Not run * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-58 Dependency task failed: mach5...14173-solaris-sparcv9-solaris-sparcv9-build-9 * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-59 Dependency task failed: mach5...olaris-sparcv9-debug-solaris-sparcv9-build-10 From tobias.hartmann at oracle.com Wed Dec 12 14:47:50 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 12 Dec 2018 15:47:50 +0100 Subject: 8212043: Request for help with Submit Failure In-Reply-To: References: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> Message-ID: <48685738-ad99-f48d-72a7-ff87d20cfea9@oracle.com> Hi Pengfei, the build on Solaris Sparc failed because: jib > * For target hotspot_variant-server_libjvm_objs_escape.o: jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 259: Error: INFINITY is not defined. jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 271: Error: INFINITY is not defined. jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 283: Error: INFINITY is not defined. jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 295: Error: INFINITY is not defined. jib > 4 Error(s) detected. jib > * For target hotspot_variant-server_libjvm_objs_g1BarrierSetC2.o: jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 259: Error: INFINITY is not defined. jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 271: Error: INFINITY is not defined. jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 283: Error: INFINITY is not defined. jib > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", line 295: Error: INFINITY is not defined. Best regards, Tobias On 12.12.18 15:43, Pengfei Li (Arm Technology China) wrote: > Hi, > > > My patch (JDK-8212043) modified hotspot shared code so it was pushed to the submit repo for test. > http://hg.openjdk.java.net/jdk/submit/rev/b61efbb70833 > > Today I got below email describing that sparc build has 2 failures. > As I don't have a sparc machine to reproduce, could any engineer from Oracle help look at this issue? > > -- > Thanks, > Pengfei > > ________________________________ > > Build Details: 2018-12-11-1622582.pengfei.li.source > 0 Failed Tests > Mach5 Tasks Results Summary > > * FAILED: 0 > * EXECUTED_WITH_FAILURE: 2 > * NA: 0 > * KILLED: 0 > * PASSED: 71 > * UNABLE_TO_RUN: 3 > Build > > 2 Executed with failure > * solaris-sparcv9-solaris-sparcv9-build-9 error while building, return value: 2 > * solaris-sparcv9-debug-solaris-sparcv9-build-10 error while building, return value: 2 > 1 Not run > * solaris-sparcv9-install-solaris-sparcv9-build-17 Dependency task failed: mach5...14173-solaris-sparcv9-solaris-sparcv9-build-9 > Test > > 2 Not run > * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-58 Dependency task failed: mach5...14173-solaris-sparcv9-solaris-sparcv9-build-9 > * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-59 Dependency task failed: mach5...olaris-sparcv9-debug-solaris-sparcv9-build-10 > From aph at redhat.com Wed Dec 12 15:30:02 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 12 Dec 2018 15:30:02 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> <754b4943-559f-31d7-c564-a1103ac6d50d@oracle.com> Message-ID: <81d7ce0c-c2f0-b6c7-476a-21731ffba293@redhat.com> On 12/11/18 9:59 AM, Yang Zhang (Arm Technology China) wrote: > Can this patch be merged in jdk12? Because jdk 12 ramp down phase is coming. It's pushed to jdk-submit. I'll let you know what happens. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From adinn at redhat.com Wed Dec 12 15:52:01 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 12 Dec 2018 15:52:01 +0000 Subject: 8212043: Request for help with Submit Failure In-Reply-To: <48685738-ad99-f48d-72a7-ff87d20cfea9@oracle.com> References: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> <48685738-ad99-f48d-72a7-ff87d20cfea9@oracle.com> Message-ID: On 12/12/2018 14:47, Tobias Hartmann wrote: > Hi Pengfei, > > the build on Solaris Sparc failed because: > > jib > * For target hotspot_variant-server_libjvm_objs_escape.o: > jib > > "/scratch/opt/mach5/mesos/work_dir/9098b9c0-92ff-43db-8cef-bdf92eac6f19/workspace/open/src/hotspot/share/opto/addnode.hpp", > line 259: Error: INFINITY is not defined. Thanks for checking up on this, Tobias. Pengfei, I think a portable way to define +/-INFINITY will be to use the values 1.0D/0.0D or -1.0D/0.0D. We might want to introduce some local defines for them. I'm just checking that this definition works on AArch64 (it ought to as it uses gcc) and will then resubmit to see if sparc also accepts it. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From aph at redhat.com Wed Dec 12 18:53:38 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 12 Dec 2018 18:53:38 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: <81d7ce0c-c2f0-b6c7-476a-21731ffba293@redhat.com> References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> <754b4943-559f-31d7-c564-a1103ac6d50d@oracle.com> <81d7ce0c-c2f0-b6c7-476a-21731ffba293@redhat.com> Message-ID: On 12/12/18 3:30 PM, Andrew Haley wrote: > On 12/11/18 9:59 AM, Yang Zhang (Arm Technology China) wrote: >> Can this patch be merged in jdk12? Because jdk 12 ramp down phase is coming. > > It's pushed to jdk-submit. I'll let you know what happens. It passed. I'll push it tomorrow. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From shade at redhat.com Wed Dec 12 20:40:34 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Dec 2018 21:40:34 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: <5feb62ad-32f1-da02-0216-ef69b32774bd@redhat.com> On 12/12/18 2:08 PM, Aleksey Shipilev wrote: > Full webrev: > http://cr.openjdk.java.net/~shade/8181143/webrev.05/ Thomas Stuefe, Robbin Ehn, would you like to ack the patch is fine with you? Meanwhile, jdk-submit returned. Seems clean, except for the intermittent infra problem? See: mach5-one-shade-JDK-8181143-20181212-1328-14309: FAILED tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-58 TimeoutException in CLEANUP. Thanks, -Aleksey From jcbeyler at google.com Wed Dec 12 21:47:37 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 13:47:37 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> References: <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> Message-ID: On Wed, Dec 12, 2018 at 11:37 AM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > Hi Jc, > > > On 12/11/18 21:16, JC Beyler wrote: > > Hi all, > > Here is the new webrev with the TEST.groups change. Serguei, let me know > if I convinced you with the static vs anonymous namespaces or if you'd > still rather have a "static" for now :-) > > > > What do you think about this post? : > > https://stackoverflow.com/questions/11623451/static-vs-non-static-variables-in-namespace > That is a different problem here because the namespace is not anonymous. Because it is not an anonymous namespace and because the variable is not declared static, the variable can be defined outside of the scope of the namespace by doing something like: namespace foo { extern int bar; } But if you declared the namespace as anonymous, that is no longer possible and you'd get an undefined reference. Also this is different because the stack overflow post was about putting this declaration in a .h and being included in two different files :-). Let me know if there are any other questions and if someone else could review the newest webrev, that would be awesome :-) Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 Thanks, Jc > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > > The update looks fine. > > Thanks, > Serguei > > > Thanks, > Serguei > > > Thanks again for the reviews! > Jc > > On Mon, Dec 10, 2018 at 3:10 PM JC Beyler wrote: > >> Hi Serguei, >> >> Yes basically it is equivalent :) I can put them in but they are not >> required. The norm actually wanted to deprecate it but then remembered that >> C compatibility would require the static key-word for this case [1] >> >> So, really, they are not required here and will amount to the same thing: >> only that file can refer to them and you cannot get to them without a >> globally available method to return a pointer to them (ie same as a static >> variable in C). >> >> I can put static if it makes it easier to see but, by being in an >> anonymous namespace they are only available for the file's translation >> unit. For example: >> >> $ cat main.cpp >> >> int totally_global; >> static int explictly_static; >> >> namespace { >> int implicitly_static; >> } >> >> void foo(); >> int main() { >> foo(); >> } >> >> $ g++ -O3 main.cpp -c >> $ nm main.o >> U _GLOBAL_OFFSET_TABLE_ >> 0000000000000000 T main >> 0000000000000000 B totally_global >> U _Z3foov >> >> As you can see, the static and anonymous namespace variables are not in >> the file due to not being used. If you were to use them, you'd see them >> show up as something like: >> 0000000000000008 b _ZL17explicitly_static >> 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE >> >> Where again, it shows that it is mangling the names so that no external >> usage can happen without tinkering. >> >> Hopefully that helps :-), >> Jc >> >> [1] http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 >> >> >> On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com < >> serguei.spitsyn at oracle.com> wrote: >> >>> Hi Jc, >>> >>> I had little experience with the C++ namespaces. >>> My understanding is that static in this context should mean internal >>> linkage. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 12/10/18 13:57, JC Beyler wrote: >>> >>> Hi Serguei, >>> >>> The variables and functions are in a anonymous namespace; my >>> understanding of C++ is that this is equivalent to putting it as >>> static.Hence, I didn't add them there. Does that make sense? >>> >>> Thanks! >>> Jc >>> >>> On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com < >>> serguei.spitsyn at oracle.com> wrote: >>> >>>> Hi Jc, >>>> >>>> It looks good in general. >>>> One question though. >>>> >>>> >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>>> >>>> I wonder if the variables and functions have to be static. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 12/5/18 11:36, JC Beyler wrote: >>>> >>>> Hi all, >>>> >>>> My apologies to having to come back for another review for this change: >>>> I ran into a snag when trying to pull the latest changes compared to the >>>> base I was working on. I basically forgot that there was an issue with >>>> snprintf and that I had solved it via JDK-8213622. >>>> >>>> Could I have a new review of this webrev: >>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> Incremental from the port of webrev.03 that got LGTMs: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ >>>> >>>> A few comments on this because it took me a while to get things in a >>>> state I thought was good: >>>> - I had to implement an itoa method, do we have something like that >>>> in the test base (remember that JDK-8213622 could not use sprintf due to >>>> being in the test code)? >>>> >>>> - The differences here compared to the one you all reviewed are: >>>> - I found that adding to the strlen/memcpy error prone and >>>> thought that I would try to make it less so. If you want to compare, I >>>> extended the strlen/memcpy with the new format to show you if you prefer [1] >>>> - Note that the diff between the "old extended way from >>>> [1]" to the webrev.04 can be found in [2] >>>> >>>> - I added a test to test the exception wrapper in tests :); I'm >>>> not sure it is deemed useful or not but helped me assure myself that I was >>>> not doing things wrong; you can find the base test file here [3]; should we >>>> have this or not? (I know that normally we don't add tests to vmTestbase >>>> but thought this might be an exception) >>>> >>>> Thanks for your help and my apologies for the snag, >>>> Jc >>>> >>>> [1]: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>> [2]: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 >>>> [3] >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>>> >>>> On Mon, Dec 3, 2018 at 11:29 PM David Holmes >>>> wrote: >>>> >>>>> Looks fine to me. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 4/12/2018 4:04 pm, JC Beyler wrote: >>>>> > Hi both, >>>>> > >>>>> > Thanks for the reviews! Since Serguei did not insist on >>>>> get_basename, I >>>>> > went for get_dirname since the method is a local static method and >>>>> won't >>>>> > have its name start spreading, I think it's ok too. >>>>> > >>>>> > For the naming of the local variable, the idea initially was to use >>>>> the >>>>> > same name as the local variable for JNIEnv already used to reduce >>>>> the >>>>> > code change. Since I'm now adding the line macro at the end anyway, >>>>> this >>>>> > does not matter anymore so I converged all local variables to "jni". >>>>> > >>>>> > So, without further ado, here is the new version: >>>>> > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >>>>> > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> > >>>>> > This passes the various tests changed by the webrev on my dev >>>>> machine. >>>>> > >>>>> > Let me know what you think, >>>>> > Jc >>>>> > >>>>> > On Mon, Dec 3, 2018 at 8:40 PM serguei.spitsyn at oracle.com >>>>> > >>>> > > wrote: >>>>> > >>>>> > On 12/3/18 20:15, Chris Plummer wrote: >>>>> > > Hi JC, >>>>> > > >>>>> > > Overall it looks good. A few naming nits thought: >>>>> > > >>>>> > > In bi01t001.cpp, why have you declared the >>>>> > ExceptionCheckingJniEnvPtr >>>>> > > using jni_env(jni). Elsewhere you use jni(jni_env) and rename >>>>> the >>>>> > > method argument passed in from jni to jni_env. >>>>> > > >>>>> > > Related to this, I also noticed in some files that already >>>>> are using >>>>> > > ExceptionCheckingJniEnvPtr, such as >>>>> CharArrayCriticalLocker.cpp, you >>>>> > > delcared it as env(jni_env). So that means there are 3 >>>>> different >>>>> > names >>>>> > > you have used for the ExceptionCheckingJniEnvPtr local >>>>> variable. >>>>> > They >>>>> > > should be consistent. >>>>> > > >>>>> > > Also, can you rename get_basename() to get_dirname()? I know >>>>> Serguei >>>>> > > suggested get_basename() a while back, but unless "basename" >>>>> is >>>>> > > commonly used for this purpose, I think "dirname" is more self >>>>> > > explanatory. >>>>> > >>>>> > In general, I'm Okay with get_dirname(). >>>>> > Just to mention dirname can be both short or full, so it is a >>>>> little >>>>> > confusing as well. >>>>> > It is the reason why the get_basename() was suggested. >>>>> > However, I do not insist on get_basename() nor >>>>> get_full_dirname(). :) >>>>> > >>>>> > Thanks, >>>>> > Serguei >>>>> > >>>>> > >>>>> > > thanks, >>>>> > > >>>>> > > Chris >>>>> > > >>>>> > > On 12/2/18 10:29 PM, David Holmes wrote: >>>>> > >> Hi Jc, >>>>> > >> >>>>> > >> I've been lurking on this one and have had a look through. >>>>> I'm okay >>>>> > >> with the FatalError approach for the tests - we don't expect >>>>> > anything >>>>> > >> to go wrong in a well written test in a correctly >>>>> functioning VM. >>>>> > >> >>>>> > >> Thanks, >>>>> > >> David >>>>> > >> >>>>> > >> >>>>> > >> >>>>> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: >>>>> > >>> Hi all, >>>>> > >>> >>>>> > >>> Would someone on the GC or runtime team be motivated to give >>>>> > this a >>>>> > >>> review? :) >>>>> > >>> >>>>> > >>> It would be much appreciated! >>>>> > >>> >>>>> > >>> Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> > >>> >>>>> > >>> Thanks for your help, >>>>> > >>> Jc >>>>> > >>> >>>>> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler < >>>>> jcbeyler at google.com >>>>> > >>>>> > >>> >> >>>>> wrote: >>>>> > >>> >>>>> > >>> Hi Chris, >>>>> > >>> >>>>> > >>> Yes I was waiting for another review since you had >>>>> explicitly >>>>> > >>> asked :) >>>>> > >>> >>>>> > >>> And sounds good that when someone from GC or runtime >>>>> gives a >>>>> > >>> review, >>>>> > >>> I'll wait for your full review on the webrev.02! >>>>> > >>> >>>>> > >>> Thanks again for your help, >>>>> > >>> Jc >>>>> > >>> >>>>> > >>> >>>>> > >>> On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer >>>>> > >>> >>>> chris.plummer at oracle.com> >>>>> > >>>> chris.plummer at oracle.com>>> >>>>> > wrote: >>>>> > >>> >>>>> > >>> Hi JC, >>>>> > >>> >>>>> > >>> I think it would be good to get a review from the >>>>> gc or >>>>> > runtime >>>>> > >>> teams, since this also affects their tests. >>>>> > >>> >>>>> > >>> Also, once we are settled on this FatalError >>>>> approach, >>>>> > I still >>>>> > >>> need to give your webrev-02 a full review. I only >>>>> > skimmed over >>>>> > >>> parts of it (I did look at all the changes in >>>>> webrevo-01). >>>>> > >>> >>>>> > >>> thanks, >>>>> > >>> >>>>> > >>> Chris >>>>> > >>> >>>>> > >>> On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com >>>>> > >>>>> > >>> >>>> > > wrote: >>>>> > >>>> Hi Jc, >>>>> > >>>> >>>>> > >>>> I've already reviewed this too. >>>>> > >>>> >>>>> > >>>> Thanks, >>>>> > >>>> Serguei >>>>> > >>>> >>>>> > >>>> >>>>> > >>>> On 11/27/18 06:56, JC Beyler wrote: >>>>> > >>>>> Thanks Chris, >>>>> > >>>>> >>>>> > >>>>> Anybody else motivated to look at this and review >>>>> it? :) >>>>> > >>>>> Jc >>>>> > >>>>> >>>>> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer >>>>> > >>>>> >>>> > >>>> chris.plummer at oracle.com >>>>> > >> >>>>> > >>>>> wrote: >>>>> > >>>>> >>>>> > >>>>> Hi JC, >>>>> > >>>>> >>>>> > >>>>> I'm ok with the FatalError approach, but would >>>>> > like to >>>>> > >>>>> hear opinions from others also. >>>>> > >>>>> >>>>> > >>>>> thanks, >>>>> > >>>>> >>>>> > >>>>> Chris >>>>> > >>>>> >>>>> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >>>>> > >>>>>> Hi Chris, >>>>> > >>>>>> >>>>> > >>>>>> Thanks for taking the time to look at it and >>>>> yes you >>>>> > >>>>>> have raised exactly why the webrev is >>>>> between two >>>>> > >>>>>> worlds: in cases where a fatal error on >>>>> failure is >>>>> > >>>>>> wanted, should we simplify the code to remove >>>>> > the return >>>>> > >>>>>> tests since we do them internally? Now that >>>>> I've >>>>> > looked >>>>> > >>>>>> around for non-fatal cases, I think the >>>>> answer >>>>> > is yes, >>>>> > >>>>>> it simplifies the code while maintaining the >>>>> checks. >>>>> > >>>>>> >>>>> > >>>>>> I looked a bit and it seems that I can't find >>>>> > easily a >>>>> > >>>>>> case where the test accepts a JNI failure to >>>>> > then move >>>>> > >>>>>> on. Therefore, perhaps, for now, the fail >>>>> with a >>>>> > Fatal >>>>> > >>>>>> is enough and we can work on the tests to >>>>> clean >>>>> > them up? >>>>> > >>>>>> >>>>> > >>>>>> That means that this is the new webrev with >>>>> only >>>>> > Fatal >>>>> > >>>>>> and cleans up the tests so that it is no >>>>> longer in >>>>> > >>>>>> between two worlds: >>>>> > >>>>>> >>>>> > >>>>>> Webrev: >>>>> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>> > >>>>>> < >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/> >>>>> > >>>>>> Bug: >>>>> > https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> > >>>>>> >>>>> > >>>>>> (This passes testing on my dev machine for >>>>> all the >>>>> > >>>>>> modified tests) >>>>> > >>>>>> >>>>> > >>>>>> with the example you provided, it now looks >>>>> like: >>>>> > >>>>>> >>>>> > >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> > >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > < >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> > >>>>> > >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>>> Where it does, to me at least, seem cleaner >>>>> and less >>>>> > >>>>>> "noisy". >>>>> > >>>>>> >>>>> > >>>>>> Let me know what you think, >>>>> > >>>>>> Jc >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>>> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >>>>> > >>>>>> >>>> > >>>>> > >>>>>> >>>> > >> wrote: >>>>> > >>>>>> >>>>> > >>>>>> Hi JC, >>>>> > >>>>>> >>>>> > >>>>>> Sorry about the delay. I had to go back >>>>> an >>>>> > look at >>>>> > >>>>>> the initial 8210842 webrev and RFR >>>>> thread to see >>>>> > >>>>>> what this was initially all about. >>>>> > >>>>>> >>>>> > >>>>>> In general the changes look good. >>>>> > >>>>>> >>>>> > >>>>>> I don't have a good answer to your >>>>> > >>>>>> FatalError/NonFatalError question. It >>>>> makes >>>>> > the code >>>>> > >>>>>> a lot cleaner to use FatalError, but >>>>> then it >>>>> > is a >>>>> > >>>>>> behavior change, and you also need to >>>>> deal with >>>>> > >>>>>> tests that intentionally induce errors >>>>> (do >>>>> > you have >>>>> > >>>>>> an example of that). >>>>> > >>>>>> >>>>> > >>>>>> In any case, right now your webrev seems >>>>> to be >>>>> > >>>>>> between two worlds. You are producing >>>>> > FatalError, >>>>> > >>>>>> but still checking results. Here's a good >>>>> > example: >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> > >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > < >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> > >>>>> > >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>>> I'm not sure if this is just a temporary >>>>> > state until >>>>> > >>>>>> it was decided which approach to take. >>>>> > >>>>>> >>>>> > >>>>>> thanks, >>>>> > >>>>>> >>>>> > >>>>>> Chris >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>>> On 11/20/18 2:14 PM, JC Beyler wrote: >>>>> > >>>>>>> Hi all, >>>>> > >>>>>>> >>>>> > >>>>>>> Chris thought it made sense to have more >>>>> > eyes on >>>>> > >>>>>>> this change than just serviceability as >>>>> it will >>>>> > >>>>>>> modify to tests that are not only >>>>> > serviceability >>>>> > >>>>>>> tests so I've moved this to conversation >>>>> > here :) >>>>> > >>>>>>> >>>>> > >>>>>>> For convenience, I've copy-pasted the >>>>> > initial RFR: >>>>> > >>>>>>> >>>>> > >>>>>>> Could I have a review for the extension >>>>> and >>>>> > usage >>>>> > >>>>>>> of the ExceptionJniWrapper. This adds >>>>> lines and >>>>> > >>>>>>> filenames to the end of the wrapper JNI >>>>> > methods, >>>>> > >>>>>>> adds tracing, and throws an error if >>>>> need >>>>> > be. I've >>>>> > >>>>>>> ported the gc/lock files to use the new >>>>> > >>>>>>> TRACE_JNI_CALL add-on and I've ported a >>>>> few >>>>> > of the >>>>> > >>>>>>> tests that were already changed for the >>>>> > assignment >>>>> > >>>>>>> webrev for JDK-8212884. >>>>> > >>>>>>> >>>>> > >>>>>>> Webrev: >>>>> > >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>>> > >>>>>>> < >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01> >>>>> > >>>>>>> Bug: >>>>> > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> > >>>>>>> >>>>> > >>>>>>> For illustration, if I force an error >>>>> to the >>>>> > >>>>>>> AP04/ap04t03 test and set the verbosity >>>>> on, >>>>> > I get >>>>> > >>>>>>> something like: >>>>> > >>>>>>> >>>>> > >>>>>>> >> Calling JNI method FindClass from >>>>> > >>>>>>> ap04t003.cpp:343 >>>>> > >>>>>>> >> Calling with these parameter(s): >>>>> > >>>>>>> java/lang/Threadd >>>>> > >>>>>>> Wait for thread to finish >>>>> > >>>>>>> << Called JNI method FindClass from >>>>> > >>>>>>> ap04t003.cpp:343 >>>>> > >>>>>>> Exception in thread "Thread-0" >>>>> > >>>>>>> java.lang.NoClassDefFoundError: >>>>> > java/lang/Threadd >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> > >>>>>>> >>>>> > >>>>>>> Method) >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> > >>>>> > >>>>>>> >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> > >>>>> > >>>>>>> >>>>> > >>>>>>> Caused by: >>>>> java.lang.ClassNotFoundException: >>>>> > >>>>>>> java.lang.Threadd >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>> > >>>>> > >>>>>>> >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>> > >>>>> > >>>>>>> >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>> > >>>>>>> ... 3 more >>>>> > >>>>>>> FATAL ERROR in native method: JNI method >>>>> > FindClass >>>>> > >>>>>>> : internal error from ap04t003.cpp:343 >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> > >>>>>>> >>>>> > >>>>>>> Method) >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> > >>>>> > >>>>>>> >>>>> > >>>>>>> at >>>>> > >>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> > >>>>> > >>>>>>> >>>>> > >>>>>>> >>>>> > >>>>>>> Questions/comments I have about this >>>>> are: >>>>> > >>>>>>> - Do we want to force fatal errors >>>>> when a JNI >>>>> > >>>>>>> call fails in general? Most of these >>>>> tests >>>>> > do the >>>>> > >>>>>>> right thing and test the return of the >>>>> JNI >>>>> > calls, >>>>> > >>>>>>> for example: >>>>> > >>>>>>> thrClass = >>>>> > jni->FindClass("java/lang/Threadd", >>>>> > >>>>>>> TRACE_JNI_CALL); >>>>> > >>>>>>> if (thrClass == NULL) { >>>>> > >>>>>>> >>>>> > >>>>>>> but now the wrapper actually would do a >>>>> > fatal if >>>>> > >>>>>>> the FindClass call would return a >>>>> nullptr, >>>>> > so we >>>>> > >>>>>>> could remove that test altogether. What >>>>> do you >>>>> > >>>>>>> think? >>>>> > >>>>>>> - I prefer to leave them as the >>>>> tests then >>>>> > >>>>>>> become closer to what real users would >>>>> have in >>>>> > >>>>>>> their code and is the "recommended" way >>>>> of >>>>> > doing it >>>>> > >>>>>>> >>>>> > >>>>>>> - The alternative is to use the >>>>> > NonFatalError I >>>>> > >>>>>>> added which then just prints out that >>>>> something >>>>> > >>>>>>> went wrong, letting the test continue. >>>>> Question >>>>> > >>>>>>> will be what should be the default? The >>>>> > fatal or >>>>> > >>>>>>> the non-fatal error handling? >>>>> > >>>>>>> >>>>> > >>>>>>> On a different subject: >>>>> > >>>>>>> - On the new tests, I've removed the >>>>> > >>>>>>> NSK_JNI_VERIFY since the JNI wrapper >>>>> > handles the >>>>> > >>>>>>> tracing and the verify in almost the >>>>> same >>>>> > way; only >>>>> > >>>>>>> difference I can really tell is that the >>>>> > complain >>>>> > >>>>>>> method from NSK has a max complain >>>>> before >>>>> > stopping >>>>> > >>>>>>> to "complain"; I have not added that >>>>> part >>>>> > of the >>>>> > >>>>>>> code in this webrev >>>>> > >>>>>>> >>>>> > >>>>>>> Once we decide on these, I can continue >>>>> on the >>>>> > >>>>>>> files from JDK-8212884 and then do both >>>>> the >>>>> > >>>>>>> assignment in an if extraction >>>>> followed-by this >>>>> > >>>>>>> type of webrev in an easier fashion. >>>>> > Depending on >>>>> > >>>>>>> decisions here, NSK*VERIFY can be >>>>> deprecated as >>>>> > >>>>>>> well as we go forward. >>>>> > >>>>>>> >>>>> > >>>>>>> Thanks! >>>>> > >>>>>>> Jc >>>>> > >>>>>>> >>>>> > >>>>>>> On Mon, Nov 19, 2018 at 11:34 AM Chris >>>>> Plummer >>>>> > >>>>>>> >>>> > >>>>> > >>>>>>> >>>> > >> wrote: >>>>> > >>>>>>> >>>>> > >>>>>>> On 11/19/18 10:07 AM, JC Beyler >>>>> wrote: >>>>> > >>>>>>>> Hi all, >>>>> > >>>>>>>> >>>>> > >>>>>>>> @David/Chris: should I then push >>>>> this >>>>> > RFR to >>>>> > >>>>>>>> the hotspot mailing or the runtime >>>>> > one? For >>>>> > >>>>>>>> what it's worth, a lot of the tests >>>>> > under the >>>>> > >>>>>>>> vmTestbase are jvmti so the review >>>>> also >>>>> > >>>>>>>> affects serviceability; it just >>>>> turns >>>>> > out I >>>>> > >>>>>>>> started with the GC originally and >>>>> > then hit >>>>> > >>>>>>>> some other tests I had touched via >>>>> the >>>>> > >>>>>>>> assignment extraction. >>>>> > >>>>>>> I think hotspot would be best. >>>>> > >>>>>>> >>>>> > >>>>>>> Chris >>>>> > >>>>>>>> >>>>> > >>>>>>>> @Serguei: Done for the method >>>>> > renaming, for >>>>> > >>>>>>>> the indent, are you talking about >>>>> > going from >>>>> > >>>>>>>> the 8-indent to 4-indent? If so, >>>>> would >>>>> > it not >>>>> > >>>>>>>> just be better to do a new JBS bug >>>>> and >>>>> > do the >>>>> > >>>>>>>> whole files in one go? I ask >>>>> because >>>>> > >>>>>>>> otherwise, it will look a bit >>>>> weird to >>>>> > have >>>>> > >>>>>>>> parts of the file as 8-indent and >>>>> others >>>>> > >>>>>>>> 4-indent? >>>>> > >>>>>>>> >>>>> > >>>>>>>> Thanks for looking at it! >>>>> > >>>>>>>> Jc >>>>> > >>>>>>>> >>>>> > >>>>>>>> On Mon, Nov 19, 2018 at 1:25 AM >>>>> > >>>>>>>> serguei.spitsyn at oracle.com >>>> serguei.spitsyn at oracle.com> >>>>> > >>>>>>>> >>>> > > >>>>> > >>>>>>>> >>>> > >>>>> > >>>>>>>> >>>> > >> wrote: >>>>> > >>>>>>>> >>>>> > >>>>>>>> Hi Jc, >>>>> > >>>>>>>> >>>>> > >>>>>>>> We have to start this review >>>>> > anyway. :) >>>>> > >>>>>>>> It looks good to me in general. >>>>> > >>>>>>>> Thank you for your consistency >>>>> in this >>>>> > >>>>>>>> refactoring! >>>>> > >>>>>>>> >>>>> > >>>>>>>> Some minor comments. >>>>> > >>>>>>>> >>>>> > >>>>>>>> >>>>> > >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>> > >>>>> > >>>>>>>> >>>>> > >>>>>>>> +static const char* >>>>> > remove_folders(const >>>>> > >>>>>>>> char* fullname) { I'd suggest >>>>> to >>>>> > rename >>>>> > >>>>>>>> the function name to something >>>>> > traditional >>>>> > >>>>>>>> like get_basename. Otherwise, >>>>> it >>>>> > sounds >>>>> > >>>>>>>> like this function has to >>>>> really >>>>> > remove >>>>> > >>>>>>>> folders. :) Also, all >>>>> *Locker.cpp have >>>>> > >>>>>>>> wrong indent in the bodies of >>>>> if >>>>> > and while >>>>> > >>>>>>>> statements. Could this be fixed >>>>> > with the >>>>> > >>>>>>>> refactoring? I did not look on >>>>> how >>>>> > this >>>>> > >>>>>>>> impacts the tests other than >>>>> > >>>>>>>> serviceability. Thanks, Serguei >>>>> > >>>>>>>> >>>>> > >>>>>>>> >>>>> > >>>>>>>> On 11/16/18 19:43, JC Beyler >>>>> wrote: >>>>> > >>>>>>>>> Hi all, >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Anybody motivated to review >>>>> this? :) >>>>> > >>>>>>>>> Jc >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> On Wed, Nov 7, 2018 at 9:53 >>>>> PM JC >>>>> > Beyler >>>>> > >>>>>>>>> >>>> > >>>>> > >>>>>>>>> >>>> > >> wrote: >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Hi all, >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Could I have a review for >>>>> the >>>>> > >>>>>>>>> extension and usage of the >>>>> > >>>>>>>>> ExceptionJniWrapper. This >>>>> > adds lines >>>>> > >>>>>>>>> and filenames to the end >>>>> of the >>>>> > >>>>>>>>> wrapper JNI methods, adds >>>>> > tracing, >>>>> > >>>>>>>>> and throws an error if >>>>> need >>>>> > be. I've >>>>> > >>>>>>>>> ported the gc/lock files >>>>> to >>>>> > use the >>>>> > >>>>>>>>> new TRACE_JNI_CALL add-on >>>>> and >>>>> > I've >>>>> > >>>>>>>>> ported a few of the tests >>>>> > that were >>>>> > >>>>>>>>> already changed for the >>>>> > assignment >>>>> > >>>>>>>>> webrev for JDK-8212884. >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Webrev: >>>>> > >>>>>>>>> >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>> > >>>>>>>>> < >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/> >>>>> > >>>>>>>>> Bug: >>>>> > >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> For illustration, if I >>>>> force >>>>> > an error >>>>> > >>>>>>>>> to the AP04/ap04t03 test >>>>> and >>>>> > set the >>>>> > >>>>>>>>> verbosity on, I get >>>>> something >>>>> > like: >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> >> Calling JNI method >>>>> > FindClass from >>>>> > >>>>>>>>> ap04t003.cpp:343 >>>>> > >>>>>>>>> >> Calling with these >>>>> > parameter(s): >>>>> > >>>>>>>>> java/lang/Threadd >>>>> > >>>>>>>>> Wait for thread to finish >>>>> > >>>>>>>>> << Called JNI method >>>>> > FindClass from >>>>> > >>>>>>>>> ap04t003.cpp:343 >>>>> > >>>>>>>>> Exception in thread >>>>> "Thread-0" >>>>> > >>>>>>>>> java.lang.NoClassDefFoundError: >>>>> > >>>>>>>>> java/lang/Threadd >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Method) >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> > >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> > >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Caused by: >>>>> > >>>>>>>>> java.lang.ClassNotFoundException: >>>>> > >>>>>>>>> java.lang.Threadd >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>> > >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>> > >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>> > >>>>>>>>> ... 3 more >>>>> > >>>>>>>>> FATAL ERROR in native >>>>> method: JNI >>>>> > >>>>>>>>> method FindClass : >>>>> internal error >>>>> > >>>>>>>>> from ap04t003.cpp:343 >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Method) >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> > >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> at >>>>> > >>>>>>>>> >>>>> > >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> > >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Questions/comments I have >>>>> about >>>>> > >>>>>>>>> this are: >>>>> > >>>>>>>>> - Do we want to force >>>>> fatal >>>>> > errors >>>>> > >>>>>>>>> when a JNI call fails in >>>>> general? >>>>> > >>>>>>>>> Most of these tests do >>>>> the right >>>>> > >>>>>>>>> thing and test the return >>>>> of >>>>> > the JNI >>>>> > >>>>>>>>> calls, for example: >>>>> > >>>>>>>>> thrClass = >>>>> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >>>>> > >>>>>>>>> TRACE_JNI_CALL); >>>>> > >>>>>>>>> if (thrClass == NULL) >>>>> { >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> but now the wrapper >>>>> actually >>>>> > would do >>>>> > >>>>>>>>> a fatal if the FindClass >>>>> call >>>>> > would >>>>> > >>>>>>>>> return a nullptr, so we >>>>> could >>>>> > remove >>>>> > >>>>>>>>> that test altogether. >>>>> What do >>>>> > you >>>>> > >>>>>>>>> think? >>>>> > >>>>>>>>> - I prefer to leave >>>>> them >>>>> > as the >>>>> > >>>>>>>>> tests then become closer >>>>> to >>>>> > what real >>>>> > >>>>>>>>> users would have in their >>>>> > code and is >>>>> > >>>>>>>>> the "recommended" way of >>>>> doing it >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> - The alternative is to >>>>> > use the >>>>> > >>>>>>>>> NonFatalError I added >>>>> which >>>>> > then just >>>>> > >>>>>>>>> prints out that something >>>>> > went wrong, >>>>> > >>>>>>>>> letting the test continue. >>>>> > Question >>>>> > >>>>>>>>> will be what should be the >>>>> > default? >>>>> > >>>>>>>>> The fatal or the >>>>> non-fatal error >>>>> > >>>>>>>>> handling? >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> On a different subject: >>>>> > >>>>>>>>> - On the new tests, I've >>>>> > removed >>>>> > >>>>>>>>> the NSK_JNI_VERIFY since >>>>> the JNI >>>>> > >>>>>>>>> wrapper handles the >>>>> tracing >>>>> > and the >>>>> > >>>>>>>>> verify in almost the same >>>>> > way; only >>>>> > >>>>>>>>> difference I can really >>>>> tell >>>>> > is that >>>>> > >>>>>>>>> the complain method from >>>>> NSK >>>>> > has a >>>>> > >>>>>>>>> max complain before >>>>> stopping to >>>>> > >>>>>>>>> "complain"; I have not >>>>> added that >>>>> > >>>>>>>>> part of the code in this >>>>> webrev >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Once we decide on these, >>>>> I can >>>>> > >>>>>>>>> continue on the files from >>>>> > >>>>>>>>> JDK-8212884 and then do >>>>> both the >>>>> > >>>>>>>>> assignment in an if >>>>> extraction >>>>> > >>>>>>>>> followed-by this type of >>>>> > webrev in an >>>>> > >>>>>>>>> easier fashion. Depending >>>>> on >>>>> > >>>>>>>>> decisions here, >>>>> NSK*VERIFY can be >>>>> > >>>>>>>>> deprecated as well as we >>>>> go >>>>> > forward. >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> Thank you for the >>>>> > reviews/comments :) >>>>> > >>>>>>>>> Jc >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> >>>>> > >>>>>>>>> -- >>>>> > >>>>>>>>> Thanks, >>>>> > >>>>>>>>> Jc >>>>> > >>>>>>>> >>>>> > >>>>>>>> >>>>> > >>>>>>>> >>>>> > >>>>>>>> -- >>>>> > >>>>>>>> Thanks, >>>>> > >>>>>>>> Jc >>>>> > >>>>>>> >>>>> > >>>>>>> >>>>> > >>>>>>> >>>>> > >>>>>>> >>>>> > >>>>>>> -- >>>>> > >>>>>>> Thanks, >>>>> > >>>>>>> Jc >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>>> >>>>> > >>>>>> -- >>>>> > >>>>>> Thanks, >>>>> > >>>>>> Jc >>>>> > >>>>> >>>>> > >>>>> >>>>> > >>>>> >>>>> > >>>>> >>>>> > >>>>> -- >>>>> > >>>>> Thanks, >>>>> > >>>>> Jc >>>>> > >>>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> -- >>>>> > >>> Thanks, >>>>> > >>> Jc >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> -- >>>>> > >>> >>>>> > >>> Thanks, >>>>> > >>> Jc >>>>> > > >>>>> > > >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > >>>>> > Thanks, >>>>> > Jc >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>>> >>>> >>>> >>> >>> -- >>> >>> Thanks, >>> Jc >>> >>> >>> >> >> -- >> >> Thanks, >> Jc >> > > > -- > > Thanks, > Jc > > > -- Thanks, Jc From david.holmes at oracle.com Wed Dec 12 21:52:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Dec 2018 07:52:01 +1000 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> References: <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> Message-ID: <839301b7-c247-df3b-e485-283e8bb7388b@oracle.com> FWIW I think namespaces are overkill in all of this test code and just obfuscates things - the declaration is easily missed. A static variable in a .cpp is clearly a global variable to the file. Cheers, David On 13/12/2018 5:37 am, serguei.spitsyn at oracle.com wrote: > Hi Jc, > > > On 12/11/18 21:16, JC Beyler wrote: >> Hi all, >> >> Here is the new webrev with the TEST.groups change. Serguei, let me >> know if I convinced you with the static vs anonymous namespaces or if >> you'd still rather have a "static" for now :-) > > > What do you think about this post? : > https://stackoverflow.com/questions/11623451/static-vs-non-static-variables-in-namespace > >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > The update looks fine. > > Thanks, > Serguei > > > Thanks, > Serguei > >> >> Thanks again for the reviews! >> Jc >> >> On Mon, Dec 10, 2018 at 3:10 PM JC Beyler > > wrote: >> >> Hi Serguei, >> >> Yes basically it is equivalent :) I can put them in but they are >> not required. The norm actually wanted to deprecate it but then >> remembered that C compatibility would require the static key-word >> for this case [1] >> >> So, really, they are not required here and will amount to the same >> thing: only that file can refer to them and you cannot get to them >> without a globally available method to return a pointer to them >> (ie same as a static variable in C). >> >> I can put static if it makes it easier to see but, by being in an >> anonymous namespace they are only available for the file's >> translation unit. For example: >> >> $ cat main.cpp >> >> int totally_global; >> static int explictly_static; >> >> namespace { >> int implicitly_static; >> } >> >> void foo(); >> int main() { >> ? foo(); >> } >> >> $ g++ -O3 main.cpp -c >> $ nm main.o >> ? ? ? ? ? ? ? ? ?U _GLOBAL_OFFSET_TABLE_ >> 0000000000000000 T main >> 0000000000000000 B totally_global >> ? ? ? ? ? ? ? ? ?U _Z3foov >> >> As you can see, the static and anonymous namespace variables are >> not in the file due to not being used. If you were to use them, >> you'd see them show up as something like: >> 0000000000000008 b _ZL17explicitly_static >> 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE >> >> Where again, it shows that it is mangling the names so that no >> external usage can happen without tinkering. >> >> Hopefully that helps :-), >> Jc >> >> [1] http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 >> >> >> On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com >> > > wrote: >> >> Hi Jc, >> >> I had little experience with the C++ namespaces. >> My understanding is that static in this context should mean >> internal linkage. >> >> Thanks, >> Serguei >> >> >> On 12/10/18 13:57, JC Beyler wrote: >>> Hi Serguei, >>> >>> The variables and functions are in a anonymous namespace; my >>> understanding of C++ is that this is equivalent to putting it >>> as static.Hence, I didn't add them there. Does that make sense? >>> >>> Thanks! >>> Jc >>> >>> On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com >>> >>> >> > wrote: >>> >>> Hi Jc, >>> >>> It looks good in general. >>> One question though. >>> >>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>> >>> I wonder if the variables and functions have to be static. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 12/5/18 11:36, JC Beyler wrote: >>>> Hi all, >>>> >>>> My apologies to having to come back for another review >>>> for this change: I ran into a snag when trying to pull >>>> the latest changes compared to the base I was working >>>> on. I basically forgot that there was an issue with >>>> snprintf and that I had solved it via?JDK-8213622. >>>> >>>> Could I have a new review of this webrev: >>>> Webrev: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> Incremental from the port of webrev.03 that got LGTMs: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ >>>> >>>> >>>> A few comments on this because it took me a while to get >>>> things in a state I thought was good: >>>> ? - I had to implement an itoa method, do we have >>>> something like that in the test base (remember that >>>> JDK-8213622 could not use sprintf due to being in the >>>> test code)? >>>> >>>> ? - The differences here compared to the one you all >>>> reviewed are: >>>> ? ? ? - I found that adding to the strlen/memcpy error >>>> prone and thought that I would try to make it less so. >>>> If you want to compare, I extended the strlen/memcpy >>>> with the new format to show you if you prefer [1] >>>> ? ? ? ? ? ? - Note that the diff between the "old >>>> extended way from [1]" to the webrev.04 can be found in [2] >>>> >>>> ? ? ?- I added a test to test the exception wrapper in >>>> tests :); I'm not sure it is deemed useful or not but >>>> helped me assure myself that I was not doing things >>>> wrong; you can find the base test file here [3]; should >>>> we have this or not? (I know that normally we don't add >>>> tests to vmTestbase but thought this might be an exception) >>>> >>>> Thanks for your help and my apologies for the snag, >>>> Jc >>>> >>>> [1]: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>> >>>> [2]: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 >>>> >>>> [3] >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>>> >>>> >>>> On Mon, Dec 3, 2018 at 11:29 PM David Holmes >>>> >>> > wrote: >>>> >>>> Looks fine to me. >>>> >>>> Thanks, >>>> David >>>> >>>> On 4/12/2018 4:04 pm, JC Beyler wrote: >>>> > Hi both, >>>> > >>>> > Thanks for the reviews! Since Serguei did not >>>> insist on get_basename, I >>>> > went for get_dirname since the method is a local >>>> static method and won't >>>> > have its name start spreading, I think it's ok too. >>>> > >>>> > For the naming of the local variable, the idea >>>> initially was to use the >>>> > same name as the local variable for JNIEnv already >>>> used to reduce the >>>> > code change. Since I'm now adding the line macro >>>> at the end anyway, this >>>> > does not matter anymore so I converged all local >>>> variables to "jni". >>>> > >>>> > So, without further ado, here is the new version: >>>> > Webrev: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >>>> >>>> > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> > >>>> > This passes the various tests changed by the >>>> webrev on my dev machine. >>>> > >>>> > Let me know what you think, >>>> > Jc >>>> > >>>> > On Mon, Dec 3, 2018 at 8:40 PM >>>> serguei.spitsyn at oracle.com >>>> >>>> > >>> > >>>> >>> >>>> > >>> >> wrote: >>>> > >>>> >? ? ?On 12/3/18 20:15, Chris Plummer wrote: >>>> >? ? ? > Hi JC, >>>> >? ? ? > >>>> >? ? ? > Overall it looks good. A few naming nits >>>> thought: >>>> >? ? ? > >>>> >? ? ? > In bi01t001.cpp, why have you declared the >>>> >? ? ?ExceptionCheckingJniEnvPtr >>>> >? ? ? > using jni_env(jni). Elsewhere you use >>>> jni(jni_env) and rename the >>>> >? ? ? > method argument passed in from jni to jni_env. >>>> >? ? ? > >>>> >? ? ? > Related to this, I also noticed in some >>>> files that already are using >>>> >? ? ? > ExceptionCheckingJniEnvPtr, such as >>>> CharArrayCriticalLocker.cpp, you >>>> >? ? ? > delcared it as env(jni_env). So that means >>>> there are 3 different >>>> >? ? ?names >>>> >? ? ? > you have used for the >>>> ExceptionCheckingJniEnvPtr local variable. >>>> >? ? ?They >>>> >? ? ? > should be consistent. >>>> >? ? ? > >>>> >? ? ? > Also, can you rename get_basename() to >>>> get_dirname()? I know Serguei >>>> >? ? ? > suggested get_basename() a while back, but >>>> unless "basename" is >>>> >? ? ? > commonly used for this purpose, I think >>>> "dirname" is more self >>>> >? ? ? > explanatory. >>>> > >>>> >? ? ?In general, I'm Okay with get_dirname(). >>>> >? ? ?Just to mention dirname can be both short or >>>> full, so it is a little >>>> >? ? ?confusing as well. >>>> >? ? ?It is the reason why the get_basename() was >>>> suggested. >>>> >? ? ?However, I do not insist on get_basename() nor >>>> get_full_dirname(). :) >>>> > >>>> >? ? ?Thanks, >>>> >? ? ?Serguei >>>> > >>>> > >>>> >? ? ? > thanks, >>>> >? ? ? > >>>> >? ? ? > Chris >>>> >? ? ? > >>>> >? ? ? > On 12/2/18 10:29 PM, David Holmes wrote: >>>> >? ? ? >> Hi Jc, >>>> >? ? ? >> >>>> >? ? ? >> I've been lurking on this one and have had >>>> a look through. I'm okay >>>> >? ? ? >> with the FatalError approach for the tests >>>> - we don't expect >>>> >? ? ?anything >>>> >? ? ? >> to go wrong in a well written test in a >>>> correctly functioning VM. >>>> >? ? ? >> >>>> >? ? ? >> Thanks, >>>> >? ? ? >> David >>>> >? ? ? >> >>>> >? ? ? >> >>>> >? ? ? >> >>>> >? ? ? >> On 3/12/2018 3:24 pm, JC Beyler wrote: >>>> >? ? ? >>> Hi all, >>>> >? ? ? >>> >>>> >? ? ? >>> Would someone on the GC or runtime team >>>> be motivated to give >>>> >? ? ?this a >>>> >? ? ? >>> review? :) >>>> >? ? ? >>> >>>> >? ? ? >>> It would be much appreciated! >>>> >? ? ? >>> >>>> >? ? ? >>> Webrev: >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>> >>>> >? ? ? >>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> >? ? ? >>> >>>> >? ? ? >>> Thanks for your help, >>>> >? ? ? >>> Jc >>>> >? ? ? >>> >>>> >? ? ? >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler >>>> >>>> >? ? ?>>> > >>>> >? ? ? >>> >>> >>>> >>> >>> wrote: >>>> >? ? ? >>> >>>> >? ? ? >>> ??? Hi Chris, >>>> >? ? ? >>> >>>> >? ? ? >>> ??? Yes I was waiting for another review >>>> since you had explicitly >>>> >? ? ? >>> asked :) >>>> >? ? ? >>> >>>> >? ? ? >>> ??? And sounds good that when someone >>>> from GC or runtime gives a >>>> >? ? ? >>> review, >>>> >? ? ? >>> ??? I'll wait for your full review on the >>>> webrev.02! >>>> >? ? ? >>> >>>> >? ? ? >>> ??? Thanks again for?your help, >>>> >? ? ? >>> ??? Jc >>>> >? ? ? >>> >>>> >? ? ? >>> >>>> >? ? ? >>> ??? On Tue, Nov 27, 2018 at 12:48 PM >>>> Chris Plummer >>>> >? ? ? >>> ??? >>> >>>> >>> > >>>> >? ? ?>>> >>>> >>> >>> >>>> >? ? ?wrote: >>>> >? ? ? >>> >>>> >? ? ? >>> ??????? Hi JC, >>>> >? ? ? >>> >>>> >? ? ? >>> ??????? I think it would be good to get a >>>> review from the gc or >>>> >? ? ?runtime >>>> >? ? ? >>> ??????? teams, since this also affects >>>> their tests. >>>> >? ? ? >>> >>>> >? ? ? >>> ??????? Also, once we are settled on this >>>> FatalError approach, >>>> >? ? ?I still >>>> >? ? ? >>> ??????? need to give your webrev-02 a >>>> full review. I only >>>> >? ? ?skimmed over >>>> >? ? ? >>> ??????? parts of it (I did look at all >>>> the changes in webrevo-01). >>>> >? ? ? >>> >>>> >? ? ? >>> ??????? thanks, >>>> >? ? ? >>> >>>> >? ? ? >>> ??????? Chris >>>> >? ? ? >>> >>>> >? ? ? >>> ??????? On 11/27/18 8:58 AM, >>>> serguei.spitsyn at oracle.com >>>> >>>> >? ? ?>>> > >>>> >? ? ? >>> >>>> >>> >>>> >? ? ?>>> >> wrote: >>>> >? ? ? >>>> ??????? Hi Jc, >>>> >? ? ? >>>> >>>> >? ? ? >>>> ??????? I've already reviewed this too. >>>> >? ? ? >>>> >>>> >? ? ? >>>> ??????? Thanks, >>>> >? ? ? >>>> ??????? Serguei >>>> >? ? ? >>>> >>>> >? ? ? >>>> >>>> >? ? ? >>>> ??????? On 11/27/18 06:56, JC Beyler wrote: >>>> >? ? ? >>>>> Thanks Chris, >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> Anybody else motivated to look at this >>>> and review it? :) >>>> >? ? ? >>>>> ??????? Jc >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> ??????? On Mon, Nov 26, 2018 at 1:26 PM >>>> Chris Plummer >>>> >? ? ? >>>>> >>> >>>> >? ? ?>>> > >>>> >>> >>>> >? ? ?>>> >>> >>>> >? ? ? >>>>> wrote: >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> Hi JC, >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> I'm ok with the FatalError approach, >>>> but would >>>> >? ? ?like to >>>> >? ? ? >>>>> hear opinions from others also. >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> thanks, >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> Chris >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >>>> >? ? ? >>>>>> ??????????? Hi Chris, >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? Thanks for taking the time >>>> to look at it and yes you >>>> >? ? ? >>>>>> ??????????? have raised exactly why >>>> the webrev is between two >>>> >? ? ? >>>>>> ??????????? worlds: in cases where a >>>> fatal error on failure is >>>> >? ? ? >>>>>> ??????????? wanted, should we simplify >>>> the code to remove >>>> >? ? ?the return >>>> >? ? ? >>>>>> ??????????? tests since we do them >>>> internally? Now that I've >>>> >? ? ?looked >>>> >? ? ? >>>>>> ??????????? around for non-fatal >>>> cases, I think the answer >>>> >? ? ?is yes, >>>> >? ? ? >>>>>> ??????????? it simplifies the code >>>> while maintaining the checks. >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? I looked a bit and it >>>> seems that I can't find >>>> >? ? ?easily a >>>> >? ? ? >>>>>> ??????????? case where the test >>>> accepts a JNI failure to >>>> >? ? ?then move >>>> >? ? ? >>>>>> ??????????? on. Therefore, perhaps, >>>> for now, the fail with a >>>> >? ? ?Fatal >>>> >? ? ? >>>>>> ??????????? is enough and we can work >>>> on the tests to clean >>>> >? ? ?them up? >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? That means that this is >>>> the new webrev with only >>>> >? ? ?Fatal >>>> >? ? ? >>>>>> ??????????? and cleans up the tests so >>>> that it is no longer in >>>> >? ? ? >>>>>> ??????????? between two worlds: >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? Webrev: >>>> >? ? ? >>>>>> >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>> >>>> >? ? ? >>>>>> >>>> >>>> >? ? ? >>>>>> ??????????? Bug: >>>> > https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? (This passes testing on my >>>> dev machine for all the >>>> >? ? ? >>>>>> ??????????? modified tests) >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? with the example you >>>> provided, it now looks like: >>>> >? ? ? >>>>>> >>>> > >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> >>>> > >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> > >>>> ? >>>> > >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? Where it does, to me at >>>> least, seem cleaner and less >>>> >? ? ? >>>>>> ??????????? "noisy". >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? Let me know what you think, >>>> >? ? ? >>>>>> ??????????? Jc >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? On Tue, Nov 20, 2018 at >>>> 9:33 PM Chris Plummer >>>> >? ? ? >>>>>> ??????????? >>> >>>> >? ? ?>>> > >>>> >? ? ? >>>>>> >>>> >>> >>>> >? ? ?>>> >>> wrote: >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? Hi JC, >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? Sorry about the delay. >>>> I had to go back an >>>> >? ? ?look at >>>> >? ? ? >>>>>> ??????????????? the initial 8210842 >>>> webrev and RFR thread to see >>>> >? ? ? >>>>>> ??????????????? what this was >>>> initially all about. >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? In general the changes >>>> look good. >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? I don't have a good >>>> answer to your >>>> >? ? ? >>>>>> >>>> FatalError/NonFatalError question. It makes >>>> >? ? ?the code >>>> >? ? ? >>>>>> ??????????????? a lot cleaner to use >>>> FatalError, but then it >>>> >? ? ?is a >>>> >? ? ? >>>>>> ??????????????? behavior change, and >>>> you also need to deal with >>>> >? ? ? >>>>>> ??????????????? tests that >>>> intentionally induce errors (do >>>> >? ? ?you have >>>> >? ? ? >>>>>> ??????????????? an example of that). >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? In any case, right now >>>> your webrev seems to be >>>> >? ? ? >>>>>> ??????????????? between two worlds. >>>> You are producing >>>> >? ? ?FatalError, >>>> >? ? ? >>>>>> ??????????????? but still checking >>>> results. Here's a good >>>> >? ? ?example: >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> > >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>> >>>> > >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> > >>>> ? >>>> > >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? I'm not sure if this >>>> is just a temporary >>>> >? ? ?state until >>>> >? ? ? >>>>>> ??????????????? it was decided which >>>> approach to take. >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? thanks, >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? Chris >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????????? On 11/20/18 2:14 PM, >>>> JC Beyler wrote: >>>> >? ? ? >>>>>>> ??????????????? Hi all, >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Chris thought it made >>>> sense to have more >>>> >? ? ?eyes on >>>> >? ? ? >>>>>>> ??????????????? this change than just >>>> serviceability as it will >>>> >? ? ? >>>>>>> ??????????????? modify to tests that >>>> are not only >>>> >? ? ?serviceability >>>> >? ? ? >>>>>>> ??????????????? tests so I've moved >>>> this to conversation >>>> >? ? ?here :) >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? For convenience, I've >>>> copy-pasted the >>>> >? ? ?initial RFR: >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Could I have a review >>>> for the extension and >>>> >? ? ?usage >>>> >? ? ? >>>>>>> ??????????????? of the >>>> ExceptionJniWrapper. This adds lines and >>>> >? ? ? >>>>>>> ??????????????? filenames to the end >>>> of the wrapper JNI >>>> >? ? ?methods, >>>> >? ? ? >>>>>>> ??????????????? adds tracing, and >>>> throws an error if need >>>> >? ? ?be. I've >>>> >? ? ? >>>>>>> ??????????????? ported the gc/lock >>>> files to use the new >>>> >? ? ? >>>>>>> ??????????????? TRACE_JNI_CALL add-on >>>> and I've ported a few >>>> >? ? ?of the >>>> >? ? ? >>>>>>> ??????????????? tests that were >>>> already changed for the >>>> >? ? ?assignment >>>> >? ? ? >>>>>>> ??????????????? webrev for JDK-8212884. >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Webrev: >>>> >? ? ? >>>>>>> >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>> >>>> >? ? ? >>>>>>> >>>> >>>> >? ? ? >>>>>>> ??????????????? Bug: >>>> >? ? ? >>>>>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? For illustration, if >>>> I force an error to the >>>> >? ? ? >>>>>>> ??????????????? AP04/ap04t03 test and >>>> set the verbosity on, >>>> >? ? ?I get >>>> >? ? ? >>>>>>> ??????????????? something like: >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? >> Calling JNI method >>>> FindClass from >>>> >? ? ? >>>>>>> ap04t003.cpp:343 >>>> >? ? ? >>>>>>> ??????????????? >> Calling with these >>>> parameter(s): >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? java/lang/Threadd >>>> >? ? ? >>>>>>> ??????????????? Wait for thread to finish >>>> >? ? ? >>>>>>> ??????????????? << Called JNI method >>>> FindClass from >>>> >? ? ? >>>>>>> ap04t003.cpp:343 >>>> >? ? ? >>>>>>> ??????????????? Exception in thread >>>> "Thread-0" >>>> >? ? ? >>>>>>> java.lang.NoClassDefFoundError: >>>> >? ? ?java/lang/Threadd >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Method) >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Caused by: >>>> java.lang.ClassNotFoundException: >>>> >? ? ? >>>>>>> ??????????????? java.lang.Threadd >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>> > >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>> > >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? ... 3 more >>>> >? ? ? >>>>>>> ??????????????? FATAL ERROR in native >>>> method: JNI method >>>> >? ? ?FindClass >>>> >? ? ? >>>>>>> ??????????????? : internal error from >>>> ap04t003.cpp:343 >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Method) >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>> >? ? ? >>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Questions/comments I >>>> have about this are: >>>> >? ? ? >>>>>>> ??????????????? ? - Do we want to >>>> force fatal errors when a JNI >>>> >? ? ? >>>>>>> ??????????????? call fails in >>>> general? Most of these tests >>>> >? ? ?do the >>>> >? ? ? >>>>>>> ??????????????? right thing and test >>>> the return of the JNI >>>> >? ? ?calls, >>>> >? ? ? >>>>>>> ??????????????? for example: >>>> >? ? ? >>>>>>> ??????????????? ? ? thrClass = >>>> > ?jni->FindClass("java/lang/Threadd", >>>> >? ? ? >>>>>>> ??????????????? TRACE_JNI_CALL); >>>> >? ? ? >>>>>>> ??????????????? ? ? if (thrClass == >>>> NULL) { >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? but now the wrapper >>>> actually would do a >>>> >? ? ?fatal if >>>> >? ? ? >>>>>>> ??????????????? the FindClass call >>>> would return a nullptr, >>>> >? ? ?so we >>>> >? ? ? >>>>>>> ??????????????? could remove that >>>> test altogether. What do you >>>> >? ? ? >>>>>>> think? >>>> >? ? ? >>>>>>> ??????????????? ? ? - I prefer to >>>> leave them as the tests then >>>> >? ? ? >>>>>>> ??????????????? become closer to what >>>> real users would have in >>>> >? ? ? >>>>>>> ??????????????? their code and is the >>>> "recommended" way of >>>> >? ? ?doing it >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? ? ?- The alternative >>>> is to use the >>>> >? ? ?NonFatalError I >>>> >? ? ? >>>>>>> ??????????????? added which then just >>>> prints out that something >>>> >? ? ? >>>>>>> ??????????????? went wrong, letting >>>> the test continue. Question >>>> >? ? ? >>>>>>> ??????????????? will be what should >>>> be the default? The >>>> >? ? ?fatal or >>>> >? ? ? >>>>>>> ??????????????? the non-fatal error >>>> handling? >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? On a different subject: >>>> >? ? ? >>>>>>> ??????????????? ? - On the new tests, >>>> I've removed the >>>> >? ? ? >>>>>>> ??????????????? NSK_JNI_VERIFY since >>>> the JNI wrapper >>>> >? ? ?handles the >>>> >? ? ? >>>>>>> ??????????????? tracing and the >>>> verify in almost the same >>>> >? ? ?way; only >>>> >? ? ? >>>>>>> ??????????????? difference I can >>>> really tell is that the >>>> >? ? ?complain >>>> >? ? ? >>>>>>> ??????????????? method from NSK has a >>>> max complain before >>>> >? ? ?stopping >>>> >? ? ? >>>>>>> ??????????????? to "complain"; I have >>>> not added that part >>>> >? ? ?of the >>>> >? ? ? >>>>>>> ??????????????? code in this webrev >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Once we decide on >>>> these, I can continue on the >>>> >? ? ? >>>>>>> ??????????????? files from >>>> JDK-8212884 and then do both the >>>> >? ? ? >>>>>>> ??????????????? assignment in an if >>>> extraction followed-by this >>>> >? ? ? >>>>>>> ??????????????? type of webrev in an >>>> easier fashion. >>>> >? ? ?Depending on >>>> >? ? ? >>>>>>> ??????????????? decisions here, >>>> NSK*VERIFY can be deprecated as >>>> >? ? ? >>>>>>> ??????????????? well as we go forward. >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? Thanks! >>>> >? ? ? >>>>>>> ??????????????? Jc >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? On Mon, Nov 19, 2018 >>>> at 11:34 AM Chris Plummer >>>> >? ? ? >>>>>>> >>>> >>> >>>> >? ? ?>>> > >>>> >? ? ? >>>>>>> >>> >>>> >? ? ?>>> >>> wrote: >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????????? On 11/19/18 10:07 >>>> AM, JC Beyler wrote: >>>> >? ? ? >>>>>>>> ??????????????????? Hi all, >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????? @David/Chris: >>>> should I then push this >>>> >? ? ?RFR to >>>> >? ? ? >>>>>>>> ??????????????????? the hotspot >>>> mailing or the runtime >>>> >? ? ?one? For >>>> >? ? ? >>>>>>>> ??????????????????? what it's worth, >>>> a lot of the tests >>>> >? ? ?under the >>>> >? ? ? >>>>>>>> ??????????????????? vmTestbase are >>>> jvmti so the review also >>>> >? ? ? >>>>>>>> ??????????????????? affects >>>> serviceability; it just turns >>>> >? ? ?out I >>>> >? ? ? >>>>>>>> ??????????????????? started with the >>>> GC originally and >>>> >? ? ?then hit >>>> >? ? ? >>>>>>>> ??????????????????? some other tests >>>> I had touched via the >>>> >? ? ? >>>>>>>> ??????????????????? assignment >>>> extraction. >>>> >? ? ? >>>>>>> ??????????????????? I think hotspot >>>> would be best. >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????????? Chris >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????? @Serguei: Done >>>> for the method >>>> >? ? ?renaming, for >>>> >? ? ? >>>>>>>> ??????????????????? the indent, are >>>> you talking about >>>> >? ? ?going from >>>> >? ? ? >>>>>>>> ??????????????????? the 8-indent to >>>> 4-indent? If so, would >>>> >? ? ?it not >>>> >? ? ? >>>>>>>> ??????????????????? just be better >>>> to do a new JBS bug and >>>> >? ? ?do the >>>> >? ? ? >>>>>>>> ??????????????????? whole files in >>>> one go? I ask because >>>> >? ? ? >>>>>>>> ??????????????????? otherwise, it >>>> will look a bit weird to >>>> >? ? ?have >>>> >? ? ? >>>>>>>> ??????????????????? parts of the >>>> file as 8-indent and others >>>> >? ? ? >>>>>>>> 4-indent? >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????? Thanks for >>>> looking at it! >>>> >? ? ? >>>>>>>> ??????????????????? Jc >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????? On Mon, Nov 19, >>>> 2018 at 1:25 AM >>>> >? ? ? >>>>>>>> serguei.spitsyn at oracle.com >>>> >>>> >>> > >>>> >? ? ? >>>>>>>> >>> >>>> >? ? ?>>> >> >>>> >? ? ? >>>>>>>> >>>> >>> >>>> >? ? ?>>> > >>>> >? ? ? >>>>>>>> >>> >>>> >? ? ?>>> >>> wrote: >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????????? Hi Jc, >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????????? We have to >>>> start this review >>>> >? ? ?anyway. :) >>>> >? ? ? >>>>>>>> ??????????????????????? It looks >>>> good to me in general. >>>> >? ? ? >>>>>>>> ??????????????????????? Thank you >>>> for your consistency in this >>>> >? ? ? >>>>>>>> ??????????????????????? refactoring! >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????????? Some minor >>>> comments. >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> >>>> > >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>> > >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????????? +static >>>> const char* >>>> >? ? ?remove_folders(const >>>> >? ? ? >>>>>>>> ??????????????????????? char* >>>> fullname) { I'd suggest to >>>> >? ? ?rename >>>> >? ? ? >>>>>>>> ??????????????????????? the function >>>> name to something >>>> >? ? ?traditional >>>> >? ? ? >>>>>>>> ??????????????????????? like >>>> get_basename. Otherwise, it >>>> >? ? ?sounds >>>> >? ? ? >>>>>>>> ??????????????????????? like this >>>> function has to really >>>> >? ? ?remove >>>> >? ? ? >>>>>>>> ??????????????????????? folders. :) >>>> Also, all *Locker.cpp have >>>> >? ? ? >>>>>>>> ??????????????????????? wrong indent >>>> in the bodies of if >>>> >? ? ?and while >>>> >? ? ? >>>>>>>> ??????????????????????? statements. >>>> Could this be fixed >>>> >? ? ?with the >>>> >? ? ? >>>>>>>> ??????????????????????? refactoring? >>>> I did not look on how >>>> >? ? ?this >>>> >? ? ? >>>>>>>> ??????????????????????? impacts the >>>> tests other than >>>> >? ? ? >>>>>>>> >>>> serviceability. Thanks, Serguei >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????????? On 11/16/18 >>>> 19:43, JC Beyler wrote: >>>> > >>>>>>>>> ??????????????????????? Hi all, >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????? Anybody >>>> motivated to review this? :) >>>> > >>>>>>>>> ??????????????????????? Jc >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????? On Wed, Nov 7, >>>> 2018 at 9:53 PM JC >>>> >? ? ?Beyler >>>> > >>>>>>>>> >>>> >>>> >? ? ?>>> > >>>> > >>>>>>>>> >>> >>>> >? ? ?>>> >>> wrote: >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Hi all, >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Could I have >>>> a review for the >>>> > >>>>>>>>> ??????????????????????????? extension >>>> and usage of the >>>> > >>>>>>>>> ExceptionJniWrapper. This >>>> >? ? ?adds lines >>>> > >>>>>>>>> ??????????????????????????? and >>>> filenames to the end of the >>>> > >>>>>>>>> ??????????????????????????? wrapper JNI >>>> methods, adds >>>> >? ? ?tracing, >>>> > >>>>>>>>> ??????????????????????????? and throws >>>> an error if need >>>> >? ? ?be. I've >>>> > >>>>>>>>> ??????????????????????????? ported the >>>> gc/lock files to >>>> >? ? ?use the >>>> > >>>>>>>>> ??????????????????????????? new >>>> TRACE_JNI_CALL add-on and >>>> >? ? ?I've >>>> > >>>>>>>>> ??????????????????????????? ported a few >>>> of the tests >>>> >? ? ?that were >>>> > >>>>>>>>> ??????????????????????????? already >>>> changed for the >>>> >? ? ?assignment >>>> > >>>>>>>>> ??????????????????????????? webrev for >>>> JDK-8212884. >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Webrev: >>>> > >>>>>>>>> >>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>> >>>> > >>>>>>>>> >>>> >>>> > >>>>>>>>> ??????????????????????????? Bug: >>>> > >>>>>>>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? For >>>> illustration, if I force >>>> >? ? ?an error >>>> > >>>>>>>>> ??????????????????????????? to the >>>> AP04/ap04t03 test and >>>> >? ? ?set the >>>> > >>>>>>>>> ??????????????????????????? verbosity >>>> on, I get something >>>> >? ? ?like: >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? >> Calling >>>> JNI method >>>> >? ? ?FindClass from >>>> > >>>>>>>>> ap04t003.cpp:343 >>>> > >>>>>>>>> ??????????????????????????? >> Calling >>>> with these >>>> >? ? ?parameter(s): >>>> > >>>>>>>>> java/lang/Threadd >>>> > >>>>>>>>> ??????????????????????????? Wait for >>>> thread to finish >>>> > >>>>>>>>> ??????????????????????????? << Called >>>> JNI method >>>> >? ? ?FindClass from >>>> > >>>>>>>>> ap04t003.cpp:343 >>>> > >>>>>>>>> ??????????????????????????? Exception in >>>> thread "Thread-0" >>>> > >>>>>>>>> java.lang.NoClassDefFoundError: >>>> > >>>>>>>>> java/lang/Threadd >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Method) >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Caused by: >>>> > >>>>>>>>> java.lang.ClassNotFoundException: >>>> > >>>>>>>>> java.lang.Threadd >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>> > >>>>>>>>> ??????????????????????????? ... 3 more >>>> > >>>>>>>>> ??????????????????????????? FATAL ERROR >>>> in native method: JNI >>>> > >>>>>>>>> ??????????????????????????? method >>>> FindClass : internal error >>>> > >>>>>>>>> ??????????????????????????? from >>>> ap04t003.cpp:343 >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Method) >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? at >>>> > >>>>>>>>> >>>> > >>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> > >>>> > >>>>>>>>> >>>> > >>>>>>>>> >>>> > >>>>>>>>> Questions/comments I have about >>>> > >>>>>>>>> this are: >>>> > >>>>>>>>> ??????????????????????????? ? - Do we >>>> want to force fatal >>>> >? ? ?errors >>>> > >>>>>>>>> ??????????????????????????? when a JNI >>>> call fails in general? >>>> > >>>>>>>>> ??????????????????????????? Most of >>>> these tests do the right >>>> > >>>>>>>>> ??????????????????????????? thing and >>>> test the return of >>>> >? ? ?the JNI >>>> > >>>>>>>>> ??????????????????????????? calls, for >>>> example: >>>> > >>>>>>>>> ??????????????????????????? thrClass = >>>> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >>>> > >>>>>>>>> TRACE_JNI_CALL); >>>> > >>>>>>>>> ??????????????????????????? ? ? if >>>> (thrClass == NULL) { >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? but now the >>>> wrapper actually >>>> >? ? ?would do >>>> > >>>>>>>>> ??????????????????????????? a fatal if >>>> the FindClass call >>>> >? ? ?would >>>> > >>>>>>>>> ??????????????????????????? return a >>>> nullptr, so we could >>>> >? ? ?remove >>>> > >>>>>>>>> ??????????????????????????? that test >>>> altogether. What do >>>> >? ? ?you >>>> > >>>>>>>>> think? >>>> > >>>>>>>>> ??????????????????????????? ? ? - I >>>> prefer to leave them >>>> >? ? ?as the >>>> > >>>>>>>>> ??????????????????????????? tests then >>>> become closer to >>>> >? ? ?what real >>>> > >>>>>>>>> ??????????????????????????? users would >>>> have in their >>>> >? ? ?code and is >>>> > >>>>>>>>> ??????????????????????????? the >>>> "recommended" way of doing it >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? ? ?- The >>>> alternative is to >>>> >? ? ?use the >>>> > >>>>>>>>> >>>> NonFatalError I added which >>>> >? ? ?then just >>>> > >>>>>>>>> ??????????????????????????? prints out >>>> that something >>>> >? ? ?went wrong, >>>> > >>>>>>>>> ??????????????????????????? letting the >>>> test continue. >>>> >? ? ?Question >>>> > >>>>>>>>> ??????????????????????????? will be what >>>> should be the >>>> >? ? ?default? >>>> > >>>>>>>>> ??????????????????????????? The fatal or >>>> the non-fatal error >>>> > >>>>>>>>> ??????????????????????????? handling? >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? On a >>>> different subject: >>>> > >>>>>>>>> ??????????????????????????? ? - On the >>>> new tests, I've >>>> >? ? ?removed >>>> > >>>>>>>>> ??????????????????????????? the >>>> NSK_JNI_VERIFY since the JNI >>>> > >>>>>>>>> ??????????????????????????? wrapper >>>> handles the tracing >>>> >? ? ?and the >>>> > >>>>>>>>> ??????????????????????????? verify in >>>> almost the same >>>> >? ? ?way; only >>>> > >>>>>>>>> ??????????????????????????? difference I >>>> can really tell >>>> >? ? ?is that >>>> > >>>>>>>>> ??????????????????????????? the complain >>>> method from NSK >>>> >? ? ?has a >>>> > >>>>>>>>> ??????????????????????????? max complain >>>> before stopping to >>>> > >>>>>>>>> ??????????????????????????? "complain"; >>>> I have not added that >>>> > >>>>>>>>> ??????????????????????????? part of the >>>> code in this webrev >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Once we >>>> decide on these, I can >>>> > >>>>>>>>> ??????????????????????????? continue on >>>> the files from >>>> > >>>>>>>>> ??????????????????????????? JDK-8212884 >>>> and then do both the >>>> > >>>>>>>>> ??????????????????????????? assignment >>>> in an if extraction >>>> > >>>>>>>>> ??????????????????????????? followed-by >>>> this type of >>>> >? ? ?webrev in an >>>> > >>>>>>>>> ??????????????????????????? easier >>>> fashion. Depending on >>>> > >>>>>>>>> ??????????????????????????? decisions >>>> here, NSK*VERIFY can be >>>> > >>>>>>>>> ??????????????????????????? deprecated >>>> as well as we go >>>> >? ? ?forward. >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????????? Thank you >>>> for the >>>> >? ? ?reviews/comments :) >>>> > >>>>>>>>> ??????????????????????????? Jc >>>> > >>>>>>>>> >>>> > >>>>>>>>> >>>> > >>>>>>>>> >>>> > >>>>>>>>> ??????????????????????? -- >>>> > >>>>>>>>> ??????????????????????? Thanks, >>>> > >>>>>>>>> ??????????????????????? Jc >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> >>>> >? ? ? >>>>>>>> ??????????????????? -- >>>> >? ? ? >>>>>>>> ??????????????????? Thanks, >>>> >? ? ? >>>>>>>> ??????????????????? Jc >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> >>>> >? ? ? >>>>>>> ??????????????? -- >>>> >? ? ? >>>>>>> ??????????????? Thanks, >>>> >? ? ? >>>>>>> ??????????????? Jc >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> >>>> >? ? ? >>>>>> ??????????? -- >>>> >? ? ? >>>>>> ??????????? Thanks, >>>> >? ? ? >>>>>> ??????????? Jc >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> >>>> >? ? ? >>>>> ??????? -- >>>> >? ? ? >>>>> Thanks, >>>> >? ? ? >>>>> ??????? Jc >>>> >? ? ? >>>> >>>> >? ? ? >>> >>>> >? ? ? >>> >>>> >? ? ? >>> >>>> >? ? ? >>> ??? -- >>>> >? ? ? >>> ??? Thanks, >>>> >? ? ? >>> ??? Jc >>>> >? ? ? >>> >>>> >? ? ? >>> >>>> >? ? ? >>> >>>> >? ? ? >>> -- >>>> >? ? ? >>> >>>> >? ? ? >>> Thanks, >>>> >? ? ? >>> Jc >>>> >? ? ? > >>>> >? ? ? > >>>> > >>>> > >>>> > >>>> > -- >>>> > >>>> > Thanks, >>>> > Jc >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> >> >> >> -- >> >> Thanks, >> Jc >> >> >> >> -- >> >> Thanks, >> Jc > From serguei.spitsyn at oracle.com Wed Dec 12 21:55:25 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 12 Dec 2018 13:55:25 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <839301b7-c247-df3b-e485-283e8bb7388b@oracle.com> References: <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> <839301b7-c247-df3b-e485-283e8bb7388b@oracle.com> Message-ID: <95fe277d-ba6e-4fec-77aa-d1f1051751aa@oracle.com> Agreed. Thanks, Serguei On 12/12/18 13:52, David Holmes wrote: > FWIW I think namespaces are overkill in all of this test code and just > obfuscates things - the declaration is easily missed. A static > variable in a .cpp is clearly a global variable to the file. > > Cheers, > David > > > > On 13/12/2018 5:37 am, serguei.spitsyn at oracle.com wrote: >> Hi Jc, >> >> >> On 12/11/18 21:16, JC Beyler wrote: >>> Hi all, >>> >>> Here is the new webrev with the TEST.groups change. Serguei, let me >>> know if I convinced you with the static vs anonymous namespaces or >>> if you'd still rather have a "static" for now :-) >> >> >> What do you think about this post? : >> https://stackoverflow.com/questions/11623451/static-vs-non-static-variables-in-namespace >> >> >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> >> The update looks fine. >> >> Thanks, >> Serguei >> >> >> Thanks, >> Serguei >> >>> >>> Thanks again for the reviews! >>> Jc >>> >>> On Mon, Dec 10, 2018 at 3:10 PM JC Beyler >> > wrote: >>> >>> ??? Hi Serguei, >>> >>> ??? Yes basically it is equivalent :) I can put them in but they are >>> ??? not required. The norm actually wanted to deprecate it but then >>> ??? remembered that C compatibility would require the static key-word >>> ??? for this case [1] >>> >>> ??? So, really, they are not required here and will amount to the same >>> ??? thing: only that file can refer to them and you cannot get to them >>> ??? without a globally available method to return a pointer to them >>> ??? (ie same as a static variable in C). >>> >>> ??? I can put static if it makes it easier to see but, by being in an >>> ??? anonymous namespace they are only available for the file's >>> ??? translation unit. For example: >>> >>> ??? $ cat main.cpp >>> >>> ??? int totally_global; >>> ??? static int explictly_static; >>> >>> ??? namespace { >>> ??? int implicitly_static; >>> ??? } >>> >>> ??? void foo(); >>> ??? int main() { >>> ??? ? foo(); >>> ??? } >>> >>> ??? $ g++ -O3 main.cpp -c >>> ??? $ nm main.o >>> ??? ? ? ? ? ? ? ? ? ?U _GLOBAL_OFFSET_TABLE_ >>> ??? 0000000000000000 T main >>> ??? 0000000000000000 B totally_global >>> ??? ? ? ? ? ? ? ? ? ?U _Z3foov >>> >>> ??? As you can see, the static and anonymous namespace variables are >>> ??? not in the file due to not being used. If you were to use them, >>> ??? you'd see them show up as something like: >>> ??? 0000000000000008 b _ZL17explicitly_static >>> ??? 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE >>> >>> ??? Where again, it shows that it is mangling the names so that no >>> ??? external usage can happen without tinkering. >>> >>> ??? Hopefully that helps :-), >>> ??? Jc >>> >>> ??? [1] >>> http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 >>> >>> >>> ??? On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com >>> ??? >> ??? > wrote: >>> >>> ??????? Hi Jc, >>> >>> ??????? I had little experience with the C++ namespaces. >>> ??????? My understanding is that static in this context should mean >>> ??????? internal linkage. >>> >>> ??????? Thanks, >>> ??????? Serguei >>> >>> >>> ??????? On 12/10/18 13:57, JC Beyler wrote: >>>> ??????? Hi Serguei, >>>> >>>> ??????? The variables and functions are in a anonymous namespace; my >>>> ??????? understanding of C++ is that this is equivalent to putting it >>>> ??????? as static.Hence, I didn't add them there. Does that make >>>> sense? >>>> >>>> ??????? Thanks! >>>> ??????? Jc >>>> >>>> ??????? On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com >>>> ??????? >>>> ??????? >>> ??????? > wrote: >>>> >>>> ??????????? Hi Jc, >>>> >>>> ??????????? It looks good in general. >>>> ??????????? One question though. >>>> >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>>> >>>> ??????????? I wonder if the variables and functions have to be static. >>>> >>>> ??????????? Thanks, >>>> ??????????? Serguei >>>> >>>> >>>> ??????????? On 12/5/18 11:36, JC Beyler wrote: >>>>> ??????????? Hi all, >>>>> >>>>> ??????????? My apologies to having to come back for another review >>>>> ??????????? for this change: I ran into a snag when trying to pull >>>>> ??????????? the latest changes compared to the base I was working >>>>> ??????????? on. I basically forgot that there was an issue with >>>>> ??????????? snprintf and that I had solved it via?JDK-8213622. >>>>> >>>>> ??????????? Could I have a new review of this webrev: >>>>> ??????????? Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ >>>>> >>>>> ??????????? Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> ??????????? Incremental from the port of webrev.03 that got LGTMs: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ >>>>> >>>>> >>>>> ??????????? A few comments on this because it took me a while to get >>>>> ??????????? things in a state I thought was good: >>>>> ??????????? ? - I had to implement an itoa method, do we have >>>>> ??????????? something like that in the test base (remember that >>>>> ??????????? JDK-8213622 could not use sprintf due to being in the >>>>> ??????????? test code)? >>>>> >>>>> ??????????? ? - The differences here compared to the one you all >>>>> ??????????? reviewed are: >>>>> ??????????? ? ? ? - I found that adding to the strlen/memcpy error >>>>> ??????????? prone and thought that I would try to make it less so. >>>>> ??????????? If you want to compare, I extended the strlen/memcpy >>>>> ??????????? with the new format to show you if you prefer [1] >>>>> ??????????? ? ? ? ? ? ? - Note that the diff between the "old >>>>> ??????????? extended way from [1]" to the webrev.04 can be found >>>>> in [2] >>>>> >>>>> ??????????? ? ? ?- I added a test to test the exception wrapper in >>>>> ??????????? tests :); I'm not sure it is deemed useful or not but >>>>> ??????????? helped me assure myself that I was not doing things >>>>> ??????????? wrong; you can find the base test file here [3]; should >>>>> ??????????? we have this or not? (I know that normally we don't add >>>>> ??????????? tests to vmTestbase but thought this might be an >>>>> exception) >>>>> >>>>> ??????????? Thanks for your help and my apologies for the snag, >>>>> ??????????? Jc >>>>> >>>>> ??????????? [1]: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>> >>>>> ??????????? [2]: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 >>>>> >>>>> ??????????? [3] >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>>>> >>>>> >>>>> ??????????? On Mon, Dec 3, 2018 at 11:29 PM David Holmes >>>>> ??????????? >>>> ??????????? > wrote: >>>>> >>>>> ??????????????? Looks fine to me. >>>>> >>>>> ??????????????? Thanks, >>>>> ??????????????? David >>>>> >>>>> ??????????????? On 4/12/2018 4:04 pm, JC Beyler wrote: >>>>> ??????????????? > Hi both, >>>>> ??????????????? > >>>>> ??????????????? > Thanks for the reviews! Since Serguei did not >>>>> ??????????????? insist on get_basename, I >>>>> ??????????????? > went for get_dirname since the method is a local >>>>> ??????????????? static method and won't >>>>> ??????????????? > have its name start spreading, I think it's ok too. >>>>> ??????????????? > >>>>> ??????????????? > For the naming of the local variable, the idea >>>>> ??????????????? initially was to use the >>>>> ??????????????? > same name as the local variable for JNIEnv already >>>>> ??????????????? used to reduce the >>>>> ??????????????? > code change. Since I'm now adding the line macro >>>>> ??????????????? at the end anyway, this >>>>> ??????????????? > does not matter anymore so I converged all local >>>>> ??????????????? variables to "jni". >>>>> ??????????????? > >>>>> ??????????????? > So, without further ado, here is the new version: >>>>> ??????????????? > Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >>>>> >>>>> ??????????????? > Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> ??????????????? > >>>>> ??????????????? > This passes the various tests changed by the >>>>> ??????????????? webrev on my dev machine. >>>>> ??????????????? > >>>>> ??????????????? > Let me know what you think, >>>>> ??????????????? > Jc >>>>> ??????????????? > >>>>> ??????????????? > On Mon, Dec 3, 2018 at 8:40 PM >>>>> ??????????????? serguei.spitsyn at oracle.com >>>>> ??????????????? >>>>> ??????????????? > >>>> > >>>>> ??????????????? >>>> ??????????????? >>>>> ??????????????? > >>>> >> wrote: >>>>> ??????????????? > >>>>> ??????????????? >? ? ?On 12/3/18 20:15, Chris Plummer wrote: >>>>> ??????????????? >? ? ? > Hi JC, >>>>> ??????????????? >? ? ? > >>>>> ??????????????? >? ? ? > Overall it looks good. A few naming nits >>>>> ??????????????? thought: >>>>> ??????????????? >? ? ? > >>>>> ??????????????? >? ? ? > In bi01t001.cpp, why have you declared the >>>>> ??????????????? >? ? ?ExceptionCheckingJniEnvPtr >>>>> ??????????????? >? ? ? > using jni_env(jni). Elsewhere you use >>>>> ??????????????? jni(jni_env) and rename the >>>>> ??????????????? >? ? ? > method argument passed in from jni to >>>>> jni_env. >>>>> ??????????????? >? ? ? > >>>>> ??????????????? >? ? ? > Related to this, I also noticed in some >>>>> ??????????????? files that already are using >>>>> ??????????????? >? ? ? > ExceptionCheckingJniEnvPtr, such as >>>>> ??????????????? CharArrayCriticalLocker.cpp, you >>>>> ??????????????? >? ? ? > delcared it as env(jni_env). So that means >>>>> ??????????????? there are 3 different >>>>> ??????????????? >? ? ?names >>>>> ??????????????? >? ? ? > you have used for the >>>>> ??????????????? ExceptionCheckingJniEnvPtr local variable. >>>>> ??????????????? >? ? ?They >>>>> ??????????????? >? ? ? > should be consistent. >>>>> ??????????????? >? ? ? > >>>>> ??????????????? >? ? ? > Also, can you rename get_basename() to >>>>> ??????????????? get_dirname()? I know Serguei >>>>> ??????????????? >? ? ? > suggested get_basename() a while back, but >>>>> ??????????????? unless "basename" is >>>>> ??????????????? >? ? ? > commonly used for this purpose, I think >>>>> ??????????????? "dirname" is more self >>>>> ??????????????? >? ? ? > explanatory. >>>>> ??????????????? > >>>>> ??????????????? >? ? ?In general, I'm Okay with get_dirname(). >>>>> ??????????????? >? ? ?Just to mention dirname can be both short or >>>>> ??????????????? full, so it is a little >>>>> ??????????????? >? ? ?confusing as well. >>>>> ??????????????? >? ? ?It is the reason why the get_basename() was >>>>> ??????????????? suggested. >>>>> ??????????????? >? ? ?However, I do not insist on get_basename() nor >>>>> ??????????????? get_full_dirname(). :) >>>>> ??????????????? > >>>>> ??????????????? >? ? ?Thanks, >>>>> ??????????????? >? ? ?Serguei >>>>> ??????????????? > >>>>> ??????????????? > >>>>> ??????????????? >? ? ? > thanks, >>>>> ??????????????? >? ? ? > >>>>> ??????????????? >? ? ? > Chris >>>>> ??????????????? >? ? ? > >>>>> ??????????????? >? ? ? > On 12/2/18 10:29 PM, David Holmes wrote: >>>>> ??????????????? >? ? ? >> Hi Jc, >>>>> ??????????????? >? ? ? >> >>>>> ??????????????? >? ? ? >> I've been lurking on this one and have had >>>>> ??????????????? a look through. I'm okay >>>>> ??????????????? >? ? ? >> with the FatalError approach for the tests >>>>> ??????????????? - we don't expect >>>>> ??????????????? >? ? ?anything >>>>> ??????????????? >? ? ? >> to go wrong in a well written test in a >>>>> ??????????????? correctly functioning VM. >>>>> ??????????????? >? ? ? >> >>>>> ??????????????? >? ? ? >> Thanks, >>>>> ??????????????? >? ? ? >> David >>>>> ??????????????? >? ? ? >> >>>>> ??????????????? >? ? ? >> >>>>> ??????????????? >? ? ? >> >>>>> ??????????????? >? ? ? >> On 3/12/2018 3:24 pm, JC Beyler wrote: >>>>> ??????????????? >? ? ? >>> Hi all, >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> Would someone on the GC or runtime team >>>>> ??????????????? be motivated to give >>>>> ??????????????? >? ? ?this a >>>>> ??????????????? >? ? ? >>> review? :) >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> It would be much appreciated! >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>> >>>>> ??????????????? >? ? ? >>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> Thanks for your help, >>>>> ??????????????? >? ? ? >>> Jc >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler >>>>> ??????????????? >>>>> ??????????????? >? ? ?>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>> >>>> ??????????????? >>>>> ??????????????? >>>> >>> wrote: >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??? Hi Chris, >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??? Yes I was waiting for another review >>>>> ??????????????? since you had explicitly >>>>> ??????????????? >? ? ? >>> asked :) >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??? And sounds good that when someone >>>>> ??????????????? from GC or runtime gives a >>>>> ??????????????? >? ? ? >>> review, >>>>> ??????????????? >? ? ? >>> ??? I'll wait for your full review on the >>>>> ??????????????? webrev.02! >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??? Thanks again for?your help, >>>>> ??????????????? >? ? ? >>> ??? Jc >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??? On Tue, Nov 27, 2018 at 12:48 PM >>>>> ??????????????? Chris Plummer >>>>> ??????????????? >? ? ? >>> >>>> ??????????????? >>>>> ??????????????? >>>> > >>>>> ??????????????? > ?>>>> ??????????????? >>>>> ??????????????? >>>> >>> >>>>> ??????????????? >? ? ?wrote: >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??????? Hi JC, >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??????? I think it would be good to get a >>>>> ??????????????? review from the gc or >>>>> ??????????????? >? ? ?runtime >>>>> ??????????????? >? ? ? >>> ??????? teams, since this also affects >>>>> ??????????????? their tests. >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??????? Also, once we are settled on this >>>>> ??????????????? FatalError approach, >>>>> ??????????????? >? ? ?I still >>>>> ??????????????? >? ? ? >>> ??????? need to give your webrev-02 a >>>>> ??????????????? full review. I only >>>>> ??????????????? >? ? ?skimmed over >>>>> ??????????????? >? ? ? >>> ??????? parts of it (I did look at all >>>>> ??????????????? the changes in webrevo-01). >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??????? thanks, >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??????? Chris >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??????? On 11/27/18 8:58 AM, >>>>> ??????????????? serguei.spitsyn at oracle.com >>>>> ??????????????? >>>>> ??????????????? > ?>>>> > >>>>> ??????????????? >? ? ? >>> ??????????????? >>>>> >>>> ??????????????? >>>>> ??????????????? > ?>>>> >> wrote: >>>>> ??????????????? >? ? ? >>>> ??????? Hi Jc, >>>>> ??????????????? >? ? ? >>>> >>>>> ??????????????? >? ? ? >>>> ??????? I've already reviewed this too. >>>>> ??????????????? >? ? ? >>>> >>>>> ??????????????? >? ? ? >>>> ??????? Thanks, >>>>> ??????????????? >? ? ? >>>> ??????? Serguei >>>>> ??????????????? >? ? ? >>>> >>>>> ??????????????? >? ? ? >>>> >>>>> ??????????????? >? ? ? >>>> ??????? On 11/27/18 06:56, JC Beyler >>>>> wrote: >>>>> ??????????????? >? ? ? >>>>> Thanks Chris, >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> Anybody else motivated to look at this >>>>> ??????????????? and review it? :) >>>>> ??????????????? >? ? ? >>>>> ??????? Jc >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> ??????? On Mon, Nov 26, 2018 at 1:26 PM >>>>> ??????????????? Chris Plummer >>>>> ??????????????? >? ? ? >>>>> >>>> ??????????????? >>>>> ??????????????? > ?>>>> > >>>>> ??????????????? >>>> ??????????????? >>>>> ??????????????? > ?>>>> >>> >>>>> ??????????????? >? ? ? >>>>> wrote: >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> Hi JC, >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> I'm ok with the FatalError approach, >>>>> ??????????????? but would >>>>> ??????????????? >? ? ?like to >>>>> ??????????????? >? ? ? >>>>> hear opinions from others also. >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> thanks, >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> Chris >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >>>>> ??????????????? >? ? ? >>>>>> ??????????? Hi Chris, >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? Thanks for taking the time >>>>> ??????????????? to look at it and yes you >>>>> ??????????????? >? ? ? >>>>>> ??????????? have raised exactly why >>>>> ??????????????? the webrev is between two >>>>> ??????????????? >? ? ? >>>>>> ??????????? worlds: in cases where a >>>>> ??????????????? fatal error on failure is >>>>> ??????????????? >? ? ? >>>>>> ??????????? wanted, should we simplify >>>>> ??????????????? the code to remove >>>>> ??????????????? >? ? ?the return >>>>> ??????????????? >? ? ? >>>>>> ??????????? tests since we do them >>>>> ??????????????? internally? Now that I've >>>>> ??????????????? >? ? ?looked >>>>> ??????????????? >? ? ? >>>>>> ??????????? around for non-fatal >>>>> ??????????????? cases, I think the answer >>>>> ??????????????? >? ? ?is yes, >>>>> ??????????????? >? ? ? >>>>>> ??????????? it simplifies the code >>>>> ??????????????? while maintaining the checks. >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? I looked a bit and it >>>>> ??????????????? seems that I can't find >>>>> ??????????????? >? ? ?easily a >>>>> ??????????????? >? ? ? >>>>>> ??????????? case where the test >>>>> ??????????????? accepts a JNI failure to >>>>> ??????????????? >? ? ?then move >>>>> ??????????????? >? ? ? >>>>>> ??????????? on. Therefore, perhaps, >>>>> ??????????????? for now, the fail with a >>>>> ??????????????? >? ? ?Fatal >>>>> ??????????????? >? ? ? >>>>>> ??????????? is enough and we can work >>>>> ??????????????? on the tests to clean >>>>> ??????????????? >? ? ?them up? >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? That means that this is >>>>> ??????????????? the new webrev with only >>>>> ??????????????? >? ? ?Fatal >>>>> ??????????????? >? ? ? >>>>>> ??????????? and cleans up the tests so >>>>> ??????????????? that it is no longer in >>>>> ??????????????? >? ? ? >>>>>> ??????????? between two worlds: >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? Webrev: >>>>> ??????????????? >? ? ? >>>>>> >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? Bug: >>>>> ??????????????? > https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? (This passes testing on my >>>>> ??????????????? dev machine for all the >>>>> ??????????????? >? ? ? >>>>>> ??????????? modified tests) >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? with the example you >>>>> ??????????????? provided, it now looks like: >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? > >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? > >>>>> ? >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? Where it does, to me at >>>>> ??????????????? least, seem cleaner and less >>>>> ??????????????? >? ? ? >>>>>> ??????????? "noisy". >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? Let me know what you think, >>>>> ??????????????? >? ? ? >>>>>> ??????????? Jc >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? On Tue, Nov 20, 2018 at >>>>> ??????????????? 9:33 PM Chris Plummer >>>>> ??????????????? >? ? ? >>>>>> ??????????? >>>> ??????????????? >>>>> ??????????????? > ?>>>> > >>>>> ??????????????? > >>>>>> >>>> ??????????????? >>>>> ??????????????? > ?>>>> >>> wrote: >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? Hi JC, >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? Sorry about the delay. >>>>> ??????????????? I had to go back an >>>>> ??????????????? >? ? ?look at >>>>> ??????????????? >? ? ? >>>>>> ??????????????? the initial 8210842 >>>>> ??????????????? webrev and RFR thread to see >>>>> ??????????????? >? ? ? >>>>>> ??????????????? what this was >>>>> ??????????????? initially all about. >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? In general the changes >>>>> ??????????????? look good. >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? I don't have a good >>>>> ??????????????? answer to your >>>>> ??????????????? > >>>>>> FatalError/NonFatalError question. It makes >>>>> ??????????????? >? ? ?the code >>>>> ??????????????? >? ? ? >>>>>> ??????????????? a lot cleaner to use >>>>> ??????????????? FatalError, but then it >>>>> ??????????????? >? ? ?is a >>>>> ??????????????? >? ? ? >>>>>> ??????????????? behavior change, and >>>>> ??????????????? you also need to deal with >>>>> ??????????????? >? ? ? >>>>>> ??????????????? tests that >>>>> ??????????????? intentionally induce errors (do >>>>> ??????????????? >? ? ?you have >>>>> ??????????????? >? ? ? >>>>>> ??????????????? an example of that). >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? In any case, right now >>>>> ??????????????? your webrev seems to be >>>>> ??????????????? >? ? ? >>>>>> ??????????????? between two worlds. >>>>> ??????????????? You are producing >>>>> ??????????????? >? ? ?FatalError, >>>>> ??????????????? >? ? ? >>>>>> ??????????????? but still checking >>>>> ??????????????? results. Here's a good >>>>> ??????????????? >? ? ?example: >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? > >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>> >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? > >>>>> ? >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? I'm not sure if this >>>>> ??????????????? is just a temporary >>>>> ??????????????? >? ? ?state until >>>>> ??????????????? >? ? ? >>>>>> ??????????????? it was decided which >>>>> ??????????????? approach to take. >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? thanks, >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? Chris >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????????? On 11/20/18 2:14 PM, >>>>> ??????????????? JC Beyler wrote: >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Hi all, >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Chris thought it made >>>>> ??????????????? sense to have more >>>>> ??????????????? >? ? ?eyes on >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? this change than just >>>>> ??????????????? serviceability as it will >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? modify to tests that >>>>> ??????????????? are not only >>>>> ??????????????? >? ? ?serviceability >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? tests so I've moved >>>>> ??????????????? this to conversation >>>>> ??????????????? >? ? ?here :) >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? For convenience, I've >>>>> ??????????????? copy-pasted the >>>>> ??????????????? >? ? ?initial RFR: >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Could I have a review >>>>> ??????????????? for the extension and >>>>> ??????????????? >? ? ?usage >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? of the >>>>> ??????????????? ExceptionJniWrapper. This adds lines and >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? filenames to the end >>>>> ??????????????? of the wrapper JNI >>>>> ??????????????? >? ? ?methods, >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? adds tracing, and >>>>> ??????????????? throws an error if need >>>>> ??????????????? >? ? ?be. I've >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ported the gc/lock >>>>> ??????????????? files to use the new >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? TRACE_JNI_CALL add-on >>>>> ??????????????? and I've ported a few >>>>> ??????????????? >? ? ?of the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? tests that were >>>>> ??????????????? already changed for the >>>>> ??????????????? >? ? ?assignment >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? webrev for >>>>> JDK-8212884. >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Webrev: >>>>> ??????????????? >? ? ? >>>>>>> >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>>> >>>>> ??????????????? >? ? ? >>>>>>> >>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Bug: >>>>> ??????????????? >? ? ? >>>>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? For illustration, if >>>>> ??????????????? I force an error to the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? AP04/ap04t03 test and >>>>> ??????????????? set the verbosity on, >>>>> ??????????????? >? ? ?I get >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? something like: >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? >> Calling JNI method >>>>> ??????????????? FindClass from >>>>> ??????????????? >? ? ? >>>>>>> ap04t003.cpp:343 >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? >> Calling with these >>>>> ??????????????? parameter(s): >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? >>>>> java/lang/Threadd >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Wait for thread to >>>>> finish >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? << Called JNI method >>>>> ??????????????? FindClass from >>>>> ??????????????? >? ? ? >>>>>>> ap04t003.cpp:343 >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Exception in thread >>>>> ??????????????? "Thread-0" >>>>> ??????????????? >? ? ? >>>>>>> java.lang.NoClassDefFoundError: >>>>> ??????????????? >? ? ?java/lang/Threadd >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Method) >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Caused by: >>>>> ??????????????? java.lang.ClassNotFoundException: >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? java.lang.Threadd >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? ... 3 more >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? FATAL ERROR in native >>>>> ??????????????? method: JNI method >>>>> ??????????????? >? ? ?FindClass >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? : internal error from >>>>> ??????????????? ap04t003.cpp:343 >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Method) >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Questions/comments I >>>>> ??????????????? have about this are: >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? - Do we want to >>>>> ??????????????? force fatal errors when a JNI >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? call fails in >>>>> ??????????????? general? Most of these tests >>>>> ??????????????? >? ? ?do the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? right thing and test >>>>> ??????????????? the return of the JNI >>>>> ??????????????? >? ? ?calls, >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? for example: >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? thrClass = >>>>> ??????????????? > ?jni->FindClass("java/lang/Threadd", >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? TRACE_JNI_CALL); >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? if (thrClass == >>>>> ??????????????? NULL) { >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? but now the wrapper >>>>> ??????????????? actually would do a >>>>> ??????????????? >? ? ?fatal if >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? the FindClass call >>>>> ??????????????? would return a nullptr, >>>>> ??????????????? >? ? ?so we >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? could remove that >>>>> ??????????????? test altogether. What do you >>>>> ??????????????? >? ? ? >>>>>>> think? >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? - I prefer to >>>>> ??????????????? leave them as the tests then >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? become closer to what >>>>> ??????????????? real users would have in >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? their code and is the >>>>> ??????????????? "recommended" way of >>>>> ??????????????? >? ? ?doing it >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ?- The alternative >>>>> ??????????????? is to use the >>>>> ??????????????? >? ? ?NonFatalError I >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? added which then just >>>>> ??????????????? prints out that something >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? went wrong, letting >>>>> ??????????????? the test continue. Question >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? will be what should >>>>> ??????????????? be the default? The >>>>> ??????????????? >? ? ?fatal or >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? the non-fatal error >>>>> ??????????????? handling? >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? On a different >>>>> subject: >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? - On the new tests, >>>>> ??????????????? I've removed the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? NSK_JNI_VERIFY since >>>>> ??????????????? the JNI wrapper >>>>> ??????????????? >? ? ?handles the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? tracing and the >>>>> ??????????????? verify in almost the same >>>>> ??????????????? >? ? ?way; only >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? difference I can >>>>> ??????????????? really tell is that the >>>>> ??????????????? >? ? ?complain >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? method from NSK has a >>>>> ??????????????? max complain before >>>>> ??????????????? >? ? ?stopping >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? to "complain"; I have >>>>> ??????????????? not added that part >>>>> ??????????????? >? ? ?of the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? code in this webrev >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Once we decide on >>>>> ??????????????? these, I can continue on the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? files from >>>>> ??????????????? JDK-8212884 and then do both the >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? assignment in an if >>>>> ??????????????? extraction followed-by this >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? type of webrev in an >>>>> ??????????????? easier fashion. >>>>> ??????????????? >? ? ?Depending on >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? decisions here, >>>>> ??????????????? NSK*VERIFY can be deprecated as >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? well as we go forward. >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Thanks! >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Jc >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? On Mon, Nov 19, 2018 >>>>> ??????????????? at 11:34 AM Chris Plummer >>>>> ??????????????? > >>>>>>> ??????????????? >>>> ??????????????? >>>>> ??????????????? > ?>>>> > >>>>> ??????????????? >? ? ? >>>>>>> >>>> ??????????????? >>>>> ??????????????? > ?>>>> >>> wrote: >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????????? On 11/19/18 10:07 >>>>> ??????????????? AM, JC Beyler wrote: >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Hi all, >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? @David/Chris: >>>>> ??????????????? should I then push this >>>>> ??????????????? >? ? ?RFR to >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? the hotspot >>>>> ??????????????? mailing or the runtime >>>>> ??????????????? >? ? ?one? For >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? what it's worth, >>>>> ??????????????? a lot of the tests >>>>> ??????????????? >? ? ?under the >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? vmTestbase are >>>>> ??????????????? jvmti so the review also >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? affects >>>>> ??????????????? serviceability; it just turns >>>>> ??????????????? >? ? ?out I >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? started with the >>>>> ??????????????? GC originally and >>>>> ??????????????? >? ? ?then hit >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? some other tests >>>>> ??????????????? I had touched via the >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? assignment >>>>> ??????????????? extraction. >>>>> ??????????????? >? ? ? >>>>>>> ??????????????????? I think hotspot >>>>> ??????????????? would be best. >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????????? Chris >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? @Serguei: Done >>>>> ??????????????? for the method >>>>> ??????????????? >? ? ?renaming, for >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? the indent, are >>>>> ??????????????? you talking about >>>>> ??????????????? >? ? ?going from >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? the 8-indent to >>>>> ??????????????? 4-indent? If so, would >>>>> ??????????????? >? ? ?it not >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? just be better >>>>> ??????????????? to do a new JBS bug and >>>>> ??????????????? >? ? ?do the >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? whole files in >>>>> ??????????????? one go? I ask because >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? otherwise, it >>>>> ??????????????? will look a bit weird to >>>>> ??????????????? >? ? ?have >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? parts of the >>>>> ??????????????? file as 8-indent and others >>>>> ??????????????? >? ? ? >>>>>>>> 4-indent? >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Thanks for >>>>> ??????????????? looking at it! >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Jc >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? On Mon, Nov 19, >>>>> ??????????????? 2018 at 1:25 AM >>>>> ??????????????? >? ? ? >>>>>>>> serguei.spitsyn at oracle.com >>>>> ??????????????? >>>>> ??????????????? >>>> > >>>>> ??????????????? >? ? ? >>>>>>>> >>>> ??????????????? >>>>> ??????????????? > ?>>>> >> >>>>> ??????????????? > >>>>>>>> ??????????????? >>>>> >>>> ??????????????? >>>>> ??????????????? > ?>>>> > >>>>> ??????????????? >? ? ? >>>>>>>> >>>> ??????????????? >>>>> ??????????????? > ?>>>> >>> wrote: >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? Hi Jc, >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? We have to >>>>> ??????????????? start this review >>>>> ??????????????? >? ? ?anyway. :) >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? It looks >>>>> ??????????????? good to me in general. >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? Thank you >>>>> ??????????????? for your consistency in this >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? refactoring! >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? Some minor >>>>> ??????????????? comments. >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? > >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>> ??????????????? > >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? +static >>>>> ??????????????? const char* >>>>> ??????????????? >? ? ?remove_folders(const >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? char* >>>>> ??????????????? fullname) { I'd suggest to >>>>> ??????????????? >? ? ?rename >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? the function >>>>> ??????????????? name to something >>>>> ??????????????? >? ? ?traditional >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? like >>>>> ??????????????? get_basename. Otherwise, it >>>>> ??????????????? >? ? ?sounds >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? like this >>>>> ??????????????? function has to really >>>>> ??????????????? >? ? ?remove >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? folders. :) >>>>> ??????????????? Also, all *Locker.cpp have >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? wrong indent >>>>> ??????????????? in the bodies of if >>>>> ??????????????? >? ? ?and while >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? statements. >>>>> ??????????????? Could this be fixed >>>>> ??????????????? >? ? ?with the >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? refactoring? >>>>> ??????????????? I did not look on how >>>>> ??????????????? >? ? ?this >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? impacts the >>>>> ??????????????? tests other than >>>>> ??????????????? > >>>>>>>> ??????????????? serviceability. Thanks, >>>>> Serguei >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? On 11/16/18 >>>>> ??????????????? 19:43, JC Beyler wrote: >>>>> ??????????????? > >>>>>>>>> ??????????????????????? Hi all, >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????? Anybody >>>>> ??????????????? motivated to review this? :) >>>>> ??????????????? > >>>>>>>>> ??????????????????????? Jc >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????? On Wed, Nov 7, >>>>> ??????????????? 2018 at 9:53 PM JC >>>>> ??????????????? >? ? ?Beyler >>>>> ??????????????? > >>>>>>>>> ??????????????? >>>> >>>>> ??????????????? >? ? ?>>>> ??????????????? > >>>>> ??????????????? > >>>>>>>>> >>>> ??????????????? >>>>> ??????????????? >? ? ?>>>> >>> wrote: >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Hi all, >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Could I have >>>>> ??????????????? a review for the >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? extension >>>>> ??????????????? and usage of the >>>>> ??????????????? > >>>>>>>>> ExceptionJniWrapper. This >>>>> ??????????????? >? ? ?adds lines >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? and >>>>> ??????????????? filenames to the end of the >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? wrapper JNI >>>>> ??????????????? methods, adds >>>>> ??????????????? >? ? ?tracing, >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? and throws >>>>> ??????????????? an error if need >>>>> ??????????????? >? ? ?be. I've >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ported the >>>>> ??????????????? gc/lock files to >>>>> ??????????????? >? ? ?use the >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? new >>>>> ??????????????? TRACE_JNI_CALL add-on and >>>>> ??????????????? >? ? ?I've >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ported a few >>>>> ??????????????? of the tests >>>>> ??????????????? >? ? ?that were >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? already >>>>> ??????????????? changed for the >>>>> ??????????????? >? ? ?assignment >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? webrev for >>>>> ??????????????? JDK-8212884. >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Webrev: >>>>> ??????????????? > >>>>>>>>> >>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>> >>>>> ??????????????? > >>>>>>>>> >>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Bug: >>>>> ??????????????? > >>>>>>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? For >>>>> ??????????????? illustration, if I force >>>>> ??????????????? >? ? ?an error >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? to the >>>>> ??????????????? AP04/ap04t03 test and >>>>> ??????????????? >? ? ?set the >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? verbosity >>>>> ??????????????? on, I get something >>>>> ??????????????? >? ? ?like: >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? >> Calling >>>>> ??????????????? JNI method >>>>> ??????????????? >? ? ?FindClass from >>>>> ??????????????? > >>>>>>>>> ap04t003.cpp:343 >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? >> Calling >>>>> ??????????????? with these >>>>> ??????????????? >? ? ?parameter(s): >>>>> ??????????????? > >>>>>>>>> java/lang/Threadd >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Wait for >>>>> ??????????????? thread to finish >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? << Called >>>>> ??????????????? JNI method >>>>> ??????????????? >? ? ?FindClass from >>>>> ??????????????? > >>>>>>>>> ap04t003.cpp:343 >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Exception in >>>>> ??????????????? thread "Thread-0" >>>>> ??????????????? > >>>>>>>>> java.lang.NoClassDefFoundError: >>>>> ??????????????? > >>>>>>>>> java/lang/Threadd >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Method) >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> ??????????????? > >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> ??????????????? > >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Caused by: >>>>> ??????????????? > >>>>>>>>> java.lang.ClassNotFoundException: >>>>> ??????????????? > >>>>>>>>> java.lang.Threadd >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>> ??????????????? > >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>> ??????????????? > >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ... 3 more >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? FATAL ERROR >>>>> ??????????????? in native method: JNI >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? method >>>>> ??????????????? FindClass : internal error >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? from >>>>> ??????????????? ap04t003.cpp:343 >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Method) >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> ??????????????? > >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> ??????????????? > >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> Questions/comments I have about >>>>> ??????????????? > >>>>>>>>> this are: >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? - Do we >>>>> ??????????????? want to force fatal >>>>> ??????????????? >? ? ?errors >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? when a JNI >>>>> ??????????????? call fails in general? >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Most of >>>>> ??????????????? these tests do the right >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? thing and >>>>> ??????????????? test the return of >>>>> ??????????????? >? ? ?the JNI >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? calls, for >>>>> ??????????????? example: >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? thrClass = >>>>> ??????????????? > >>>>>>>>> jni->FindClass("java/lang/Threadd", >>>>> ??????????????? > >>>>>>>>> TRACE_JNI_CALL); >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? ? if >>>>> ??????????????? (thrClass == NULL) { >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? but now the >>>>> ??????????????? wrapper actually >>>>> ??????????????? >? ? ?would do >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? a fatal if >>>>> ??????????????? the FindClass call >>>>> ??????????????? >? ? ?would >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? return a >>>>> ??????????????? nullptr, so we could >>>>> ??????????????? >? ? ?remove >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? that test >>>>> ??????????????? altogether. What do >>>>> ??????????????? >? ? ?you >>>>> ??????????????? > >>>>>>>>> think? >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? ? - I >>>>> ??????????????? prefer to leave them >>>>> ??????????????? >? ? ?as the >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? tests then >>>>> ??????????????? become closer to >>>>> ??????????????? >? ? ?what real >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? users would >>>>> ??????????????? have in their >>>>> ??????????????? >? ? ?code and is >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? the >>>>> ??????????????? "recommended" way of doing it >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? ?- The >>>>> ??????????????? alternative is to >>>>> ??????????????? >? ? ?use the >>>>> ??????????????? > >>>>>>>>> ??????????????? NonFatalError I added >>>>> which >>>>> ??????????????? >? ? ?then just >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? prints out >>>>> ??????????????? that something >>>>> ??????????????? >? ? ?went wrong, >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? letting the >>>>> ??????????????? test continue. >>>>> ??????????????? >? ? ?Question >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? will be what >>>>> ??????????????? should be the >>>>> ??????????????? >? ? ?default? >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? The fatal or >>>>> ??????????????? the non-fatal error >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? handling? >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? On a >>>>> ??????????????? different subject: >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? - On the >>>>> ??????????????? new tests, I've >>>>> ??????????????? >? ? ?removed >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? the >>>>> ??????????????? NSK_JNI_VERIFY since the JNI >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? wrapper >>>>> ??????????????? handles the tracing >>>>> ??????????????? >? ? ?and the >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? verify in >>>>> ??????????????? almost the same >>>>> ??????????????? >? ? ?way; only >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? difference I >>>>> ??????????????? can really tell >>>>> ??????????????? >? ? ?is that >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? the complain >>>>> ??????????????? method from NSK >>>>> ??????????????? >? ? ?has a >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? max complain >>>>> ??????????????? before stopping to >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? "complain"; >>>>> ??????????????? I have not added that >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? part of the >>>>> ??????????????? code in this webrev >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Once we >>>>> ??????????????? decide on these, I can >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? continue on >>>>> ??????????????? the files from >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? JDK-8212884 >>>>> ??????????????? and then do both the >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? assignment >>>>> ??????????????? in an if extraction >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? followed-by >>>>> ??????????????? this type of >>>>> ??????????????? >? ? ?webrev in an >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? easier >>>>> ??????????????? fashion. Depending on >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? decisions >>>>> ??????????????? here, NSK*VERIFY can be >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? deprecated >>>>> ??????????????? as well as we go >>>>> ??????????????? >? ? ?forward. >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Thank you >>>>> ??????????????? for the >>>>> ??????????????? >? ? ?reviews/comments :) >>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Jc >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? > >>>>>>>>> ??????????????????????? -- >>>>> ??????????????? > >>>>>>>>> ??????????????????????? Thanks, >>>>> ??????????????? > >>>>>>>>> ??????????????????????? Jc >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? -- >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Thanks, >>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Jc >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? -- >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Thanks, >>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Jc >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> >>>>> ??????????????? >? ? ? >>>>>> ??????????? -- >>>>> ??????????????? >? ? ? >>>>>> ??????????? Thanks, >>>>> ??????????????? >? ? ? >>>>>> ??????????? Jc >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >? ? ? >>>>> ??????? -- >>>>> ??????????????? >? ? ? >>>>> Thanks, >>>>> ??????????????? >? ? ? >>>>> ??????? Jc >>>>> ??????????????? >? ? ? >>>> >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> ??? -- >>>>> ??????????????? >? ? ? >>> ??? Thanks, >>>>> ??????????????? >? ? ? >>> ??? Jc >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> -- >>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >? ? ? >>> Thanks, >>>>> ??????????????? >? ? ? >>> Jc >>>>> ??????????????? >? ? ? > >>>>> ??????????????? >? ? ? > >>>>> ??????????????? > >>>>> ??????????????? > >>>>> ??????????????? > >>>>> ??????????????? > -- >>>>> ??????????????? > >>>>> ??????????????? > Thanks, >>>>> ??????????????? > Jc >>>>> >>>>> >>>>> >>>>> ??????????? -- >>>>> ??????????? Thanks, >>>>> ??????????? Jc >>>> >>>> >>>> >>>> ??????? -- >>>> ??????? Thanks, >>>> ??????? Jc >>> >>> >>> >>> ??? -- >>> ??? Thanks, >>> ??? Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> From david.holmes at oracle.com Wed Dec 12 21:57:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Dec 2018 07:57:10 +1000 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: On 12/12/2018 11:08 pm, Aleksey Shipilev wrote: > On 12/12/18 1:28 PM, David Holmes wrote: >>> ?? http://cr.openjdk.java.net/~shade/8181143/webrev.04/ >> >> Okay I can live with that. :) > > Thanks! > >> Three comments: >> >> 1. We don't use bug id's to name test directories any more (I see a couple? have snuck in this past >> year! so please place the test in a suitable directory ... Safepoints? > > Dropped. The test needs to placed in a different location. > >> 2. -Xmx1g might be an issue on some test systems. Does it have to be that large? > > Yeah, I was trying to make enough objects for Serial to definitely choke up. I think I can trim it > to 256m, let me run it through jdk-submit. > >> 3. Can you disable core dumping as part of the test so we don't get large core files generated. > > Supplying -XX:-CreateCoredumpOnCrash is going to be enough, right? Right. > Did this: > > diff -r e4bf4aa1100c test/hotspot/jtreg/runtime/8181143/TestAbortOnVMOperationTimeout.java > --- a/test/hotspot/jtreg/runtime/8181143/TestAbortOnVMOperationTimeout.java Tue Dec 11 22:45:38 > 2018 +0100 > +++ b/test/hotspot/jtreg/runtime/8181143/TestAbortOnVMOperationTimeout.java Wed Dec 12 13:46:15 > 2018 +0100 Test needs to be in e.g. test/hotspot/jtreg/runtime/Safepoint/TestAbortOnVMOperationTimeout.java > @@ -27,5 +27,4 @@ > /* > * @test TestAbortOnVMOperationTimeout > - * @bug 8181143 This should stay. > * @summary Check abort on VM timeout is working > * @library /test/lib > @@ -38,5 +37,5 @@ > public static void main(String[] args) throws Exception { > if (args.length > 0) { > - Object[] arr = new Object[40_000_000]; > + Object[] arr = new Object[10_000_000]; > for (int i = 0; i < arr.length; i++) { > arr[i] = new Object(); > @@ -51,6 +50,6 @@ > } > > - // These should fail: Serial is not very fast. Traversing 40M objects in 5 ms > - // means less than 0.125 ns per object, which is not doable. > + // These should fail: Serial is not very fast. Traversing 10M objects in 5 ms > + // means less than 0.5 ns per object, which is not doable. > for (int delay : new int[]{0, 1, 5}) { > testWith(delay, false); > @@ -63,6 +62,7 @@ > "-XX:+AbortVMOnVMOperationTimeout", > "-XX:AbortVMOnVMOperationTimeoutDelay=" + delay, > - "-Xmx1g", > + "-Xmx256m", > "-XX:+UseSerialGC", > + "-XX:-CreateCoredumpOnCrash", > "TestAbortOnVMOperationTimeout", > "foo" Other changes look good. Thanks, David > Full webrev: > http://cr.openjdk.java.net/~shade/8181143/webrev.05/ > > -Aleksey > From shade at redhat.com Wed Dec 12 22:07:52 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Dec 2018 23:07:52 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: On 12/12/18 10:57 PM, David Holmes wrote: > On 12/12/2018 11:08 pm, Aleksey Shipilev wrote: >> On 12/12/18 1:28 PM, David Holmes wrote: >>> 1. We don't use bug id's to name test directories any more (I see a couple? have snuck in this past >>> year! so please place the test in a suitable directory ... Safepoints? >> >> Dropped. > > The test needs to placed in a different location. Ah, shucks! I misread the review comment, sorry. End-of-the-year fatigue... Moved the test and reinstated @bug. Updated webrev in-place: http://cr.openjdk.java.net/~shade/8181143/webrev.05/ Thanks, -Aleksey From david.holmes at oracle.com Wed Dec 12 22:23:38 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Dec 2018 08:23:38 +1000 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: <21d2347b-86b9-6333-eb3a-4a249061ee0c@oracle.com> Sorry Aleksey the existing test directory is runtime/Safepoint not Safepoints. Please place the new test there. Thanks, David On 13/12/2018 8:07 am, Aleksey Shipilev wrote: > On 12/12/18 10:57 PM, David Holmes wrote: >> On 12/12/2018 11:08 pm, Aleksey Shipilev wrote: >>> On 12/12/18 1:28 PM, David Holmes wrote: >>>> 1. We don't use bug id's to name test directories any more (I see a couple? have snuck in this past >>>> year! so please place the test in a suitable directory ... Safepoints? >>> >>> Dropped. >> >> The test needs to placed in a different location. > > Ah, shucks! I misread the review comment, sorry. End-of-the-year fatigue... > Moved the test and reinstated @bug. Updated webrev in-place: > http://cr.openjdk.java.net/~shade/8181143/webrev.05/ > > Thanks, > -Aleksey > > From shade at redhat.com Wed Dec 12 22:29:41 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Dec 2018 23:29:41 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: <21d2347b-86b9-6333-eb3a-4a249061ee0c@oracle.com> References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> <21d2347b-86b9-6333-eb3a-4a249061ee0c@oracle.com> Message-ID: On 12/12/18 11:23 PM, David Holmes wrote: > Sorry Aleksey the existing test directory is runtime/Safepoint not Safepoints. Please place the new > test there. /me facepalms Updated in-place: http://cr.openjdk.java.net/~shade/8181143/webrev.05/ -Aleksey From david.holmes at oracle.com Wed Dec 12 22:41:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Dec 2018 08:41:43 +1000 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> <21d2347b-86b9-6333-eb3a-4a249061ee0c@oracle.com> Message-ID: LGTM! Thanks, David On 13/12/2018 8:29 am, Aleksey Shipilev wrote: > On 12/12/18 11:23 PM, David Holmes wrote: >> Sorry Aleksey the existing test directory is runtime/Safepoint not Safepoints. Please place the new >> test there. > > /me facepalms > > Updated in-place: > http://cr.openjdk.java.net/~shade/8181143/webrev.05/ > > -Aleksey > From jesper.wilhelmsson at oracle.com Wed Dec 12 22:58:52 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Wed, 12 Dec 2018 23:58:52 +0100 Subject: CFV: New hotspot Group Member: Chris Plummer Message-ID: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Thanks, /Jesper [1]http://openjdk.java.net/census#hotspot [2]http://openjdk.java.net/groups/#member-vote From Peter.B.Kessler at Oracle.COM Wed Dec 12 23:05:53 2018 From: Peter.B.Kessler at Oracle.COM (Peter B. Kessler) Date: Wed, 12 Dec 2018 15:05:53 -0800 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: <9f63cb6b-69f6-3ad9-3a6b-4eb1636e83e4@Oracle.COM> Vote: yes. ... peter On 12/12/18 02:58 PM, jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From igor.ignatyev at oracle.com Wed Dec 12 23:03:59 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 12 Dec 2018 15:03:59 -0800 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes -- Igor > On Dec 12, 2018, at 2:58 PM, jesper.wilhelmsson at oracle.com wrote: > > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. From vladimir.kozlov at oracle.com Wed Dec 12 23:07:29 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Dec 2018 15:07:29 -0800 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes On 12/12/18 2:58 PM, jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From alexey.menkov at oracle.com Wed Dec 12 23:09:39 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 12 Dec 2018 15:09:39 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <95fe277d-ba6e-4fec-77aa-d1f1051751aa@oracle.com> References: <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> <839301b7-c247-df3b-e485-283e8bb7388b@oracle.com> <95fe277d-ba6e-4fec-77aa-d1f1051751aa@oracle.com> Message-ID: <72bf2f4a-5bf7-98de-5f00-68485072923d@oracle.com> Hm.. I considered unnamed namespaces "C++ style" (and static globals as "C style"). Static globals were deprecated in C++ (but some time ago the deprecation was reverted). --alex On 12/12/2018 13:55, serguei.spitsyn at oracle.com wrote: > Agreed. > > Thanks, > Serguei > > > On 12/12/18 13:52, David Holmes wrote: >> FWIW I think namespaces are overkill in all of this test code and just >> obfuscates things - the declaration is easily missed. A static >> variable in a .cpp is clearly a global variable to the file. >> >> Cheers, >> David >> >> >> >> On 13/12/2018 5:37 am, serguei.spitsyn at oracle.com wrote: >>> Hi Jc, >>> >>> >>> On 12/11/18 21:16, JC Beyler wrote: >>>> Hi all, >>>> >>>> Here is the new webrev with the TEST.groups change. Serguei, let me >>>> know if I convinced you with the static vs anonymous namespaces or >>>> if you'd still rather have a "static" for now :-) >>> >>> >>> What do you think about this post? : >>> https://stackoverflow.com/questions/11623451/static-vs-non-static-variables-in-namespace >>> >>> >>>> >>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> >>> The update looks fine. >>> >>> Thanks, >>> Serguei >>> >>> >>> Thanks, >>> Serguei >>> >>>> >>>> Thanks again for the reviews! >>>> Jc >>>> >>>> On Mon, Dec 10, 2018 at 3:10 PM JC Beyler >>> > wrote: >>>> >>>> ??? Hi Serguei, >>>> >>>> ??? Yes basically it is equivalent :) I can put them in but they are >>>> ??? not required. The norm actually wanted to deprecate it but then >>>> ??? remembered that C compatibility would require the static key-word >>>> ??? for this case [1] >>>> >>>> ??? So, really, they are not required here and will amount to the same >>>> ??? thing: only that file can refer to them and you cannot get to them >>>> ??? without a globally available method to return a pointer to them >>>> ??? (ie same as a static variable in C). >>>> >>>> ??? I can put static if it makes it easier to see but, by being in an >>>> ??? anonymous namespace they are only available for the file's >>>> ??? translation unit. For example: >>>> >>>> ??? $ cat main.cpp >>>> >>>> ??? int totally_global; >>>> ??? static int explictly_static; >>>> >>>> ??? namespace { >>>> ??? int implicitly_static; >>>> ??? } >>>> >>>> ??? void foo(); >>>> ??? int main() { >>>> ??? ? foo(); >>>> ??? } >>>> >>>> ??? $ g++ -O3 main.cpp -c >>>> ??? $ nm main.o >>>> ??? ? ? ? ? ? ? ? ? ?U _GLOBAL_OFFSET_TABLE_ >>>> ??? 0000000000000000 T main >>>> ??? 0000000000000000 B totally_global >>>> ??? ? ? ? ? ? ? ? ? ?U _Z3foov >>>> >>>> ??? As you can see, the static and anonymous namespace variables are >>>> ??? not in the file due to not being used. If you were to use them, >>>> ??? you'd see them show up as something like: >>>> ??? 0000000000000008 b _ZL17explicitly_static >>>> ??? 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE >>>> >>>> ??? Where again, it shows that it is mangling the names so that no >>>> ??? external usage can happen without tinkering. >>>> >>>> ??? Hopefully that helps :-), >>>> ??? Jc >>>> >>>> ??? [1] >>>> http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 >>>> >>>> >>>> ??? On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com >>>> ??? >>> ??? > wrote: >>>> >>>> ??????? Hi Jc, >>>> >>>> ??????? I had little experience with the C++ namespaces. >>>> ??????? My understanding is that static in this context should mean >>>> ??????? internal linkage. >>>> >>>> ??????? Thanks, >>>> ??????? Serguei >>>> >>>> >>>> ??????? On 12/10/18 13:57, JC Beyler wrote: >>>>> ??????? Hi Serguei, >>>>> >>>>> ??????? The variables and functions are in a anonymous namespace; my >>>>> ??????? understanding of C++ is that this is equivalent to putting it >>>>> ??????? as static.Hence, I didn't add them there. Does that make >>>>> sense? >>>>> >>>>> ??????? Thanks! >>>>> ??????? Jc >>>>> >>>>> ??????? On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com >>>>> ??????? >>>>> ??????? >>>> ??????? > wrote: >>>>> >>>>> ??????????? Hi Jc, >>>>> >>>>> ??????????? It looks good in general. >>>>> ??????????? One question though. >>>>> >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>>>> >>>>> >>>>> ??????????? I wonder if the variables and functions have to be static. >>>>> >>>>> ??????????? Thanks, >>>>> ??????????? Serguei >>>>> >>>>> >>>>> ??????????? On 12/5/18 11:36, JC Beyler wrote: >>>>>> ??????????? Hi all, >>>>>> >>>>>> ??????????? My apologies to having to come back for another review >>>>>> ??????????? for this change: I ran into a snag when trying to pull >>>>>> ??????????? the latest changes compared to the base I was working >>>>>> ??????????? on. I basically forgot that there was an issue with >>>>>> ??????????? snprintf and that I had solved it via?JDK-8213622. >>>>>> >>>>>> ??????????? Could I have a new review of this webrev: >>>>>> ??????????? Webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ >>>>>> >>>>>> ??????????? Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> ??????????? Incremental from the port of webrev.03 that got LGTMs: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ >>>>>> >>>>>> >>>>>> ??????????? A few comments on this because it took me a while to get >>>>>> ??????????? things in a state I thought was good: >>>>>> ??????????? ? - I had to implement an itoa method, do we have >>>>>> ??????????? something like that in the test base (remember that >>>>>> ??????????? JDK-8213622 could not use sprintf due to being in the >>>>>> ??????????? test code)? >>>>>> >>>>>> ??????????? ? - The differences here compared to the one you all >>>>>> ??????????? reviewed are: >>>>>> ??????????? ? ? ? - I found that adding to the strlen/memcpy error >>>>>> ??????????? prone and thought that I would try to make it less so. >>>>>> ??????????? If you want to compare, I extended the strlen/memcpy >>>>>> ??????????? with the new format to show you if you prefer [1] >>>>>> ??????????? ? ? ? ? ? ? - Note that the diff between the "old >>>>>> ??????????? extended way from [1]" to the webrev.04 can be found >>>>>> in [2] >>>>>> >>>>>> ??????????? ? ? ?- I added a test to test the exception wrapper in >>>>>> ??????????? tests :); I'm not sure it is deemed useful or not but >>>>>> ??????????? helped me assure myself that I was not doing things >>>>>> ??????????? wrong; you can find the base test file here [3]; should >>>>>> ??????????? we have this or not? (I know that normally we don't add >>>>>> ??????????? tests to vmTestbase but thought this might be an >>>>>> exception) >>>>>> >>>>>> ??????????? Thanks for your help and my apologies for the snag, >>>>>> ??????????? Jc >>>>>> >>>>>> ??????????? [1]: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>>> >>>>>> >>>>>> >>>>>> ??????????? [2]: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 >>>>>> >>>>>> ??????????? [3] >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ??????????? On Mon, Dec 3, 2018 at 11:29 PM David Holmes >>>>>> ??????????? >>>>> ??????????? > wrote: >>>>>> >>>>>> ??????????????? Looks fine to me. >>>>>> >>>>>> ??????????????? Thanks, >>>>>> ??????????????? David >>>>>> >>>>>> ??????????????? On 4/12/2018 4:04 pm, JC Beyler wrote: >>>>>> ??????????????? > Hi both, >>>>>> ??????????????? > >>>>>> ??????????????? > Thanks for the reviews! Since Serguei did not >>>>>> ??????????????? insist on get_basename, I >>>>>> ??????????????? > went for get_dirname since the method is a local >>>>>> ??????????????? static method and won't >>>>>> ??????????????? > have its name start spreading, I think it's ok too. >>>>>> ??????????????? > >>>>>> ??????????????? > For the naming of the local variable, the idea >>>>>> ??????????????? initially was to use the >>>>>> ??????????????? > same name as the local variable for JNIEnv already >>>>>> ??????????????? used to reduce the >>>>>> ??????????????? > code change. Since I'm now adding the line macro >>>>>> ??????????????? at the end anyway, this >>>>>> ??????????????? > does not matter anymore so I converged all local >>>>>> ??????????????? variables to "jni". >>>>>> ??????????????? > >>>>>> ??????????????? > So, without further ado, here is the new version: >>>>>> ??????????????? > Webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >>>>>> >>>>>> ??????????????? > Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> ??????????????? > >>>>>> ??????????????? > This passes the various tests changed by the >>>>>> ??????????????? webrev on my dev machine. >>>>>> ??????????????? > >>>>>> ??????????????? > Let me know what you think, >>>>>> ??????????????? > Jc >>>>>> ??????????????? > >>>>>> ??????????????? > On Mon, Dec 3, 2018 at 8:40 PM >>>>>> ??????????????? serguei.spitsyn at oracle.com >>>>>> ??????????????? >>>>>> ??????????????? > >>>>> > >>>>>> ??????????????? >>>>> ??????????????? >>>>>> ??????????????? > >>>>> >> wrote: >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ?On 12/3/18 20:15, Chris Plummer wrote: >>>>>> ??????????????? >? ? ? > Hi JC, >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? >? ? ? > Overall it looks good. A few naming nits >>>>>> ??????????????? thought: >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? >? ? ? > In bi01t001.cpp, why have you declared the >>>>>> ??????????????? >? ? ?ExceptionCheckingJniEnvPtr >>>>>> ??????????????? >? ? ? > using jni_env(jni). Elsewhere you use >>>>>> ??????????????? jni(jni_env) and rename the >>>>>> ??????????????? >? ? ? > method argument passed in from jni to >>>>>> jni_env. >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? >? ? ? > Related to this, I also noticed in some >>>>>> ??????????????? files that already are using >>>>>> ??????????????? >? ? ? > ExceptionCheckingJniEnvPtr, such as >>>>>> ??????????????? CharArrayCriticalLocker.cpp, you >>>>>> ??????????????? >? ? ? > delcared it as env(jni_env). So that means >>>>>> ??????????????? there are 3 different >>>>>> ??????????????? >? ? ?names >>>>>> ??????????????? >? ? ? > you have used for the >>>>>> ??????????????? ExceptionCheckingJniEnvPtr local variable. >>>>>> ??????????????? >? ? ?They >>>>>> ??????????????? >? ? ? > should be consistent. >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? >? ? ? > Also, can you rename get_basename() to >>>>>> ??????????????? get_dirname()? I know Serguei >>>>>> ??????????????? >? ? ? > suggested get_basename() a while back, but >>>>>> ??????????????? unless "basename" is >>>>>> ??????????????? >? ? ? > commonly used for this purpose, I think >>>>>> ??????????????? "dirname" is more self >>>>>> ??????????????? >? ? ? > explanatory. >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ?In general, I'm Okay with get_dirname(). >>>>>> ??????????????? >? ? ?Just to mention dirname can be both short or >>>>>> ??????????????? full, so it is a little >>>>>> ??????????????? >? ? ?confusing as well. >>>>>> ??????????????? >? ? ?It is the reason why the get_basename() was >>>>>> ??????????????? suggested. >>>>>> ??????????????? >? ? ?However, I do not insist on get_basename() nor >>>>>> ??????????????? get_full_dirname(). :) >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ?Thanks, >>>>>> ??????????????? >? ? ?Serguei >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? > thanks, >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? >? ? ? > Chris >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? >? ? ? > On 12/2/18 10:29 PM, David Holmes wrote: >>>>>> ??????????????? >? ? ? >> Hi Jc, >>>>>> ??????????????? >? ? ? >> >>>>>> ??????????????? >? ? ? >> I've been lurking on this one and have had >>>>>> ??????????????? a look through. I'm okay >>>>>> ??????????????? >? ? ? >> with the FatalError approach for the tests >>>>>> ??????????????? - we don't expect >>>>>> ??????????????? >? ? ?anything >>>>>> ??????????????? >? ? ? >> to go wrong in a well written test in a >>>>>> ??????????????? correctly functioning VM. >>>>>> ??????????????? >? ? ? >> >>>>>> ??????????????? >? ? ? >> Thanks, >>>>>> ??????????????? >? ? ? >> David >>>>>> ??????????????? >? ? ? >> >>>>>> ??????????????? >? ? ? >> >>>>>> ??????????????? >? ? ? >> >>>>>> ??????????????? >? ? ? >> On 3/12/2018 3:24 pm, JC Beyler wrote: >>>>>> ??????????????? >? ? ? >>> Hi all, >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> Would someone on the GC or runtime team >>>>>> ??????????????? be motivated to give >>>>>> ??????????????? >? ? ?this a >>>>>> ??????????????? >? ? ? >>> review? :) >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> It would be much appreciated! >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> Webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>>> >>>>>> ??????????????? >? ? ? >>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> Thanks for your help, >>>>>> ??????????????? >? ? ? >>> Jc >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler >>>>>> ??????????????? >>>>>> ??????????????? >? ? ?>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >>>>>> ??????????????? >>>>> >>> wrote: >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??? Hi Chris, >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??? Yes I was waiting for another review >>>>>> ??????????????? since you had explicitly >>>>>> ??????????????? >? ? ? >>> asked :) >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??? And sounds good that when someone >>>>>> ??????????????? from GC or runtime gives a >>>>>> ??????????????? >? ? ? >>> review, >>>>>> ??????????????? >? ? ? >>> ??? I'll wait for your full review on the >>>>>> ??????????????? webrev.02! >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??? Thanks again for?your help, >>>>>> ??????????????? >? ? ? >>> ??? Jc >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??? On Tue, Nov 27, 2018 at 12:48 PM >>>>>> ??????????????? Chris Plummer >>>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >>>>>> ??????????????? >>>>> > >>>>>> ??????????????? > ?>>>>> ??????????????? >>>>>> ??????????????? >>>>> >>> >>>>>> ??????????????? >? ? ?wrote: >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??????? Hi JC, >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??????? I think it would be good to get a >>>>>> ??????????????? review from the gc or >>>>>> ??????????????? >? ? ?runtime >>>>>> ??????????????? >? ? ? >>> ??????? teams, since this also affects >>>>>> ??????????????? their tests. >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??????? Also, once we are settled on this >>>>>> ??????????????? FatalError approach, >>>>>> ??????????????? >? ? ?I still >>>>>> ??????????????? >? ? ? >>> ??????? need to give your webrev-02 a >>>>>> ??????????????? full review. I only >>>>>> ??????????????? >? ? ?skimmed over >>>>>> ??????????????? >? ? ? >>> ??????? parts of it (I did look at all >>>>>> ??????????????? the changes in webrevo-01). >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??????? thanks, >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??????? Chris >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??????? On 11/27/18 8:58 AM, >>>>>> ??????????????? serguei.spitsyn at oracle.com >>>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> > >>>>>> ??????????????? >? ? ? >>> >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> >> wrote: >>>>>> ??????????????? >? ? ? >>>> ??????? Hi Jc, >>>>>> ??????????????? >? ? ? >>>> >>>>>> ??????????????? >? ? ? >>>> ??????? I've already reviewed this too. >>>>>> ??????????????? >? ? ? >>>> >>>>>> ??????????????? >? ? ? >>>> ??????? Thanks, >>>>>> ??????????????? >? ? ? >>>> ??????? Serguei >>>>>> ??????????????? >? ? ? >>>> >>>>>> ??????????????? >? ? ? >>>> >>>>>> ??????????????? >? ? ? >>>> ??????? On 11/27/18 06:56, JC Beyler >>>>>> wrote: >>>>>> ??????????????? >? ? ? >>>>> Thanks Chris, >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> Anybody else motivated to look at this >>>>>> ??????????????? and review it? :) >>>>>> ??????????????? >? ? ? >>>>> ??????? Jc >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> ??????? On Mon, Nov 26, 2018 at 1:26 PM >>>>>> ??????????????? Chris Plummer >>>>>> ??????????????? >? ? ? >>>>> >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> > >>>>>> ??????????????? >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> >>> >>>>>> ??????????????? >? ? ? >>>>> wrote: >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> Hi JC, >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> I'm ok with the FatalError approach, >>>>>> ??????????????? but would >>>>>> ??????????????? >? ? ?like to >>>>>> ??????????????? >? ? ? >>>>> hear opinions from others also. >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> thanks, >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> Chris >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Hi Chris, >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Thanks for taking the time >>>>>> ??????????????? to look at it and yes you >>>>>> ??????????????? >? ? ? >>>>>> ??????????? have raised exactly why >>>>>> ??????????????? the webrev is between two >>>>>> ??????????????? >? ? ? >>>>>> ??????????? worlds: in cases where a >>>>>> ??????????????? fatal error on failure is >>>>>> ??????????????? >? ? ? >>>>>> ??????????? wanted, should we simplify >>>>>> ??????????????? the code to remove >>>>>> ??????????????? >? ? ?the return >>>>>> ??????????????? >? ? ? >>>>>> ??????????? tests since we do them >>>>>> ??????????????? internally? Now that I've >>>>>> ??????????????? >? ? ?looked >>>>>> ??????????????? >? ? ? >>>>>> ??????????? around for non-fatal >>>>>> ??????????????? cases, I think the answer >>>>>> ??????????????? >? ? ?is yes, >>>>>> ??????????????? >? ? ? >>>>>> ??????????? it simplifies the code >>>>>> ??????????????? while maintaining the checks. >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? I looked a bit and it >>>>>> ??????????????? seems that I can't find >>>>>> ??????????????? >? ? ?easily a >>>>>> ??????????????? >? ? ? >>>>>> ??????????? case where the test >>>>>> ??????????????? accepts a JNI failure to >>>>>> ??????????????? >? ? ?then move >>>>>> ??????????????? >? ? ? >>>>>> ??????????? on. Therefore, perhaps, >>>>>> ??????????????? for now, the fail with a >>>>>> ??????????????? >? ? ?Fatal >>>>>> ??????????????? >? ? ? >>>>>> ??????????? is enough and we can work >>>>>> ??????????????? on the tests to clean >>>>>> ??????????????? >? ? ?them up? >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? That means that this is >>>>>> ??????????????? the new webrev with only >>>>>> ??????????????? >? ? ?Fatal >>>>>> ??????????????? >? ? ? >>>>>> ??????????? and cleans up the tests so >>>>>> ??????????????? that it is no longer in >>>>>> ??????????????? >? ? ? >>>>>> ??????????? between two worlds: >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Webrev: >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Bug: >>>>>> ??????????????? > https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? (This passes testing on my >>>>>> ??????????????? dev machine for all the >>>>>> ??????????????? >? ? ? >>>>>> ??????????? modified tests) >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? with the example you >>>>>> ??????????????? provided, it now looks like: >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>>> >>>>>> >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? > >>>>>> ? >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Where it does, to me at >>>>>> ??????????????? least, seem cleaner and less >>>>>> ??????????????? >? ? ? >>>>>> ??????????? "noisy". >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Let me know what you think, >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Jc >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? On Tue, Nov 20, 2018 at >>>>>> ??????????????? 9:33 PM Chris Plummer >>>>>> ??????????????? >? ? ? >>>>>> ??????????? >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> > >>>>>> ??????????????? > >>>>>> >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> >>> wrote: >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? Hi JC, >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? Sorry about the delay. >>>>>> ??????????????? I had to go back an >>>>>> ??????????????? >? ? ?look at >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? the initial 8210842 >>>>>> ??????????????? webrev and RFR thread to see >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? what this was >>>>>> ??????????????? initially all about. >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? In general the changes >>>>>> ??????????????? look good. >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? I don't have a good >>>>>> ??????????????? answer to your >>>>>> ??????????????? > >>>>>> FatalError/NonFatalError question. It makes >>>>>> ??????????????? >? ? ?the code >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? a lot cleaner to use >>>>>> ??????????????? FatalError, but then it >>>>>> ??????????????? >? ? ?is a >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? behavior change, and >>>>>> ??????????????? you also need to deal with >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? tests that >>>>>> ??????????????? intentionally induce errors (do >>>>>> ??????????????? >? ? ?you have >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? an example of that). >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? In any case, right now >>>>>> ??????????????? your webrev seems to be >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? between two worlds. >>>>>> ??????????????? You are producing >>>>>> ??????????????? >? ? ?FatalError, >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? but still checking >>>>>> ??????????????? results. Here's a good >>>>>> ??????????????? >? ? ?example: >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>>>>> >>>>>> >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? > >>>>>> ? >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? I'm not sure if this >>>>>> ??????????????? is just a temporary >>>>>> ??????????????? >? ? ?state until >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? it was decided which >>>>>> ??????????????? approach to take. >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? thanks, >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? Chris >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????????? On 11/20/18 2:14 PM, >>>>>> ??????????????? JC Beyler wrote: >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Hi all, >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Chris thought it made >>>>>> ??????????????? sense to have more >>>>>> ??????????????? >? ? ?eyes on >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? this change than just >>>>>> ??????????????? serviceability as it will >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? modify to tests that >>>>>> ??????????????? are not only >>>>>> ??????????????? >? ? ?serviceability >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? tests so I've moved >>>>>> ??????????????? this to conversation >>>>>> ??????????????? >? ? ?here :) >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? For convenience, I've >>>>>> ??????????????? copy-pasted the >>>>>> ??????????????? >? ? ?initial RFR: >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Could I have a review >>>>>> ??????????????? for the extension and >>>>>> ??????????????? >? ? ?usage >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? of the >>>>>> ??????????????? ExceptionJniWrapper. This adds lines and >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? filenames to the end >>>>>> ??????????????? of the wrapper JNI >>>>>> ??????????????? >? ? ?methods, >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? adds tracing, and >>>>>> ??????????????? throws an error if need >>>>>> ??????????????? >? ? ?be. I've >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ported the gc/lock >>>>>> ??????????????? files to use the new >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? TRACE_JNI_CALL add-on >>>>>> ??????????????? and I've ported a few >>>>>> ??????????????? >? ? ?of the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? tests that were >>>>>> ??????????????? already changed for the >>>>>> ??????????????? >? ? ?assignment >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? webrev for >>>>>> JDK-8212884. >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Webrev: >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Bug: >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? For illustration, if >>>>>> ??????????????? I force an error to the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? AP04/ap04t03 test and >>>>>> ??????????????? set the verbosity on, >>>>>> ??????????????? >? ? ?I get >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? something like: >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? >> Calling JNI method >>>>>> ??????????????? FindClass from >>>>>> ??????????????? >? ? ? >>>>>>> ap04t003.cpp:343 >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? >> Calling with these >>>>>> ??????????????? parameter(s): >>>>>> ??????????????? >? ? ? >>>>>>> java/lang/Threadd >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Wait for thread to >>>>>> finish >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? << Called JNI method >>>>>> ??????????????? FindClass from >>>>>> ??????????????? >? ? ? >>>>>>> ap04t003.cpp:343 >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Exception in thread >>>>>> ??????????????? "Thread-0" >>>>>> ??????????????? >? ? ? >>>>>>> java.lang.NoClassDefFoundError: >>>>>> ??????????????? >? ? ?java/lang/Threadd >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Method) >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Caused by: >>>>>> ??????????????? java.lang.ClassNotFoundException: >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? java.lang.Threadd >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? ... 3 more >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? FATAL ERROR in native >>>>>> ??????????????? method: JNI method >>>>>> ??????????????? >? ? ?FindClass >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? : internal error from >>>>>> ??????????????? ap04t003.cpp:343 >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Method) >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? ? ? at >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Questions/comments I >>>>>> ??????????????? have about this are: >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? - Do we want to >>>>>> ??????????????? force fatal errors when a JNI >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? call fails in >>>>>> ??????????????? general? Most of these tests >>>>>> ??????????????? >? ? ?do the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? right thing and test >>>>>> ??????????????? the return of the JNI >>>>>> ??????????????? >? ? ?calls, >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? for example: >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? thrClass = >>>>>> ??????????????? > ?jni->FindClass("java/lang/Threadd", >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? TRACE_JNI_CALL); >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? if (thrClass == >>>>>> ??????????????? NULL) { >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? but now the wrapper >>>>>> ??????????????? actually would do a >>>>>> ??????????????? >? ? ?fatal if >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? the FindClass call >>>>>> ??????????????? would return a nullptr, >>>>>> ??????????????? >? ? ?so we >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? could remove that >>>>>> ??????????????? test altogether. What do you >>>>>> ??????????????? >? ? ? >>>>>>> think? >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ? - I prefer to >>>>>> ??????????????? leave them as the tests then >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? become closer to what >>>>>> ??????????????? real users would have in >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? their code and is the >>>>>> ??????????????? "recommended" way of >>>>>> ??????????????? >? ? ?doing it >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? ?- The alternative >>>>>> ??????????????? is to use the >>>>>> ??????????????? >? ? ?NonFatalError I >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? added which then just >>>>>> ??????????????? prints out that something >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? went wrong, letting >>>>>> ??????????????? the test continue. Question >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? will be what should >>>>>> ??????????????? be the default? The >>>>>> ??????????????? >? ? ?fatal or >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? the non-fatal error >>>>>> ??????????????? handling? >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? On a different >>>>>> subject: >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? ? - On the new tests, >>>>>> ??????????????? I've removed the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? NSK_JNI_VERIFY since >>>>>> ??????????????? the JNI wrapper >>>>>> ??????????????? >? ? ?handles the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? tracing and the >>>>>> ??????????????? verify in almost the same >>>>>> ??????????????? >? ? ?way; only >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? difference I can >>>>>> ??????????????? really tell is that the >>>>>> ??????????????? >? ? ?complain >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? method from NSK has a >>>>>> ??????????????? max complain before >>>>>> ??????????????? >? ? ?stopping >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? to "complain"; I have >>>>>> ??????????????? not added that part >>>>>> ??????????????? >? ? ?of the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? code in this webrev >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Once we decide on >>>>>> ??????????????? these, I can continue on the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? files from >>>>>> ??????????????? JDK-8212884 and then do both the >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? assignment in an if >>>>>> ??????????????? extraction followed-by this >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? type of webrev in an >>>>>> ??????????????? easier fashion. >>>>>> ??????????????? >? ? ?Depending on >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? decisions here, >>>>>> ??????????????? NSK*VERIFY can be deprecated as >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? well as we go forward. >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Thanks! >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Jc >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? On Mon, Nov 19, 2018 >>>>>> ??????????????? at 11:34 AM Chris Plummer >>>>>> ??????????????? > >>>>>>> ??????????????? >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> > >>>>>> ??????????????? >? ? ? >>>>>>> >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> >>> wrote: >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????????? On 11/19/18 10:07 >>>>>> ??????????????? AM, JC Beyler wrote: >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Hi all, >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? @David/Chris: >>>>>> ??????????????? should I then push this >>>>>> ??????????????? >? ? ?RFR to >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? the hotspot >>>>>> ??????????????? mailing or the runtime >>>>>> ??????????????? >? ? ?one? For >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? what it's worth, >>>>>> ??????????????? a lot of the tests >>>>>> ??????????????? >? ? ?under the >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? vmTestbase are >>>>>> ??????????????? jvmti so the review also >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? affects >>>>>> ??????????????? serviceability; it just turns >>>>>> ??????????????? >? ? ?out I >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? started with the >>>>>> ??????????????? GC originally and >>>>>> ??????????????? >? ? ?then hit >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? some other tests >>>>>> ??????????????? I had touched via the >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? assignment >>>>>> ??????????????? extraction. >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????????? I think hotspot >>>>>> ??????????????? would be best. >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????????? Chris >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? @Serguei: Done >>>>>> ??????????????? for the method >>>>>> ??????????????? >? ? ?renaming, for >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? the indent, are >>>>>> ??????????????? you talking about >>>>>> ??????????????? >? ? ?going from >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? the 8-indent to >>>>>> ??????????????? 4-indent? If so, would >>>>>> ??????????????? >? ? ?it not >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? just be better >>>>>> ??????????????? to do a new JBS bug and >>>>>> ??????????????? >? ? ?do the >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? whole files in >>>>>> ??????????????? one go? I ask because >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? otherwise, it >>>>>> ??????????????? will look a bit weird to >>>>>> ??????????????? >? ? ?have >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? parts of the >>>>>> ??????????????? file as 8-indent and others >>>>>> ??????????????? >? ? ? >>>>>>>> 4-indent? >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Thanks for >>>>>> ??????????????? looking at it! >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Jc >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? On Mon, Nov 19, >>>>>> ??????????????? 2018 at 1:25 AM >>>>>> ??????????????? >? ? ? >>>>>>>> serguei.spitsyn at oracle.com >>>>>> ??????????????? >>>>>> ??????????????? >>>>> > >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> >> >>>>>> ??????????????? > >>>>>>>> >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> > >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>> ??????????????? >>>>>> ??????????????? > ?>>>>> >>> wrote: >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? Hi Jc, >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? We have to >>>>>> ??????????????? start this review >>>>>> ??????????????? >? ? ?anyway. :) >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? It looks >>>>>> ??????????????? good to me in general. >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? Thank you >>>>>> ??????????????? for your consistency in this >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? refactoring! >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? Some minor >>>>>> ??????????????? comments. >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? > >>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? +static >>>>>> ??????????????? const char* >>>>>> ??????????????? >? ? ?remove_folders(const >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? char* >>>>>> ??????????????? fullname) { I'd suggest to >>>>>> ??????????????? >? ? ?rename >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? the function >>>>>> ??????????????? name to something >>>>>> ??????????????? >? ? ?traditional >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? like >>>>>> ??????????????? get_basename. Otherwise, it >>>>>> ??????????????? >? ? ?sounds >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? like this >>>>>> ??????????????? function has to really >>>>>> ??????????????? >? ? ?remove >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? folders. :) >>>>>> ??????????????? Also, all *Locker.cpp have >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? wrong indent >>>>>> ??????????????? in the bodies of if >>>>>> ??????????????? >? ? ?and while >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? statements. >>>>>> ??????????????? Could this be fixed >>>>>> ??????????????? >? ? ?with the >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? refactoring? >>>>>> ??????????????? I did not look on how >>>>>> ??????????????? >? ? ?this >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? impacts the >>>>>> ??????????????? tests other than >>>>>> ??????????????? > >>>>>>>> ??????????????? serviceability. Thanks, >>>>>> Serguei >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????????? On 11/16/18 >>>>>> ??????????????? 19:43, JC Beyler wrote: >>>>>> ??????????????? > >>>>>>>>> ??????????????????????? Hi all, >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????? Anybody >>>>>> ??????????????? motivated to review this? :) >>>>>> ??????????????? > >>>>>>>>> ??????????????????????? Jc >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????? On Wed, Nov 7, >>>>>> ??????????????? 2018 at 9:53 PM JC >>>>>> ??????????????? >? ? ?Beyler >>>>>> ??????????????? > >>>>>>>>> ??????????????? >>>>> >>>>>> ??????????????? >? ? ?>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>>>>> >>>>> ??????????????? >>>>>> ??????????????? >? ? ?>>>>> >>> wrote: >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Hi all, >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Could I have >>>>>> ??????????????? a review for the >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? extension >>>>>> ??????????????? and usage of the >>>>>> ??????????????? > >>>>>>>>> ExceptionJniWrapper. This >>>>>> ??????????????? >? ? ?adds lines >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? and >>>>>> ??????????????? filenames to the end of the >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? wrapper JNI >>>>>> ??????????????? methods, adds >>>>>> ??????????????? >? ? ?tracing, >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? and throws >>>>>> ??????????????? an error if need >>>>>> ??????????????? >? ? ?be. I've >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ported the >>>>>> ??????????????? gc/lock files to >>>>>> ??????????????? >? ? ?use the >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? new >>>>>> ??????????????? TRACE_JNI_CALL add-on and >>>>>> ??????????????? >? ? ?I've >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ported a few >>>>>> ??????????????? of the tests >>>>>> ??????????????? >? ? ?that were >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? already >>>>>> ??????????????? changed for the >>>>>> ??????????????? >? ? ?assignment >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? webrev for >>>>>> ??????????????? JDK-8212884. >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Webrev: >>>>>> ??????????????? > >>>>>>>>> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>>> >>>>>> ??????????????? > >>>>>>>>> >>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Bug: >>>>>> ??????????????? > >>>>>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? For >>>>>> ??????????????? illustration, if I force >>>>>> ??????????????? >? ? ?an error >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? to the >>>>>> ??????????????? AP04/ap04t03 test and >>>>>> ??????????????? >? ? ?set the >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? verbosity >>>>>> ??????????????? on, I get something >>>>>> ??????????????? >? ? ?like: >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? >> Calling >>>>>> ??????????????? JNI method >>>>>> ??????????????? >? ? ?FindClass from >>>>>> ??????????????? > >>>>>>>>> ap04t003.cpp:343 >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? >> Calling >>>>>> ??????????????? with these >>>>>> ??????????????? >? ? ?parameter(s): >>>>>> ??????????????? > >>>>>>>>> java/lang/Threadd >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Wait for >>>>>> ??????????????? thread to finish >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? << Called >>>>>> ??????????????? JNI method >>>>>> ??????????????? >? ? ?FindClass from >>>>>> ??????????????? > >>>>>>>>> ap04t003.cpp:343 >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Exception in >>>>>> ??????????????? thread "Thread-0" >>>>>> ??????????????? > >>>>>>>>> java.lang.NoClassDefFoundError: >>>>>> ??????????????? > >>>>>>>>> java/lang/Threadd >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Method) >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Caused by: >>>>>> ??????????????? > >>>>>>>>> java.lang.ClassNotFoundException: >>>>>> ??????????????? > >>>>>>>>> java.lang.Threadd >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ... 3 more >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? FATAL ERROR >>>>>> ??????????????? in native method: JNI >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? method >>>>>> ??????????????? FindClass : internal error >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? from >>>>>> ??????????????? ap04t003.cpp:343 >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Method) >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? at >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>> ?nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> Questions/comments I have about >>>>>> ??????????????? > >>>>>>>>> this are: >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? - Do we >>>>>> ??????????????? want to force fatal >>>>>> ??????????????? >? ? ?errors >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? when a JNI >>>>>> ??????????????? call fails in general? >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Most of >>>>>> ??????????????? these tests do the right >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? thing and >>>>>> ??????????????? test the return of >>>>>> ??????????????? >? ? ?the JNI >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? calls, for >>>>>> ??????????????? example: >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? thrClass = >>>>>> ??????????????? > >>>>>>>>> jni->FindClass("java/lang/Threadd", >>>>>> ??????????????? > >>>>>>>>> TRACE_JNI_CALL); >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? ? if >>>>>> ??????????????? (thrClass == NULL) { >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? but now the >>>>>> ??????????????? wrapper actually >>>>>> ??????????????? >? ? ?would do >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? a fatal if >>>>>> ??????????????? the FindClass call >>>>>> ??????????????? >? ? ?would >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? return a >>>>>> ??????????????? nullptr, so we could >>>>>> ??????????????? >? ? ?remove >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? that test >>>>>> ??????????????? altogether. What do >>>>>> ??????????????? >? ? ?you >>>>>> ??????????????? > >>>>>>>>> think? >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? ? - I >>>>>> ??????????????? prefer to leave them >>>>>> ??????????????? >? ? ?as the >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? tests then >>>>>> ??????????????? become closer to >>>>>> ??????????????? >? ? ?what real >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? users would >>>>>> ??????????????? have in their >>>>>> ??????????????? >? ? ?code and is >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? the >>>>>> ??????????????? "recommended" way of doing it >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? ?- The >>>>>> ??????????????? alternative is to >>>>>> ??????????????? >? ? ?use the >>>>>> ??????????????? > >>>>>>>>> ??????????????? NonFatalError I added >>>>>> which >>>>>> ??????????????? >? ? ?then just >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? prints out >>>>>> ??????????????? that something >>>>>> ??????????????? >? ? ?went wrong, >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? letting the >>>>>> ??????????????? test continue. >>>>>> ??????????????? >? ? ?Question >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? will be what >>>>>> ??????????????? should be the >>>>>> ??????????????? >? ? ?default? >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? The fatal or >>>>>> ??????????????? the non-fatal error >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? handling? >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? On a >>>>>> ??????????????? different subject: >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? ? - On the >>>>>> ??????????????? new tests, I've >>>>>> ??????????????? >? ? ?removed >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? the >>>>>> ??????????????? NSK_JNI_VERIFY since the JNI >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? wrapper >>>>>> ??????????????? handles the tracing >>>>>> ??????????????? >? ? ?and the >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? verify in >>>>>> ??????????????? almost the same >>>>>> ??????????????? >? ? ?way; only >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? difference I >>>>>> ??????????????? can really tell >>>>>> ??????????????? >? ? ?is that >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? the complain >>>>>> ??????????????? method from NSK >>>>>> ??????????????? >? ? ?has a >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? max complain >>>>>> ??????????????? before stopping to >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? "complain"; >>>>>> ??????????????? I have not added that >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? part of the >>>>>> ??????????????? code in this webrev >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Once we >>>>>> ??????????????? decide on these, I can >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? continue on >>>>>> ??????????????? the files from >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? JDK-8212884 >>>>>> ??????????????? and then do both the >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? assignment >>>>>> ??????????????? in an if extraction >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? followed-by >>>>>> ??????????????? this type of >>>>>> ??????????????? >? ? ?webrev in an >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? easier >>>>>> ??????????????? fashion. Depending on >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? decisions >>>>>> ??????????????? here, NSK*VERIFY can be >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? deprecated >>>>>> ??????????????? as well as we go >>>>>> ??????????????? >? ? ?forward. >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Thank you >>>>>> ??????????????? for the >>>>>> ??????????????? >? ? ?reviews/comments :) >>>>>> ??????????????? > >>>>>>>>> ??????????????????????????? Jc >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> >>>>>> ??????????????? > >>>>>>>>> ??????????????????????? -- >>>>>> ??????????????? > >>>>>>>>> ??????????????????????? Thanks, >>>>>> ??????????????? > >>>>>>>>> ??????????????????????? Jc >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? -- >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Thanks, >>>>>> ??????????????? >? ? ? >>>>>>>> ??????????????????? Jc >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? -- >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Thanks, >>>>>> ??????????????? >? ? ? >>>>>>> ??????????????? Jc >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> >>>>>> ??????????????? >? ? ? >>>>>> ??????????? -- >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Thanks, >>>>>> ??????????????? >? ? ? >>>>>> ??????????? Jc >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> >>>>>> ??????????????? >? ? ? >>>>> ??????? -- >>>>>> ??????????????? >? ? ? >>>>> Thanks, >>>>>> ??????????????? >? ? ? >>>>> ??????? Jc >>>>>> ??????????????? >? ? ? >>>> >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> ??? -- >>>>>> ??????????????? >? ? ? >>> ??? Thanks, >>>>>> ??????????????? >? ? ? >>> ??? Jc >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> -- >>>>>> ??????????????? >? ? ? >>> >>>>>> ??????????????? >? ? ? >>> Thanks, >>>>>> ??????????????? >? ? ? >>> Jc >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? >? ? ? > >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>> ??????????????? > >>>>>> ??????????????? > -- >>>>>> ??????????????? > >>>>>> ??????????????? > Thanks, >>>>>> ??????????????? > Jc >>>>>> >>>>>> >>>>>> >>>>>> ??????????? -- >>>>>> ??????????? Thanks, >>>>>> ??????????? Jc >>>>> >>>>> >>>>> >>>>> ??????? -- >>>>> ??????? Thanks, >>>>> ??????? Jc >>>> >>>> >>>> >>>> ??? -- >>>> ??? Thanks, >>>> ??? Jc >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> > From david.holmes at oracle.com Wed Dec 12 23:12:58 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Dec 2018 09:12:58 +1000 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: <30608b20-2069-de92-783f-523e2bd07063@oracle.com> Vote: yes David On 13/12/2018 8:58 am, jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From dean.long at oracle.com Wed Dec 12 23:43:58 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 12 Dec 2018 15:43:58 -0800 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: <5440b82e-e94f-2120-04f5-42864ba09fdc@oracle.com> |Vote: yes| From daniel.daugherty at oracle.com Thu Dec 13 00:43:55 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 12 Dec 2018 19:43:55 -0500 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: <5d3c84ef-b6d6-1583-adee-23cd6c5c575f@oracle.com> Vote: yes Dan On 12/12/18 5:58 PM, jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > > From kim.barrett at oracle.com Thu Dec 13 00:59:21 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 12 Dec 2018 19:59:21 -0500 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: <193EA921-8677-46CC-89E7-F24FD12FE5B1@oracle.com> vote: yes > On Dec 12, 2018, at 5:58 PM, jesper.wilhelmsson at oracle.com wrote: > > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote From zhaixiang at loongson.cn Thu Dec 13 02:58:06 2018 From: zhaixiang at loongson.cn (Leslie Zhai) Date: Thu, 13 Dec 2018 10:58:06 +0800 Subject: System.out.println lead to assert(resolved_method->method_holder()->is_linked()) failed: must be linked Message-ID: Hi Coleen, System.out.println was added on purpose: diff -r 8449dc3cbebe src/java.base/share/classes/java/lang/Object.java --- a/src/java.base/share/classes/java/lang/Object.java??? Mon Nov 12 15:34:21 2018 +0800 +++ b/src/java.base/share/classes/java/lang/Object.java??? Thu Dec 13 10:40:33 2018 +0800 @@ -47,7 +47,9 @@ ????? * Constructs a new object. ????? */ ???? @HotSpotIntrinsicCandidate -??? public Object() {} +??? public Object() { +????? System.out.println("DEBUG: on purpose"); +??? } ???? /** ????? * Returns the runtime class of this {@code Object}. The returned ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- Then it lead to the assertion error, similar as: https://bugs.openjdk.java.net/browse/JDK-8069005 $ ./build/linux-x86_64-normal-server-fastdebug/jdk/bin/java -version # To suppress the following error report, specify this argument # after -XX: or in .hotspotrc: SuppressErrorAt=/linkResolver.cpp:1380 # # A fatal error has been detected by the Java Runtime Environment: # #? Internal Error (/home/zhaixiang/jdk/src/hotspot/share/interpreter/linkResolver.cpp:1380), pid=11090, tid=11091 #? assert(resolved_method->method_holder()->is_linked()) failed: must be linked ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- But just add java.lang.Object initialization in bootstrapping the jvm couldn't fix the issue: diff -r 2c5bf37c8f8e src/hotspot/share/runtime/thread.cpp --- a/src/hotspot/share/runtime/thread.cpp??? Thu Dec 06 20:39:03 2018 +0800 +++ b/src/hotspot/share/runtime/thread.cpp??? Thu Dec 13 10:44:56 2018 +0800 @@ -3524,6 +3524,7 @@ ?? // Initialize java_lang.System (needed before creating the thread) ?? initialize_class(vmSymbols::java_lang_System(), CHECK); ?? // The VM creates & returns objects of this class. Make sure it's initialized. +? initialize_class(vmSymbols::java_lang_Object(), CHECK); ?? initialize_class(vmSymbols::java_lang_Class(), CHECK); ?? initialize_class(vmSymbols::java_lang_ThreadGroup(), CHECK); ?? Handle thread_group = create_initial_thread_group(CHECK); Please give me some advice how to fix the issue, thanks a lot! Regards, Leslie Zhai From david.holmes at oracle.com Thu Dec 13 03:26:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Dec 2018 13:26:37 +1000 Subject: System.out.println lead to assert(resolved_method->method_holder()->is_linked()) failed: must be linked In-Reply-To: References: Message-ID: <8f7b3fb8-8908-0b3e-f1c7-6b8519a9b5fb@oracle.com> On 13/12/2018 12:58 pm, Leslie Zhai wrote: > Hi Coleen, > > System.out.println was added on purpose: You can't do that. You can't put any code in the Object constructor (or various other places - static initializers etc) because nothing else is initialized yet - including String, System etc etc. The code will hit exceptions and try to throw but can't because the Exception classes can't be initialized yet. The assert is what we always see when trying to throw an exception before the VM is initialized enough to throw exceptions. Cheers, David ---- > diff -r 8449dc3cbebe src/java.base/share/classes/java/lang/Object.java > --- a/src/java.base/share/classes/java/lang/Object.java??? Mon Nov 12 > 15:34:21 2018 +0800 > +++ b/src/java.base/share/classes/java/lang/Object.java??? Thu Dec 13 > 10:40:33 2018 +0800 > @@ -47,7 +47,9 @@ > ????? * Constructs a new object. > ????? */ > ???? @HotSpotIntrinsicCandidate > -??? public Object() {} > +??? public Object() { > +????? System.out.println("DEBUG: on purpose"); > +??? } > > ???? /** > ????? * Returns the runtime class of this {@code Object}. The returned > > ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- > > Then it lead to the assertion error, similar as: > > https://bugs.openjdk.java.net/browse/JDK-8069005 > > $ ./build/linux-x86_64-normal-server-fastdebug/jdk/bin/java -version > # To suppress the following error report, specify this argument > # after -XX: or in .hotspotrc: SuppressErrorAt=/linkResolver.cpp:1380 > # > # A fatal error has been detected by the Java Runtime Environment: > # > #? Internal Error > (/home/zhaixiang/jdk/src/hotspot/share/interpreter/linkResolver.cpp:1380), > pid=11090, tid=11091 > #? assert(resolved_method->method_holder()->is_linked()) failed: must be > linked > > ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- > > But just add java.lang.Object initialization in bootstrapping the jvm > couldn't fix the issue: > > diff -r 2c5bf37c8f8e src/hotspot/share/runtime/thread.cpp > --- a/src/hotspot/share/runtime/thread.cpp??? Thu Dec 06 20:39:03 2018 > +0800 > +++ b/src/hotspot/share/runtime/thread.cpp??? Thu Dec 13 10:44:56 2018 > +0800 > @@ -3524,6 +3524,7 @@ > ?? // Initialize java_lang.System (needed before creating the thread) > ?? initialize_class(vmSymbols::java_lang_System(), CHECK); > ?? // The VM creates & returns objects of this class. Make sure it's > initialized. > +? initialize_class(vmSymbols::java_lang_Object(), CHECK); > ?? initialize_class(vmSymbols::java_lang_Class(), CHECK); > ?? initialize_class(vmSymbols::java_lang_ThreadGroup(), CHECK); > ?? Handle thread_group = create_initial_thread_group(CHECK); > > Please give me some advice how to fix the issue, thanks a lot! > > Regards, > > Leslie Zhai > > From dean.long at oracle.com Thu Dec 13 04:01:19 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 12 Dec 2018 20:01:19 -0800 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> Message-ID: Nick, can you please confirm that you are covered under the # ARM Ltd. - OpenJDK, MySQL OCA agreement?? I'm guessing Arm Technology China is under ARM Ltd., judging by your email domain, but please confirm. dl On 12/11/18 5:30 PM, dean.long at oracle.com wrote: > I can help you commit it, but first I think another review would be good. > > dl > > On 12/11/18 5:15 PM, Nick Gasson (Arm Technology China) wrote: >>> Hi Nick.? This change looks good to me. >> Thanks Dean. Could you help me to commit this patch if it doesn't >> need other reviews? >> >> Nick >> >> >>> -----Original Message----- >>> From: dean.long at oracle.com >>> Sent: 12 December 2018 06:05 >>> To: Nick Gasson (Arm Technology China) ; hotspot- >>> dev at openjdk.java.net; Boris Ulasevich >>> Cc: nd ; hotspot-compiler-dev at openjdk.java.net compiler >>> ; aarch32-port- >>> dev at openjdk.java.net >>> Subject: Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java >>> fails on >>> ARM32 >>> >>> Hi Nick.? This change looks good to me. >>> >>> dl >>> >>> On 12/2/18 10:06 PM, Nick Gasson wrote: >>>> Hi, >>>> >>>> Could someone please help me review this fix for a test failure on >>>> ARM32: >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 >>>> Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ >>>> >>>> This fixes two assertion failures related to biased locking when C2's >>>> bias inlining feature is disabled (-XX:-OptoBiasInlining): >>>> >>>> MacroAssembler::atomic_cas_bool takes three register arguments plus a >>>> temporary register for use in the CAS loop. If the caller passes >>>> `noreg' as this temporary register (which happens in the >>>> !OptoBiasInlining call path from MacroAssembler::fast_lock) then >>>> MacroAssembler::atomic_cas_bool will default to using LR as a >>>> temporary. But it's possible with C2 that LR is one of the other three >>>> arguments which then triggers an assert_different_registers assertion >>>> failure. Fix this by supplying an additional scratch register to >>>> MacroAssembler::fast_lock if !OptoBiasInlining. >>>> >>>> In the !OptoBiasInlining case MacroAssembler::fast_lock calls >>>> MacroAssembler::biased_locking_enter to handle the case where the mark >>>> word contains the biased lock pattern which fast_lock wouldn't >>>> otherwise see if OptoBiasInlining was true. However in the case that >>>> we fail to acquire the biased lock this falls through to label >>>> `failed' and runs the normal fast_lock code that implicitly assumes >>>> the mark word does not have the bias pattern. This means we can end up >>>> with a BasicLock where _displaced_header contains the biased lock >>>> pattern which is an illegal state and subsequently triggers an >>>> assertion failure in ObjectSynchronizer::fast_exit. >>>> >>>> The right thing to do here is branch to `done' whenever the bias lock >>>> pattern is present and let the runtime handle the failure case.? Also >>>> edited the comment describing MacroAssembler::biased_locking_enter to >>>> more accurately describe what it does (the comment about `slow_case' >>>> being optional is wrong). >>>> >>>> Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this >>>> patch. >>>> >>>> Thanks, >>>> Nick > From jcbeyler at google.com Thu Dec 13 04:06:00 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 20:06:00 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <72bf2f4a-5bf7-98de-5f00-68485072923d@oracle.com> References: <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> <839301b7-c247-df3b-e485-283e8bb7388b@oracle.com> <95fe277d-ba6e-4fec-77aa-d1f1051751aa@oracle.com> <72bf2f4a-5bf7-98de-5f00-68485072923d@oracle.com> Message-ID: So did I Alexey but with David & Serguei preferring static, it seems more reasonable to go down their route :-) So here is the latest webrev with static instead of an anonymous namespace: Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 Let me know what you think, can I get a webrev 06 review? Thanks! Jc On Wed, Dec 12, 2018 at 3:10 PM Alex Menkov wrote: > Hm.. > I considered unnamed namespaces "C++ style" (and static globals as "C > style"). > Static globals were deprecated in C++ (but some time ago the deprecation > was reverted). > > --alex > > On 12/12/2018 13:55, serguei.spitsyn at oracle.com wrote: > > Agreed. > > > > Thanks, > > Serguei > > > > > > On 12/12/18 13:52, David Holmes wrote: > >> FWIW I think namespaces are overkill in all of this test code and just > >> obfuscates things - the declaration is easily missed. A static > >> variable in a .cpp is clearly a global variable to the file. > >> > >> Cheers, > >> David > >> > >> > >> > >> On 13/12/2018 5:37 am, serguei.spitsyn at oracle.com wrote: > >>> Hi Jc, > >>> > >>> > >>> On 12/11/18 21:16, JC Beyler wrote: > >>>> Hi all, > >>>> > >>>> Here is the new webrev with the TEST.groups change. Serguei, let me > >>>> know if I convinced you with the static vs anonymous namespaces or > >>>> if you'd still rather have a "static" for now :-) > >>> > >>> > >>> What do you think about this post? : > >>> > https://stackoverflow.com/questions/11623451/static-vs-non-static-variables-in-namespace > >>> > >>> > >>>> > >>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ > >>>> > >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > >>> > >>> The update looks fine. > >>> > >>> Thanks, > >>> Serguei > >>> > >>> > >>> Thanks, > >>> Serguei > >>> > >>>> > >>>> Thanks again for the reviews! > >>>> Jc > >>>> > >>>> On Mon, Dec 10, 2018 at 3:10 PM JC Beyler >>>> > wrote: > >>>> > >>>> Hi Serguei, > >>>> > >>>> Yes basically it is equivalent :) I can put them in but they are > >>>> not required. The norm actually wanted to deprecate it but then > >>>> remembered that C compatibility would require the static key-word > >>>> for this case [1] > >>>> > >>>> So, really, they are not required here and will amount to the same > >>>> thing: only that file can refer to them and you cannot get to them > >>>> without a globally available method to return a pointer to them > >>>> (ie same as a static variable in C). > >>>> > >>>> I can put static if it makes it easier to see but, by being in an > >>>> anonymous namespace they are only available for the file's > >>>> translation unit. For example: > >>>> > >>>> $ cat main.cpp > >>>> > >>>> int totally_global; > >>>> static int explictly_static; > >>>> > >>>> namespace { > >>>> int implicitly_static; > >>>> } > >>>> > >>>> void foo(); > >>>> int main() { > >>>> foo(); > >>>> } > >>>> > >>>> $ g++ -O3 main.cpp -c > >>>> $ nm main.o > >>>> U _GLOBAL_OFFSET_TABLE_ > >>>> 0000000000000000 T main > >>>> 0000000000000000 B totally_global > >>>> U _Z3foov > >>>> > >>>> As you can see, the static and anonymous namespace variables are > >>>> not in the file due to not being used. If you were to use them, > >>>> you'd see them show up as something like: > >>>> 0000000000000008 b _ZL17explicitly_static > >>>> 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE > >>>> > >>>> Where again, it shows that it is mangling the names so that no > >>>> external usage can happen without tinkering. > >>>> > >>>> Hopefully that helps :-), > >>>> Jc > >>>> > >>>> [1] > >>>> http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 > >>>> > >>>> > >>>> On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com > >>>> >>>> > wrote: > >>>> > >>>> Hi Jc, > >>>> > >>>> I had little experience with the C++ namespaces. > >>>> My understanding is that static in this context should mean > >>>> internal linkage. > >>>> > >>>> Thanks, > >>>> Serguei > >>>> > >>>> > >>>> On 12/10/18 13:57, JC Beyler wrote: > >>>>> Hi Serguei, > >>>>> > >>>>> The variables and functions are in a anonymous namespace; my > >>>>> understanding of C++ is that this is equivalent to putting it > >>>>> as static.Hence, I didn't add them there. Does that make > >>>>> sense? > >>>>> > >>>>> Thanks! > >>>>> Jc > >>>>> > >>>>> On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com > >>>>> > >>>>> >>>>> > wrote: > >>>>> > >>>>> Hi Jc, > >>>>> > >>>>> It looks good in general. > >>>>> One question though. > >>>>> > >>>>> > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > >>>>> > >>>>> > >>>>> I wonder if the variables and functions have to be > static. > >>>>> > >>>>> Thanks, > >>>>> Serguei > >>>>> > >>>>> > >>>>> On 12/5/18 11:36, JC Beyler wrote: > >>>>>> Hi all, > >>>>>> > >>>>>> My apologies to having to come back for another review > >>>>>> for this change: I ran into a snag when trying to pull > >>>>>> the latest changes compared to the base I was working > >>>>>> on. I basically forgot that there was an issue with > >>>>>> snprintf and that I had solved it via JDK-8213622. > >>>>>> > >>>>>> Could I have a new review of this webrev: > >>>>>> Webrev: > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ > >>>>>> > >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> Incremental from the port of webrev.03 that got LGTMs: > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ > >>>>>> > >>>>>> > >>>>>> A few comments on this because it took me a while to get > >>>>>> things in a state I thought was good: > >>>>>> - I had to implement an itoa method, do we have > >>>>>> something like that in the test base (remember that > >>>>>> JDK-8213622 could not use sprintf due to being in the > >>>>>> test code)? > >>>>>> > >>>>>> - The differences here compared to the one you all > >>>>>> reviewed are: > >>>>>> - I found that adding to the strlen/memcpy error > >>>>>> prone and thought that I would try to make it less so. > >>>>>> If you want to compare, I extended the strlen/memcpy > >>>>>> with the new format to show you if you prefer [1] > >>>>>> - Note that the diff between the "old > >>>>>> extended way from [1]" to the webrev.04 can be found > >>>>>> in [2] > >>>>>> > >>>>>> - I added a test to test the exception wrapper in > >>>>>> tests :); I'm not sure it is deemed useful or not but > >>>>>> helped me assure myself that I was not doing things > >>>>>> wrong; you can find the base test file here [3]; should > >>>>>> we have this or not? (I know that normally we don't add > >>>>>> tests to vmTestbase but thought this might be an > >>>>>> exception) > >>>>>> > >>>>>> Thanks for your help and my apologies for the snag, > >>>>>> Jc > >>>>>> > >>>>>> [1]: > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > >>>>>> > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html> > > >>>>>> > >>>>>> [2]: > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 > >>>>>> > >>>>>> [3] > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > >>>>>> > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html> > > >>>>>> > >>>>>> > >>>>>> On Mon, Dec 3, 2018 at 11:29 PM David Holmes > >>>>>> >>>>>> > wrote: > >>>>>> > >>>>>> Looks fine to me. > >>>>>> > >>>>>> Thanks, > >>>>>> David > >>>>>> > >>>>>> On 4/12/2018 4:04 pm, JC Beyler wrote: > >>>>>> > Hi both, > >>>>>> > > >>>>>> > Thanks for the reviews! Since Serguei did not > >>>>>> insist on get_basename, I > >>>>>> > went for get_dirname since the method is a local > >>>>>> static method and won't > >>>>>> > have its name start spreading, I think it's ok > too. > >>>>>> > > >>>>>> > For the naming of the local variable, the idea > >>>>>> initially was to use the > >>>>>> > same name as the local variable for JNIEnv already > >>>>>> used to reduce the > >>>>>> > code change. Since I'm now adding the line macro > >>>>>> at the end anyway, this > >>>>>> > does not matter anymore so I converged all local > >>>>>> variables to "jni". > >>>>>> > > >>>>>> > So, without further ado, here is the new version: > >>>>>> > Webrev: > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ > >>>>>> > >>>>>> > Bug: > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> > > >>>>>> > This passes the various tests changed by the > >>>>>> webrev on my dev machine. > >>>>>> > > >>>>>> > Let me know what you think, > >>>>>> > Jc > >>>>>> > > >>>>>> > On Mon, Dec 3, 2018 at 8:40 PM > >>>>>> serguei.spitsyn at oracle.com > >>>>>> > >>>>>> > >>>>>> > > >>>>>> >>>>>> > >>>>>> > >>>>>> >> wrote: > >>>>>> > > >>>>>> > On 12/3/18 20:15, Chris Plummer wrote: > >>>>>> > > Hi JC, > >>>>>> > > > >>>>>> > > Overall it looks good. A few naming nits > >>>>>> thought: > >>>>>> > > > >>>>>> > > In bi01t001.cpp, why have you declared the > >>>>>> > ExceptionCheckingJniEnvPtr > >>>>>> > > using jni_env(jni). Elsewhere you use > >>>>>> jni(jni_env) and rename the > >>>>>> > > method argument passed in from jni to > >>>>>> jni_env. > >>>>>> > > > >>>>>> > > Related to this, I also noticed in some > >>>>>> files that already are using > >>>>>> > > ExceptionCheckingJniEnvPtr, such as > >>>>>> CharArrayCriticalLocker.cpp, you > >>>>>> > > delcared it as env(jni_env). So that means > >>>>>> there are 3 different > >>>>>> > names > >>>>>> > > you have used for the > >>>>>> ExceptionCheckingJniEnvPtr local variable. > >>>>>> > They > >>>>>> > > should be consistent. > >>>>>> > > > >>>>>> > > Also, can you rename get_basename() to > >>>>>> get_dirname()? I know Serguei > >>>>>> > > suggested get_basename() a while back, but > >>>>>> unless "basename" is > >>>>>> > > commonly used for this purpose, I think > >>>>>> "dirname" is more self > >>>>>> > > explanatory. > >>>>>> > > >>>>>> > In general, I'm Okay with get_dirname(). > >>>>>> > Just to mention dirname can be both short or > >>>>>> full, so it is a little > >>>>>> > confusing as well. > >>>>>> > It is the reason why the get_basename() was > >>>>>> suggested. > >>>>>> > However, I do not insist on get_basename() nor > >>>>>> get_full_dirname(). :) > >>>>>> > > >>>>>> > Thanks, > >>>>>> > Serguei > >>>>>> > > >>>>>> > > >>>>>> > > thanks, > >>>>>> > > > >>>>>> > > Chris > >>>>>> > > > >>>>>> > > On 12/2/18 10:29 PM, David Holmes wrote: > >>>>>> > >> Hi Jc, > >>>>>> > >> > >>>>>> > >> I've been lurking on this one and have had > >>>>>> a look through. I'm okay > >>>>>> > >> with the FatalError approach for the tests > >>>>>> - we don't expect > >>>>>> > anything > >>>>>> > >> to go wrong in a well written test in a > >>>>>> correctly functioning VM. > >>>>>> > >> > >>>>>> > >> Thanks, > >>>>>> > >> David > >>>>>> > >> > >>>>>> > >> > >>>>>> > >> > >>>>>> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: > >>>>>> > >>> Hi all, > >>>>>> > >>> > >>>>>> > >>> Would someone on the GC or runtime team > >>>>>> be motivated to give > >>>>>> > this a > >>>>>> > >>> review? :) > >>>>>> > >>> > >>>>>> > >>> It would be much appreciated! > >>>>>> > >>> > >>>>>> > >>> Webrev: > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > >>>>>> > >>>>>> > >>> Bug: > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> > >>> > >>>>>> > >>> Thanks for your help, > >>>>>> > >>> Jc > >>>>>> > >>> > >>>>>> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC Beyler > >>>>>> > >>>>>> > >>>>>> > > >>>>>> > >>> >>>>>> > >>>>>> >>>>>> >>> wrote: > >>>>>> > >>> > >>>>>> > >>> Hi Chris, > >>>>>> > >>> > >>>>>> > >>> Yes I was waiting for another review > >>>>>> since you had explicitly > >>>>>> > >>> asked :) > >>>>>> > >>> > >>>>>> > >>> And sounds good that when someone > >>>>>> from GC or runtime gives a > >>>>>> > >>> review, > >>>>>> > >>> I'll wait for your full review on the > >>>>>> webrev.02! > >>>>>> > >>> > >>>>>> > >>> Thanks again for your help, > >>>>>> > >>> Jc > >>>>>> > >>> > >>>>>> > >>> > >>>>>> > >>> On Tue, Nov 27, 2018 at 12:48 PM > >>>>>> Chris Plummer > >>>>>> > >>> >>>>>> > >>>>>> >>>>>> > > >>>>>> > >>>>>> > >>>>>> >>>>>> >>> > >>>>>> > wrote: > >>>>>> > >>> > >>>>>> > >>> Hi JC, > >>>>>> > >>> > >>>>>> > >>> I think it would be good to get a > >>>>>> review from the gc or > >>>>>> > runtime > >>>>>> > >>> teams, since this also affects > >>>>>> their tests. > >>>>>> > >>> > >>>>>> > >>> Also, once we are settled on this > >>>>>> FatalError approach, > >>>>>> > I still > >>>>>> > >>> need to give your webrev-02 a > >>>>>> full review. I only > >>>>>> > skimmed over > >>>>>> > >>> parts of it (I did look at all > >>>>>> the changes in webrevo-01). > >>>>>> > >>> > >>>>>> > >>> thanks, > >>>>>> > >>> > >>>>>> > >>> Chris > >>>>>> > >>> > >>>>>> > >>> On 11/27/18 8:58 AM, > >>>>>> serguei.spitsyn at oracle.com > >>>>>> > >>>>>> > >>>>>> > > >>>>>> > >>> >>>>>> > >>>>>> > >>>>>> >> wrote: > >>>>>> > >>>> Hi Jc, > >>>>>> > >>>> > >>>>>> > >>>> I've already reviewed this too. > >>>>>> > >>>> > >>>>>> > >>>> Thanks, > >>>>>> > >>>> Serguei > >>>>>> > >>>> > >>>>>> > >>>> > >>>>>> > >>>> On 11/27/18 06:56, JC Beyler > >>>>>> wrote: > >>>>>> > >>>>> Thanks Chris, > >>>>>> > >>>>> > >>>>>> > >>>>> Anybody else motivated to look at this > >>>>>> and review it? :) > >>>>>> > >>>>> Jc > >>>>>> > >>>>> > >>>>>> > >>>>> On Mon, Nov 26, 2018 at 1:26 PM > >>>>>> Chris Plummer > >>>>>> > >>>>> >>>>>> > >>>>>> > >>>>>> > > >>>>>> >>>>>> > >>>>>> > >>>>>> >>> > >>>>>> > >>>>> wrote: > >>>>>> > >>>>> > >>>>>> > >>>>> Hi JC, > >>>>>> > >>>>> > >>>>>> > >>>>> I'm ok with the FatalError approach, > >>>>>> but would > >>>>>> > like to > >>>>>> > >>>>> hear opinions from others also. > >>>>>> > >>>>> > >>>>>> > >>>>> thanks, > >>>>>> > >>>>> > >>>>>> > >>>>> Chris > >>>>>> > >>>>> > >>>>>> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: > >>>>>> > >>>>>> Hi Chris, > >>>>>> > >>>>>> > >>>>>> > >>>>>> Thanks for taking the time > >>>>>> to look at it and yes you > >>>>>> > >>>>>> have raised exactly why > >>>>>> the webrev is between two > >>>>>> > >>>>>> worlds: in cases where a > >>>>>> fatal error on failure is > >>>>>> > >>>>>> wanted, should we simplify > >>>>>> the code to remove > >>>>>> > the return > >>>>>> > >>>>>> tests since we do them > >>>>>> internally? Now that I've > >>>>>> > looked > >>>>>> > >>>>>> around for non-fatal > >>>>>> cases, I think the answer > >>>>>> > is yes, > >>>>>> > >>>>>> it simplifies the code > >>>>>> while maintaining the checks. > >>>>>> > >>>>>> > >>>>>> > >>>>>> I looked a bit and it > >>>>>> seems that I can't find > >>>>>> > easily a > >>>>>> > >>>>>> case where the test > >>>>>> accepts a JNI failure to > >>>>>> > then move > >>>>>> > >>>>>> on. Therefore, perhaps, > >>>>>> for now, the fail with a > >>>>>> > Fatal > >>>>>> > >>>>>> is enough and we can work > >>>>>> on the tests to clean > >>>>>> > them up? > >>>>>> > >>>>>> > >>>>>> > >>>>>> That means that this is > >>>>>> the new webrev with only > >>>>>> > Fatal > >>>>>> > >>>>>> and cleans up the tests so > >>>>>> that it is no longer in > >>>>>> > >>>>>> between two worlds: > >>>>>> > >>>>>> > >>>>>> > >>>>>> Webrev: > >>>>>> > >>>>>> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> Bug: > >>>>>> > https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> > >>>>>> > >>>>>> > >>>>>> (This passes testing on my > >>>>>> dev machine for all the > >>>>>> > >>>>>> modified tests) > >>>>>> > >>>>>> > >>>>>> > >>>>>> with the example you > >>>>>> provided, it now looks like: > >>>>>> > >>>>>> > >>>>>> > > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > >>>>>> > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> > > >>>>>> > >>>>>> > > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> > > >>>>>> > >>>>>> > > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> Where it does, to me at > >>>>>> least, seem cleaner and less > >>>>>> > >>>>>> "noisy". > >>>>>> > >>>>>> > >>>>>> > >>>>>> Let me know what you > think, > >>>>>> > >>>>>> Jc > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Tue, Nov 20, 2018 at > >>>>>> 9:33 PM Chris Plummer > >>>>>> > >>>>>> >>>>>> > >>>>>> > >>>>>> > > >>>>>> > >>>>>> >>>>>> > >>>>>> > >>>>>> >>> wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>> Hi JC, > >>>>>> > >>>>>> > >>>>>> > >>>>>> Sorry about the delay. > >>>>>> I had to go back an > >>>>>> > look at > >>>>>> > >>>>>> the initial 8210842 > >>>>>> webrev and RFR thread to see > >>>>>> > >>>>>> what this was > >>>>>> initially all about. > >>>>>> > >>>>>> > >>>>>> > >>>>>> In general the changes > >>>>>> look good. > >>>>>> > >>>>>> > >>>>>> > >>>>>> I don't have a good > >>>>>> answer to your > >>>>>> > >>>>>> FatalError/NonFatalError question. It makes > >>>>>> > the code > >>>>>> > >>>>>> a lot cleaner to use > >>>>>> FatalError, but then it > >>>>>> > is a > >>>>>> > >>>>>> behavior change, and > >>>>>> you also need to deal with > >>>>>> > >>>>>> tests that > >>>>>> intentionally induce errors (do > >>>>>> > you have > >>>>>> > >>>>>> an example of that). > >>>>>> > >>>>>> > >>>>>> > >>>>>> In any case, right now > >>>>>> your webrev seems to be > >>>>>> > >>>>>> between two worlds. > >>>>>> You are producing > >>>>>> > FatalError, > >>>>>> > >>>>>> but still checking > >>>>>> results. Here's a good > >>>>>> > example: > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > > >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > >>>>>> > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> > > >>>>>> > >>>>>> > > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > > >>>>>> < > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> > > >>>>>> > >>>>>> > > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> I'm not sure if this > >>>>>> is just a temporary > >>>>>> > state until > >>>>>> > >>>>>> it was decided which > >>>>>> approach to take. > >>>>>> > >>>>>> > >>>>>> > >>>>>> thanks, > >>>>>> > >>>>>> > >>>>>> > >>>>>> Chris > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 11/20/18 2:14 PM, > >>>>>> JC Beyler wrote: > >>>>>> > >>>>>>> Hi all, > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Chris thought it made > >>>>>> sense to have more > >>>>>> > eyes on > >>>>>> > >>>>>>> this change than just > >>>>>> serviceability as it will > >>>>>> > >>>>>>> modify to tests that > >>>>>> are not only > >>>>>> > serviceability > >>>>>> > >>>>>>> tests so I've moved > >>>>>> this to conversation > >>>>>> > here :) > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> For convenience, I've > >>>>>> copy-pasted the > >>>>>> > initial RFR: > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Could I have a review > >>>>>> for the extension and > >>>>>> > usage > >>>>>> > >>>>>>> of the > >>>>>> ExceptionJniWrapper. This adds lines and > >>>>>> > >>>>>>> filenames to the end > >>>>>> of the wrapper JNI > >>>>>> > methods, > >>>>>> > >>>>>>> adds tracing, and > >>>>>> throws an error if need > >>>>>> > be. I've > >>>>>> > >>>>>>> ported the gc/lock > >>>>>> files to use the new > >>>>>> > >>>>>>> TRACE_JNI_CALL add-on > >>>>>> and I've ported a few > >>>>>> > of the > >>>>>> > >>>>>>> tests that were > >>>>>> already changed for the > >>>>>> > assignment > >>>>>> > >>>>>>> webrev for > >>>>>> JDK-8212884. > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Webrev: > >>>>>> > >>>>>>> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 > >>>>>> > >>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>>> Bug: > >>>>>> > >>>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> For illustration, if > >>>>>> I force an error to the > >>>>>> > >>>>>>> AP04/ap04t03 test and > >>>>>> set the verbosity on, > >>>>>> > I get > >>>>>> > >>>>>>> something like: > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> >> Calling JNI method > >>>>>> FindClass from > >>>>>> > >>>>>>> ap04t003.cpp:343 > >>>>>> > >>>>>>> >> Calling with these > >>>>>> parameter(s): > >>>>>> > >>>>>>> java/lang/Threadd > >>>>>> > >>>>>>> Wait for thread to > >>>>>> finish > >>>>>> > >>>>>>> << Called JNI method > >>>>>> FindClass from > >>>>>> > >>>>>>> ap04t003.cpp:343 > >>>>>> > >>>>>>> Exception in thread > >>>>>> "Thread-0" > >>>>>> > >>>>>>> java.lang.NoClassDefFoundError: > >>>>>> > java/lang/Threadd > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>> > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Method) > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Caused by: > >>>>>> java.lang.ClassNotFoundException: > >>>>>> > >>>>>>> java.lang.Threadd > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >>>>>> > >>>>>>> ... 3 more > >>>>>> > >>>>>>> FATAL ERROR in native > >>>>>> method: JNI method > >>>>>> > FindClass > >>>>>> > >>>>>>> : internal error from > >>>>>> ap04t003.cpp:343 > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>> > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Method) > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> at > >>>>>> > >>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Questions/comments I > >>>>>> have about this are: > >>>>>> > >>>>>>> - Do we want to > >>>>>> force fatal errors when a JNI > >>>>>> > >>>>>>> call fails in > >>>>>> general? Most of these tests > >>>>>> > do the > >>>>>> > >>>>>>> right thing and test > >>>>>> the return of the JNI > >>>>>> > calls, > >>>>>> > >>>>>>> for example: > >>>>>> > >>>>>>> thrClass = > >>>>>> > jni->FindClass("java/lang/Threadd", > >>>>>> > >>>>>>> TRACE_JNI_CALL); > >>>>>> > >>>>>>> if (thrClass == > >>>>>> NULL) { > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> but now the wrapper > >>>>>> actually would do a > >>>>>> > fatal if > >>>>>> > >>>>>>> the FindClass call > >>>>>> would return a nullptr, > >>>>>> > so we > >>>>>> > >>>>>>> could remove that > >>>>>> test altogether. What do you > >>>>>> > >>>>>>> think? > >>>>>> > >>>>>>> - I prefer to > >>>>>> leave them as the tests then > >>>>>> > >>>>>>> become closer to what > >>>>>> real users would have in > >>>>>> > >>>>>>> their code and is the > >>>>>> "recommended" way of > >>>>>> > doing it > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> - The alternative > >>>>>> is to use the > >>>>>> > NonFatalError I > >>>>>> > >>>>>>> added which then just > >>>>>> prints out that something > >>>>>> > >>>>>>> went wrong, letting > >>>>>> the test continue. Question > >>>>>> > >>>>>>> will be what should > >>>>>> be the default? The > >>>>>> > fatal or > >>>>>> > >>>>>>> the non-fatal error > >>>>>> handling? > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> On a different > >>>>>> subject: > >>>>>> > >>>>>>> - On the new tests, > >>>>>> I've removed the > >>>>>> > >>>>>>> NSK_JNI_VERIFY since > >>>>>> the JNI wrapper > >>>>>> > handles the > >>>>>> > >>>>>>> tracing and the > >>>>>> verify in almost the same > >>>>>> > way; only > >>>>>> > >>>>>>> difference I can > >>>>>> really tell is that the > >>>>>> > complain > >>>>>> > >>>>>>> method from NSK has a > >>>>>> max complain before > >>>>>> > stopping > >>>>>> > >>>>>>> to "complain"; I have > >>>>>> not added that part > >>>>>> > of the > >>>>>> > >>>>>>> code in this webrev > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Once we decide on > >>>>>> these, I can continue on the > >>>>>> > >>>>>>> files from > >>>>>> JDK-8212884 and then do both the > >>>>>> > >>>>>>> assignment in an if > >>>>>> extraction followed-by this > >>>>>> > >>>>>>> type of webrev in an > >>>>>> easier fashion. > >>>>>> > Depending on > >>>>>> > >>>>>>> decisions here, > >>>>>> NSK*VERIFY can be deprecated as > >>>>>> > >>>>>>> well as we go > forward. > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Thanks! > >>>>>> > >>>>>>> Jc > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> On Mon, Nov 19, 2018 > >>>>>> at 11:34 AM Chris Plummer > >>>>>> > >>>>>>> >>>>>> > >>>>>> > >>>>>> > > >>>>>> > >>>>>>> >>>>>> > >>>>>> > >>>>>> >>> wrote: > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> On 11/19/18 10:07 > >>>>>> AM, JC Beyler wrote: > >>>>>> > >>>>>>>> Hi all, > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> @David/Chris: > >>>>>> should I then push this > >>>>>> > RFR to > >>>>>> > >>>>>>>> the hotspot > >>>>>> mailing or the runtime > >>>>>> > one? For > >>>>>> > >>>>>>>> what it's worth, > >>>>>> a lot of the tests > >>>>>> > under the > >>>>>> > >>>>>>>> vmTestbase are > >>>>>> jvmti so the review also > >>>>>> > >>>>>>>> affects > >>>>>> serviceability; it just turns > >>>>>> > out I > >>>>>> > >>>>>>>> started with the > >>>>>> GC originally and > >>>>>> > then hit > >>>>>> > >>>>>>>> some other tests > >>>>>> I had touched via the > >>>>>> > >>>>>>>> assignment > >>>>>> extraction. > >>>>>> > >>>>>>> I think hotspot > >>>>>> would be best. > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> Chris > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> @Serguei: Done > >>>>>> for the method > >>>>>> > renaming, for > >>>>>> > >>>>>>>> the indent, are > >>>>>> you talking about > >>>>>> > going from > >>>>>> > >>>>>>>> the 8-indent to > >>>>>> 4-indent? If so, would > >>>>>> > it not > >>>>>> > >>>>>>>> just be better > >>>>>> to do a new JBS bug and > >>>>>> > do the > >>>>>> > >>>>>>>> whole files in > >>>>>> one go? I ask because > >>>>>> > >>>>>>>> otherwise, it > >>>>>> will look a bit weird to > >>>>>> > have > >>>>>> > >>>>>>>> parts of the > >>>>>> file as 8-indent and others > >>>>>> > >>>>>>>> 4-indent? > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> Thanks for > >>>>>> looking at it! > >>>>>> > >>>>>>>> Jc > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> On Mon, Nov 19, > >>>>>> 2018 at 1:25 AM > >>>>>> > >>>>>>>> serguei.spitsyn at oracle.com > >>>>>> > >>>>>> >>>>>> > > >>>>>> > >>>>>>>> >>>>>> > >>>>>> > >>>>>> >> > >>>>>> > >>>>>>>> >>>>>> > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>> >>>>>> > >>>>>> > >>>>>> >>> wrote: > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> Hi Jc, > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> We have to > >>>>>> start this review > >>>>>> > anyway. :) > >>>>>> > >>>>>>>> It looks > >>>>>> good to me in general. > >>>>>> > >>>>>>>> Thank you > >>>>>> for your consistency in this > >>>>>> > >>>>>>>> refactoring! > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> Some minor > >>>>>> comments. > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> > >>>>>> > > >>>>>> > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> +static > >>>>>> const char* > >>>>>> > remove_folders(const > >>>>>> > >>>>>>>> char* > >>>>>> fullname) { I'd suggest to > >>>>>> > rename > >>>>>> > >>>>>>>> the function > >>>>>> name to something > >>>>>> > traditional > >>>>>> > >>>>>>>> like > >>>>>> get_basename. Otherwise, it > >>>>>> > sounds > >>>>>> > >>>>>>>> like this > >>>>>> function has to really > >>>>>> > remove > >>>>>> > >>>>>>>> folders. :) > >>>>>> Also, all *Locker.cpp have > >>>>>> > >>>>>>>> wrong indent > >>>>>> in the bodies of if > >>>>>> > and while > >>>>>> > >>>>>>>> statements. > >>>>>> Could this be fixed > >>>>>> > with the > >>>>>> > >>>>>>>> refactoring? > >>>>>> I did not look on how > >>>>>> > this > >>>>>> > >>>>>>>> impacts the > >>>>>> tests other than > >>>>>> > >>>>>>>> serviceability. Thanks, > >>>>>> Serguei > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> On 11/16/18 > >>>>>> 19:43, JC Beyler wrote: > >>>>>> > >>>>>>>>> Hi all, > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Anybody > >>>>>> motivated to review this? :) > >>>>>> > >>>>>>>>> Jc > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> On Wed, Nov 7, > >>>>>> 2018 at 9:53 PM JC > >>>>>> > Beyler > >>>>>> > >>>>>>>>> >>>>>> > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>>> >>>>>> > >>>>>> > >>>>>> >>> wrote: > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Hi all, > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Could I have > >>>>>> a review for the > >>>>>> > >>>>>>>>> extension > >>>>>> and usage of the > >>>>>> > >>>>>>>>> ExceptionJniWrapper. This > >>>>>> > adds lines > >>>>>> > >>>>>>>>> and > >>>>>> filenames to the end of the > >>>>>> > >>>>>>>>> wrapper JNI > >>>>>> methods, adds > >>>>>> > tracing, > >>>>>> > >>>>>>>>> and throws > >>>>>> an error if need > >>>>>> > be. I've > >>>>>> > >>>>>>>>> ported the > >>>>>> gc/lock files to > >>>>>> > use the > >>>>>> > >>>>>>>>> new > >>>>>> TRACE_JNI_CALL add-on and > >>>>>> > I've > >>>>>> > >>>>>>>>> ported a few > >>>>>> of the tests > >>>>>> > that were > >>>>>> > >>>>>>>>> already > >>>>>> changed for the > >>>>>> > assignment > >>>>>> > >>>>>>>>> webrev for > >>>>>> JDK-8212884. > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Webrev: > >>>>>> > >>>>>>>>> > >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ > >>>>>> > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>> > >>>>>>>>> Bug: > >>>>>> > >>>>>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> For > >>>>>> illustration, if I force > >>>>>> > an error > >>>>>> > >>>>>>>>> to the > >>>>>> AP04/ap04t03 test and > >>>>>> > set the > >>>>>> > >>>>>>>>> verbosity > >>>>>> on, I get something > >>>>>> > like: > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> >> Calling > >>>>>> JNI method > >>>>>> > FindClass from > >>>>>> > >>>>>>>>> ap04t003.cpp:343 > >>>>>> > >>>>>>>>> >> Calling > >>>>>> with these > >>>>>> > parameter(s): > >>>>>> > >>>>>>>>> java/lang/Threadd > >>>>>> > >>>>>>>>> Wait for > >>>>>> thread to finish > >>>>>> > >>>>>>>>> << Called > >>>>>> JNI method > >>>>>> > FindClass from > >>>>>> > >>>>>>>>> ap04t003.cpp:343 > >>>>>> > >>>>>>>>> Exception in > >>>>>> thread "Thread-0" > >>>>>> > >>>>>>>>> java.lang.NoClassDefFoundError: > >>>>>> > >>>>>>>>> java/lang/Threadd > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>> > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Method) > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Caused by: > >>>>>> > >>>>>>>>> java.lang.ClassNotFoundException: > >>>>>> > >>>>>>>>> java.lang.Threadd > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >>>>>> > >>>>>>>>> ... 3 more > >>>>>> > >>>>>>>>> FATAL ERROR > >>>>>> in native method: JNI > >>>>>> > >>>>>>>>> method > >>>>>> FindClass : internal error > >>>>>> > >>>>>>>>> from > >>>>>> ap04t003.cpp:343 > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > >>>>>> > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Method) > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> at > >>>>>> > >>>>>>>>> > >>>>>> > > >>>>>> > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > >>>>>> > >>>>>> > > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Questions/comments I have about > >>>>>> > >>>>>>>>> this are: > >>>>>> > >>>>>>>>> - Do we > >>>>>> want to force fatal > >>>>>> > errors > >>>>>> > >>>>>>>>> when a JNI > >>>>>> call fails in general? > >>>>>> > >>>>>>>>> Most of > >>>>>> these tests do the right > >>>>>> > >>>>>>>>> thing and > >>>>>> test the return of > >>>>>> > the JNI > >>>>>> > >>>>>>>>> calls, for > >>>>>> example: > >>>>>> > >>>>>>>>> thrClass = > >>>>>> > >>>>>>>>> jni->FindClass("java/lang/Threadd", > >>>>>> > >>>>>>>>> TRACE_JNI_CALL); > >>>>>> > >>>>>>>>> if > >>>>>> (thrClass == NULL) { > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> but now the > >>>>>> wrapper actually > >>>>>> > would do > >>>>>> > >>>>>>>>> a fatal if > >>>>>> the FindClass call > >>>>>> > would > >>>>>> > >>>>>>>>> return a > >>>>>> nullptr, so we could > >>>>>> > remove > >>>>>> > >>>>>>>>> that test > >>>>>> altogether. What do > >>>>>> > you > >>>>>> > >>>>>>>>> think? > >>>>>> > >>>>>>>>> - I > >>>>>> prefer to leave them > >>>>>> > as the > >>>>>> > >>>>>>>>> tests then > >>>>>> become closer to > >>>>>> > what real > >>>>>> > >>>>>>>>> users would > >>>>>> have in their > >>>>>> > code and is > >>>>>> > >>>>>>>>> the > >>>>>> "recommended" way of doing it > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> - The > >>>>>> alternative is to > >>>>>> > use the > >>>>>> > >>>>>>>>> NonFatalError I added > >>>>>> which > >>>>>> > then just > >>>>>> > >>>>>>>>> prints out > >>>>>> that something > >>>>>> > went wrong, > >>>>>> > >>>>>>>>> letting the > >>>>>> test continue. > >>>>>> > Question > >>>>>> > >>>>>>>>> will be what > >>>>>> should be the > >>>>>> > default? > >>>>>> > >>>>>>>>> The fatal or > >>>>>> the non-fatal error > >>>>>> > >>>>>>>>> handling? > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> On a > >>>>>> different subject: > >>>>>> > >>>>>>>>> - On the > >>>>>> new tests, I've > >>>>>> > removed > >>>>>> > >>>>>>>>> the > >>>>>> NSK_JNI_VERIFY since the JNI > >>>>>> > >>>>>>>>> wrapper > >>>>>> handles the tracing > >>>>>> > and the > >>>>>> > >>>>>>>>> verify in > >>>>>> almost the same > >>>>>> > way; only > >>>>>> > >>>>>>>>> difference I > >>>>>> can really tell > >>>>>> > is that > >>>>>> > >>>>>>>>> the complain > >>>>>> method from NSK > >>>>>> > has a > >>>>>> > >>>>>>>>> max complain > >>>>>> before stopping to > >>>>>> > >>>>>>>>> "complain"; > >>>>>> I have not added that > >>>>>> > >>>>>>>>> part of the > >>>>>> code in this webrev > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Once we > >>>>>> decide on these, I can > >>>>>> > >>>>>>>>> continue on > >>>>>> the files from > >>>>>> > >>>>>>>>> JDK-8212884 > >>>>>> and then do both the > >>>>>> > >>>>>>>>> assignment > >>>>>> in an if extraction > >>>>>> > >>>>>>>>> followed-by > >>>>>> this type of > >>>>>> > webrev in an > >>>>>> > >>>>>>>>> easier > >>>>>> fashion. Depending on > >>>>>> > >>>>>>>>> decisions > >>>>>> here, NSK*VERIFY can be > >>>>>> > >>>>>>>>> deprecated > >>>>>> as well as we go > >>>>>> > forward. > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> Thank you > >>>>>> for the > >>>>>> > reviews/comments :) > >>>>>> > >>>>>>>>> Jc > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> > >>>>>> > >>>>>>>>> -- > >>>>>> > >>>>>>>>> Thanks, > >>>>>> > >>>>>>>>> Jc > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> > >>>>>> > >>>>>>>> -- > >>>>>> > >>>>>>>> Thanks, > >>>>>> > >>>>>>>> Jc > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> > >>>>>> > >>>>>>> -- > >>>>>> > >>>>>>> Thanks, > >>>>>> > >>>>>>> Jc > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Jc > >>>>>> > >>>>> > >>>>>> > >>>>> > >>>>>> > >>>>> > >>>>>> > >>>>> > >>>>>> > >>>>> -- > >>>>>> > >>>>> Thanks, > >>>>>> > >>>>> Jc > >>>>>> > >>>> > >>>>>> > >>> > >>>>>> > >>> > >>>>>> > >>> > >>>>>> > >>> -- > >>>>>> > >>> Thanks, > >>>>>> > >>> Jc > >>>>>> > >>> > >>>>>> > >>> > >>>>>> > >>> > >>>>>> > >>> -- > >>>>>> > >>> > >>>>>> > >>> Thanks, > >>>>>> > >>> Jc > >>>>>> > > > >>>>>> > > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > -- > >>>>>> > > >>>>>> > Thanks, > >>>>>> > Jc > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Thanks, > >>>>>> Jc > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Thanks, > >>>>> Jc > >>>> > >>>> > >>>> > >>>> -- > >>>> Thanks, > >>>> Jc > >>>> > >>>> > >>>> > >>>> -- > >>>> > >>>> Thanks, > >>>> Jc > >>> > > > -- Thanks, Jc From mikael.vidstedt at oracle.com Thu Dec 13 04:57:25 2018 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 12 Dec 2018 20:57:25 -0800 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: <6901870C-E287-4C45-82DF-1E8D0C9A9D5F@oracle.com> Vote: yes Cheers, Mikael > On Dec 12, 2018, at 2:58 PM, jesper.wilhelmsson at oracle.com wrote: > > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From Yang.Zhang at arm.com Thu Dec 13 06:20:04 2018 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Thu, 13 Dec 2018 06:20:04 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> <754b4943-559f-31d7-c564-a1103ac6d50d@oracle.com> <81d7ce0c-c2f0-b6c7-476a-21731ffba293@redhat.com> Message-ID: >>> Can this patch be merged in jdk12? Because jdk 12 ramp down phase is coming. >> >> It's pushed to jdk-submit. I'll let you know what happens. > It passed. I'll push it tomorrow. Thanks very much. Regards Yang -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From zhaixiang at loongson.cn Thu Dec 13 08:57:35 2018 From: zhaixiang at loongson.cn (Leslie Zhai) Date: Thu, 13 Dec 2018 16:57:35 +0800 Subject: System.out.println lead to assert(resolved_method->method_holder()->is_linked()) failed: must be linked In-Reply-To: <8f7b3fb8-8908-0b3e-f1c7-6b8519a9b5fb@oracle.com> References: <8f7b3fb8-8908-0b3e-f1c7-6b8519a9b5fb@oracle.com> Message-ID: <7f6c75bb-3814-3e91-c5a6-c1c3fd08341d@loongson.cn> Hi David, Thanks for your kind response! ? 2018/12/13 ??11:26, David Holmes ??: > On 13/12/2018 12:58 pm, Leslie Zhai wrote: >> Hi Coleen, >> >> System.out.println was added on purpose: > > You can't do that. You can't put any code in the Object constructor > (or various other places - static initializers etc) because nothing > else is initialized yet - including String, System etc etc. The code > will hit exceptions and try to throw but can't because the Exception > classes can't be initialized yet. I will use other way to debug during vm initializing. > > > The assert is what we always see when trying to throw an exception > before the VM is initialized enough to throw exceptions. > > Cheers, > David > ---- > >> diff -r 8449dc3cbebe src/java.base/share/classes/java/lang/Object.java >> --- a/src/java.base/share/classes/java/lang/Object.java??? Mon Nov 12 >> 15:34:21 2018 +0800 >> +++ b/src/java.base/share/classes/java/lang/Object.java??? Thu Dec 13 >> 10:40:33 2018 +0800 >> @@ -47,7 +47,9 @@ >> ?????? * Constructs a new object. >> ?????? */ >> ????? @HotSpotIntrinsicCandidate >> -??? public Object() {} >> +??? public Object() { >> +????? System.out.println("DEBUG: on purpose"); >> +??? } >> >> ????? /** >> ?????? * Returns the runtime class of this {@code Object}. The returned >> >> ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- >> >> Then it lead to the assertion error, similar as: >> >> https://bugs.openjdk.java.net/browse/JDK-8069005 >> >> $ ./build/linux-x86_64-normal-server-fastdebug/jdk/bin/java -version >> # To suppress the following error report, specify this argument >> # after -XX: or in .hotspotrc: SuppressErrorAt=/linkResolver.cpp:1380 >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> #? Internal Error >> (/home/zhaixiang/jdk/src/hotspot/share/interpreter/linkResolver.cpp:1380), >> pid=11090, tid=11091 >> #? assert(resolved_method->method_holder()->is_linked()) failed: must >> be linked >> >> ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- >> >> But just add java.lang.Object initialization in bootstrapping the jvm >> couldn't fix the issue: >> >> diff -r 2c5bf37c8f8e src/hotspot/share/runtime/thread.cpp >> --- a/src/hotspot/share/runtime/thread.cpp??? Thu Dec 06 20:39:03 >> 2018 +0800 >> +++ b/src/hotspot/share/runtime/thread.cpp??? Thu Dec 13 10:44:56 >> 2018 +0800 >> @@ -3524,6 +3524,7 @@ >> ??? // Initialize java_lang.System (needed before creating the thread) >> ??? initialize_class(vmSymbols::java_lang_System(), CHECK); >> ??? // The VM creates & returns objects of this class. Make sure it's >> initialized. >> +? initialize_class(vmSymbols::java_lang_Object(), CHECK); >> ??? initialize_class(vmSymbols::java_lang_Class(), CHECK); The order of initialize_class give the illusion that java_lang_String, java_lang_System had been initilized.? Sorry for that! Thanks, Leslie Zhai >> initialize_class(vmSymbols::java_lang_ThreadGroup(), CHECK); >> ??? Handle thread_group = create_initial_thread_group(CHECK); >> >> Please give me some advice how to fix the issue, thanks a lot! >> >> Regards, >> >> Leslie Zhai >> >> From volker.simonis at gmail.com Thu Dec 13 09:09:14 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 13 Dec 2018 10:09:14 +0100 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes On Thu, Dec 13, 2018 at 12:01 AM wrote: > > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From Pengfei.Li at arm.com Thu Dec 13 09:41:19 2018 From: Pengfei.Li at arm.com (Pengfei Li (Arm Technology China)) Date: Thu, 13 Dec 2018 09:41:19 +0000 Subject: 8212043: Request for help with Submit Failure In-Reply-To: References: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> <48685738-ad99-f48d-72a7-ff87d20cfea9@oracle.com> Message-ID: Hi Andrew Dinn, > Pengfei, I think a portable way to define +/-INFINITY will be to use the values > 1.0D/0.0D or -1.0D/0.0D. > > We might want to introduce some local defines for them. > > I'm just checking that this definition works on AArch64 (it ought to as it uses > gcc) and will then resubmit to see if sparc also accepts it. This is my rebased webrev with the INFINITY issue fixed. http://cr.openjdk.java.net/~pli/rfr/8212043/webrev.02/ The +/-INFINITY definitions for float/double are added into type.cpp/hpp. I've tested the build on x86 and AArch64 and it's good. Hope other platforms wouldn't be broken this time. -- Thanks, Pengfei From amith.pawar at gmail.com Thu Dec 13 09:41:49 2018 From: amith.pawar at gmail.com (amith pawar) Date: Thu, 13 Dec 2018 15:11:49 +0530 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> Message-ID: Hi Thomas, Please find the attached patch updated as per your suggestion. If everything OK then can you please commit this to repo ? Thanks, Amit Pawar On Wed, Dec 12, 2018 at 6:36 PM Thomas Schatzl wrote: > Hi Amit, > > On Sat, 2018-12-08 at 04:50 +0530, amith pawar wrote: > > Hi Thomas, > > > > Please check my inline comments and attached patch is updated as per > > your suggestion. > > thanks a lot! Looks very good now. > > Fixed issues: > > - you can use the LogStream directly for writing, you do not need to > create an outputStream from it. > > - "isrunning_..." should be renamed "is_running_..." > > There are still some "is" prefixes glued to the predicate left. > > Here are webrevs: > > http://cr.openjdk.java.net/~tschatzl/8213827/webrev.3/ (latest webrev) > http://cr.openjdk.java.net/~tschatzl/8213827/webrev.2_to_3/ (above > suggested changes) > http://cr.openjdk.java.net/~tschatzl/8213827/webrev.2/ (original > changes) > > Some comments further down. > > > > > On Mon, Dec 3, 2018 at 6:12 PM Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > Hi, > > > > > > On Fri, 2018-11-30 at 11:11 +0100, Thomas Schatzl wrote: > > > > Hi Amit, > > > > > > > > [...] > > > > > > > > > > Sorry. Please check the attached patch. > > > > > > > > I created a webrev at > > > > http://cr.openjdk.java.net/~tschatzl/8213827/webrev/ . > > > > > > > > Let the reviews begin :) > > > > > > > > > > style issues: > > > > > > - random whitespaces in many places. > > > - all if's or if-then-else require braces > > > - only class members have a leading underscore > > > - do not declare local variables long before they are used. That > > > makes the code hard to read. > > > - do not introduce local variables for no reason. E.g. > > > > > > 2907 struct bitmask *bmp; > > > > > > [...unrelated code...] > > > > > > 2911 bmp = _numa_get_interleave_mask(); > > > 2912 set_numa_interleave_ptr(&bmp); > > > 2913 bmp = _numa_get_membind(); > > > 2914 set_numa_membind_ptr(&bmp); > > > > > > is much less readable than > > > > > > set_numa_interleave_ptr(_numa_get_interleave_mask()); > > > set_numa_membind_ptr(_numa_get_membind_mask()); > > > > > > - in the type declaration, please put the * next to the type. > > > While > > > this is kind-of wrong, the majority of existing code does so (not > > > only > > > in these files). > > > > > > code issues: > > > > > > - interleave/membind mode are mutually exclusive, which the code > > > kind-of tries to establish (and check) using special values for eg. > > > the > > > value of Linux::_numa_interleave_ptr. > > > > > > I think the code gets a lot more readable if you introduce an enum > > > and > > > an explicit global variable holding the current NUMA node. If you > > > do > > > not want to do that, move the condition > > > "Linux::_numa_interleave_ptr != > > > 0" into a new method. > > > > > > - move the new NUMA initialization code in os::init_2() into a new > > > method please. It's gotten very long now. > > > > > > - please avoid duplicating code; ie. > > > > > > 5039 // Check for membind mode. > > > 5040 bmp = Linux::_numa_membind_ptr; > > > 5041 for (int node = 0; node <= Linux::numa_max_node() ; > > > node++) > > > { > > > 5042 if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > > 5043 _is_membind = true; > > > 5044 } > > > 5045 } > > > > > > 5048 bmp = Linux::_numa_interleave_ptr; > > > 5049 for (int node = 0; node <= Linux::numa_max_node() ; > > > node++) > > > { > > > 5050 if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > > 5051 _is_interleaved = true; > > > 5052 // Set membind to false as interleave mode allows > > > all > > > nodes to be used. > > > 5053 _is_membind = false; > > > 5054 } > > > 5055 } > > > > > > and right after: > > > > > > Something like: > > > > > > is_interleave = > > > is_any_bit_in_bitmask_set(Linux::_numa_interleave_ptr); > > > is_membind = !is_interleave && > > > is_any_bit_in_bitmask_set(Linux::_numa_membind_ptr); > > > > > > with an appropriate is_any_bit_in_bitmask_set() helper method is > > > much > > > more clear. > > > > > > (Even better, as suggested above use a global "NUMA mode" enum that > > > is > > > filled in somewhere instead of multiple flags) > > > > > > - Use a LogStream to print the log messages instead of rolling > > > your > > > own code to create a string containing the node number. The code > > > that > > > is there exhibits a potential buffer overflow (with many NUMA > > > nodes), > > > and a potential out-of-bounds write. > > > > > > - the log message for selected NUMA mode and nodes should be a > > > single > > > log message. > > > > > > - partially pre-existing: the numa_set_xxx_ptr methods take a > > > "struct bitmask**". Could you explain why "**"? All of those store > > > the dereferenced result anyway, so why not pass the dereferenced > > > pointer right away? > > > > Following bitmask pointers are defined in libnuma library. 'dlvsym' > > will return the address of pointer which needs to be de-referenced. > > Updated other methods as suggested. > > struct bitmask *numa_all_nodes_ptr; > > struct bitmask *numa_no_nodes_ptr; > > struct bitmask *numa_all_cpus_ptr; > > > This also simplifies the code at the callers. > > > > > > - partially pre-existing: I would prefer if the numa_set_xx_ptr > > > methods were all renamed to numa_set_xx_bitmask. The "ptr" suffix > > > seems redundant, and most importantly very unspecific. > > > > Newly introduced membind and interleave bitmask pointer names are > > changed. Also above mentioned three pointers are imported. If you > > suggest will rename these also. > > If you want we can do this in a separate changeset. Otherwise, please > do right away. > Please also fix the glued "is" in the isnode_in_bound_nodes, > isbound_to_single_node, isnode_in_existing_nodes, > isnode_in_configured_nodes methods. > > > > - in numa_interleave_memory(), why does the code not make the > > > membind/interleave mode distinction when the v1 API is used? > > > > v2 API's works as expected for membind/interleave case but > > numa_get_membind_v1 API's always returns null . v1 and v2 prototypes > > are also not same. Please check below. > > nodemask_t numa_get_membind_v1(void) > > struct bitmask * numa_get_membind_v2(void) > > > > void numa_interleave_memory_v1(void *mem, size_t size, const > > nodemask_t *mask) > > void numa_interleave_memory_v2(void *mem, size_t size, struct bitmask > > *bmp) > > Okay, understood. Thanks! > > Thanks, > Thomas > > > -- With best regards, amit pawar -------------- next part -------------- diff -r b9d34a97a4be src/hotspot/os/linux/os_linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp Thu Dec 13 13:03:26 2018 +0530 +++ b/src/hotspot/os/linux/os_linux.cpp Thu Dec 13 15:21:58 2018 +0530 @@ -33,6 +33,7 @@ #include "compiler/disassembler.hpp" #include "interpreter/interpreter.hpp" #include "logging/log.hpp" +#include "logging/logStream.hpp" #include "memory/allocation.inline.hpp" #include "memory/filemap.hpp" #include "oops/oop.inline.hpp" @@ -2776,7 +2777,7 @@ // Get the total number of nodes in the system including nodes without memory. for (node = 0; node <= highest_node_number; node++) { - if (isnode_in_existing_nodes(node)) { + if (is_node_in_existing_nodes(node)) { num_nodes++; } } @@ -2792,7 +2793,7 @@ // node number. If the nodes have been bound explicitly using numactl membind, // then allocate memory from those nodes only. for (int node = 0; node <= highest_node_number; node++) { - if (Linux::isnode_in_bound_nodes((unsigned int)node)) { + if (Linux::is_node_in_bound_nodes((unsigned int)node)) { ids[i++] = node; } } @@ -2895,11 +2896,15 @@ libnuma_dlsym(handle, "numa_distance"))); set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, libnuma_v2_dlsym(handle, "numa_get_membind"))); + set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, + libnuma_v2_dlsym(handle, "numa_get_interleave_mask"))); if (numa_available() != -1) { set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); set_numa_all_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_all_nodes_ptr")); set_numa_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_nodes_ptr")); + set_numa_interleave_bitmask(_numa_get_interleave_mask()); + set_numa_membind_bitmask(_numa_get_membind()); // Create an index -> node mapping, since nodes are not always consecutive _nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) GrowableArray(0, true); rebuild_nindex_to_node_map(); @@ -2925,7 +2930,7 @@ nindex_to_node()->clear(); for (int node = 0; node <= highest_node_number; node++) { - if (Linux::isnode_in_existing_nodes(node)) { + if (Linux::is_node_in_existing_nodes(node)) { nindex_to_node()->append(node); } } @@ -2962,16 +2967,16 @@ // the closest configured node. Check also if node is bound, i.e. it's allowed // to allocate memory from the node. If it's not allowed, map cpus in that node // to the closest node from which memory allocation is allowed. - if (!isnode_in_configured_nodes(nindex_to_node()->at(i)) || - !isnode_in_bound_nodes(nindex_to_node()->at(i))) { + if (!is_node_in_configured_nodes(nindex_to_node()->at(i)) || + !is_node_in_bound_nodes(nindex_to_node()->at(i))) { closest_distance = INT_MAX; // Check distance from all remaining nodes in the system. Ignore distance // from itself, from another non-configured node, and from another non-bound // node. for (size_t m = 0; m < node_num; m++) { if (m != i && - isnode_in_configured_nodes(nindex_to_node()->at(m)) && - isnode_in_bound_nodes(nindex_to_node()->at(m))) { + is_node_in_configured_nodes(nindex_to_node()->at(m)) && + is_node_in_bound_nodes(nindex_to_node()->at(m))) { distance = numa_distance(nindex_to_node()->at(i), nindex_to_node()->at(m)); // If a closest node is found, update. There is always at least one // configured and bound node in the system so there is always at least @@ -3026,9 +3031,13 @@ os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; os::Linux::numa_distance_func_t os::Linux::_numa_distance; os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; +os::Linux::numa_get_interleave_mask_func_t os::Linux::_numa_get_interleave_mask; +os::Linux::Numa_allocation_policy os::Linux::_current_numa_policy; unsigned long* os::Linux::_numa_all_nodes; struct bitmask* os::Linux::_numa_all_nodes_ptr; struct bitmask* os::Linux::_numa_nodes_ptr; +struct bitmask* os::Linux::_numa_interleave_bitmask; +struct bitmask* os::Linux::_numa_membind_bitmask; bool os::pd_uncommit_memory(char* addr, size_t size) { uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, @@ -4958,6 +4967,74 @@ OSContainer::init(); } +void os::Linux::numa_init() { + + // Java can be invoked as + // 1. Without numactl and heap will be allocated/configured on all nodes as + // per the system policy. + // 2. With numactl --interleave: + // Use numa_get_interleave_mask(v2) API to get nodes bitmask. The same + // API for membind case bitmask is reset. + // Interleave is only hint and Kernel can fallback to other nodes if + // no memory is available on the target nodes. + // 3. With numactl --membind: + // Use numa_get_membind(v2) API to get nodes bitmask. The same API for + // interleave case returns bitmask of all nodes. + // numa_all_nodes_ptr holds bitmask of all nodes. + // numa_get_interleave_mask(v2) and numa_get_membind(v2) APIs returns correct + // bitmask when externally configured to run on all or fewer nodes. + + if (!Linux::libnuma_init()) { + UseNUMA = false; + } else { + if ((Linux::numa_max_node() < 1) || Linux::is_bound_to_single_node()) { + // If there's only one node (they start from 0) or if the process + // is bound explicitly to a single node using membind, disable NUMA. + UseNUMA = false; + } else { + + LogTarget(Info,os) log; + LogStream ls(log); + + Linux::set_configured_numa_policy (Linux::identify_numa_policy()); + + struct bitmask* bmp = Linux::_numa_membind_bitmask; + const char* numa_mode = "membind"; + + if (Linux::is_running_in_interleave_mode()) { + bmp = Linux::_numa_interleave_bitmask; + numa_mode = "interleave"; + } + + ls.print("UseNUMA is enabled and invoked in '%s' mode." + " Heap will be configured using NUMA memory nodes:", numa_mode); + + for (int node = 0; node <= Linux::numa_max_node(); node++) { + if (Linux::_numa_bitmask_isbitset(bmp, node)) { + ls.print(" %d", node); + } + } + } + } + + if (UseParallelGC && UseNUMA && UseLargePages && !can_commit_large_page_memory()) { + // With SHM and HugeTLBFS large pages we cannot uncommit a page, so there's no way + // we can make the adaptive lgrp chunk resizing work. If the user specified both + // UseNUMA and UseLargePages (or UseSHM/UseHugeTLBFS) on the command line - warn + // and disable adaptive resizing. + if (UseAdaptiveSizePolicy || UseAdaptiveNUMAChunkSizing) { + warning("UseNUMA is not fully compatible with SHM/HugeTLBFS large pages, " + "disabling adaptive resizing (-XX:-UseAdaptiveSizePolicy -XX:-UseAdaptiveNUMAChunkSizing)"); + UseAdaptiveSizePolicy = false; + UseAdaptiveNUMAChunkSizing = false; + } + } + + if (!UseNUMA && ForceNUMA) { + UseNUMA = true; + } +} + // this is called _after_ the global arguments have been parsed jint os::init_2(void) { @@ -5002,32 +5079,7 @@ Linux::glibc_version(), Linux::libpthread_version()); if (UseNUMA) { - if (!Linux::libnuma_init()) { - UseNUMA = false; - } else { - if ((Linux::numa_max_node() < 1) || Linux::isbound_to_single_node()) { - // If there's only one node (they start from 0) or if the process - // is bound explicitly to a single node using membind, disable NUMA. - UseNUMA = false; - } - } - - if (UseParallelGC && UseNUMA && UseLargePages && !can_commit_large_page_memory()) { - // With SHM and HugeTLBFS large pages we cannot uncommit a page, so there's no way - // we can make the adaptive lgrp chunk resizing work. If the user specified both - // UseNUMA and UseLargePages (or UseSHM/UseHugeTLBFS) on the command line - warn - // and disable adaptive resizing. - if (UseAdaptiveSizePolicy || UseAdaptiveNUMAChunkSizing) { - warning("UseNUMA is not fully compatible with SHM/HugeTLBFS large pages, " - "disabling adaptive resizing (-XX:-UseAdaptiveSizePolicy -XX:-UseAdaptiveNUMAChunkSizing)"); - UseAdaptiveSizePolicy = false; - UseAdaptiveNUMAChunkSizing = false; - } - } - - if (!UseNUMA && ForceNUMA) { - UseNUMA = true; - } + Linux::numa_init(); } if (MaxFDLimit) { diff -r b9d34a97a4be src/hotspot/os/linux/os_linux.hpp --- a/src/hotspot/os/linux/os_linux.hpp Thu Dec 13 13:03:26 2018 +0530 +++ b/src/hotspot/os/linux/os_linux.hpp Thu Dec 13 15:21:58 2018 +0530 @@ -211,6 +211,7 @@ // none present private: + static void numa_init(); static void expand_stack_to(address bottom); typedef int (*sched_getcpu_func_t)(void); @@ -222,6 +223,7 @@ typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); typedef struct bitmask* (*numa_get_membind_func_t)(void); + typedef struct bitmask* (*numa_get_interleave_mask_func_t)(void); typedef void (*numa_set_bind_policy_func_t)(int policy); typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); @@ -239,9 +241,12 @@ static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; static numa_distance_func_t _numa_distance; static numa_get_membind_func_t _numa_get_membind; + static numa_get_interleave_mask_func_t _numa_get_interleave_mask; static unsigned long* _numa_all_nodes; static struct bitmask* _numa_all_nodes_ptr; static struct bitmask* _numa_nodes_ptr; + static struct bitmask* _numa_interleave_bitmask; + static struct bitmask* _numa_membind_bitmask; static void set_sched_getcpu(sched_getcpu_func_t func) { _sched_getcpu = func; } static void set_numa_node_to_cpus(numa_node_to_cpus_func_t func) { _numa_node_to_cpus = func; } @@ -255,10 +260,21 @@ static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } + static void set_numa_get_interleave_mask(numa_get_interleave_mask_func_t func) { _numa_get_interleave_mask = func; } static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } + static void set_numa_interleave_bitmask(struct bitmask* ptr) { _numa_interleave_bitmask = ptr ; } + static void set_numa_membind_bitmask(struct bitmask* ptr) { _numa_membind_bitmask = ptr ; } static int sched_getcpu_syscall(void); + + enum Numa_allocation_policy{ + not_initialized, + membind, + interleave + }; + static Numa_allocation_policy _current_numa_policy; + public: static int sched_getcpu() { return _sched_getcpu != NULL ? _sched_getcpu() : -1; } static int numa_node_to_cpus(int node, unsigned long *buffer, int bufferlen) { @@ -272,11 +288,35 @@ static int numa_tonode_memory(void *start, size_t size, int node) { return _numa_tonode_memory != NULL ? _numa_tonode_memory(start, size, node) : -1; } + + static bool is_running_in_interleave_mode() { + return _current_numa_policy == interleave ? true : false; + } + + static void set_configured_numa_policy(Numa_allocation_policy numa_policy) { + _current_numa_policy = numa_policy ; + } + + static Numa_allocation_policy identify_numa_policy() { + Numa_allocation_policy current_policy = membind; + for (int node = 0; node <= Linux::numa_max_node() ; node++) { + if (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_bitmask, node)) { + current_policy = interleave; + } + } + return current_policy ; + } + static void numa_interleave_memory(void *start, size_t size) { // Use v2 api if available - if (_numa_interleave_memory_v2 != NULL && _numa_all_nodes_ptr != NULL) { - _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr); - } else if (_numa_interleave_memory != NULL && _numa_all_nodes != NULL) { + if (_numa_interleave_memory_v2 != NULL && _numa_membind_bitmask != NULL) { + // Use interleave bitmask while running interleave mode. + if (is_running_in_interleave_mode()) { + _numa_interleave_memory_v2(start, size, _numa_interleave_bitmask); + } else if (_numa_membind_bitmask != NULL) { + _numa_interleave_memory_v2(start, size, _numa_membind_bitmask); + } + } else { _numa_interleave_memory(start, size, _numa_all_nodes); } } @@ -291,14 +331,14 @@ static int get_node_by_cpu(int cpu_id); static int get_existing_num_nodes(); // Check if numa node is configured (non-zero memory node). - static bool isnode_in_configured_nodes(unsigned int n) { + static bool is_node_in_configured_nodes(unsigned int n) { if (_numa_bitmask_isbitset != NULL && _numa_all_nodes_ptr != NULL) { return _numa_bitmask_isbitset(_numa_all_nodes_ptr, n); } else return false; } // Check if numa node exists in the system (including zero memory nodes). - static bool isnode_in_existing_nodes(unsigned int n) { + static bool is_node_in_existing_nodes(unsigned int n) { if (_numa_bitmask_isbitset != NULL && _numa_nodes_ptr != NULL) { return _numa_bitmask_isbitset(_numa_nodes_ptr, n); } else if (_numa_bitmask_isbitset != NULL && _numa_all_nodes_ptr != NULL) { @@ -317,16 +357,19 @@ return false; } // Check if node is in bound node set. - static bool isnode_in_bound_nodes(int node) { - if (_numa_get_membind != NULL && _numa_bitmask_isbitset != NULL) { - return _numa_bitmask_isbitset(_numa_get_membind(), node); - } else { - return false; + static bool is_node_in_bound_nodes(int node) { + if (_numa_bitmask_isbitset != NULL) { + if (is_running_in_interleave_mode()) { + return _numa_bitmask_isbitset(_numa_interleave_bitmask, node); + } else { + return _numa_membind_bitmask != NULL ? _numa_bitmask_isbitset(_numa_membind_bitmask, node) : false; + } } + return false; } // Check if bound to only one numa node. // Returns true if bound to a single numa node, otherwise returns false. - static bool isbound_to_single_node() { + static bool is_bound_to_single_node() { int nodes = 0; struct bitmask* bmp = NULL; unsigned int node = 0; From per.liden at oracle.com Thu Dec 13 09:53:49 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 13 Dec 2018 10:53:49 +0100 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes /Per On 12/12/2018 11:58 PM, jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From markus.gronlund at oracle.com Thu Dec 13 10:08:54 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 13 Dec 2018 02:08:54 -0800 (PST) Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes Thanks Markus -----Original Message----- From: Jesper Wilhelmsson Sent: den 12 december 2018 23:59 To: hotspot-dev developers Subject: CFV: New hotspot Group Member: Chris Plummer I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Thanks, /Jesper [1]http://openjdk.java.net/census#hotspot [2]http://openjdk.java.net/groups/#member-vote From thomas.schatzl at oracle.com Thu Dec 13 10:33:14 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 13 Dec 2018 11:33:14 +0100 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> Message-ID: Hi Amit, On Thu, 2018-12-13 at 15:11 +0530, amith pawar wrote: > Hi Thomas, > > Please find the attached patch updated as per your suggestion. > If everything OK then can you please commit this to repo ? looks good. We will need a second reviewer though, I am going to ask around. Latest webrev: http://cr.openjdk.java.net/~tschatzl/8213827/webrev.3/ Thanks, Thomas From magnus.ihse.bursie at oracle.com Thu Dec 13 10:37:52 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 13 Dec 2018 11:37:52 +0100 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> <236420f2-cd56-75a7-db52-59a7ef9f9be3@oracle.com> Message-ID: On 2018-12-12 13:17, David Holmes wrote: > Okay I went away and did some homework ... > > Let me back up a bit and see if I have the evolution of this correctly. > > The native implementation of Java methods should be declared and > defined with JNIEXPORT and JNICALL. > > JNIEXPORT controls the export visibility so they can be linked. > > JNICALL controls the calling convention and is needed so that the > JVM's calling convention matches the native code. [This part was > unclear to me.] Yes. And JNICALL is empty on all platforms except Windows 32, that's why we're only seeing issues about mismatch there. > > Other native methods exported from the same (or different) native > libraries may also be decorated with JNIEXPORT and JNICALL. But in > places there is a mismatch between the declaration in the header and > the definition leading to errors. Yes; this mismatch has typically happened when the linking has not been done by simply including the relevant header file, but either duplicating the definition, or looking up the symbol dynamically. Otherwise things should basically work out. > > There are two main types of such native functions: > > a) those publicly defined in the various native APIs: JNI itself > (jni.h), JVM TI (jvmti.h), AWT (jawt.h) ... > > b) those intended only for use by other native code within the JDK > libraries (JLI_* - though I note Alan's comment re javafxpackager, > others??) > > and a possible third type are callback functions like the JPLISAgent > event handlers (e.g. eventHandlerVMInit). I'm not sure I understand what the third type is, but if it is a publically defined API (which, at a first look, the JPLISAgent API seems to be), then it's part of catagory a, otherwise it's part of category b. I must also state that my initial proposal didn't separate these two cases. I had in mind only the b cases -- I have no intention of changing public specifications, but I did not express this in my proposal. That might have been one source of confusion. I apologize for this omission. > > There is a view that no native method that isn't the native-half of a > Java method should use JNICALL. [Okay I can see how that makes sense - > regardless of the actual calling convention used marking such methods > as "must use the same calling convention as the VM native method call" > is somewhat meaningless. Yet obviously the public native APIs do > specify JNICALL so this is not a hard and fast rule. Further the > callbacks must also follow a convention known to the VM doing the > calling!] Yes, or rather the rule is "only native half Java methods should use JNICALL, and also all public APIs where so is specified". > > Where we have (introduced?) a discrepancy between the definition and > declaration the approach has been (because of the previous view) to > remove JNICALL. [This should only relate to methods of kind (b) above.] Actually, it's not so much as we have *removed* JNICALL, as that we have *introduced* JNIEXPORT. When I removed the map files, I also removed the .def files and /export command lines for Windows. As a result, I needed to add JNIEXPORT to a lot of places. Initially, I followed the rule of adding JNICALL to those calls -- but I can surely have missed a couple of places, since things will work fine anyway, as long as caller and callee agree on the calling convention; and a mismatch can only happen on Windows 32, which we do not build and test. So there is a risk that even with the initial patch, not all newly added JNIEXPORTs had JNICALL. Then, it turned out, that a lot of this code did not compile with Windows 32. With no deeper analysis of the flaw, the blame was put on the absence or presence of JNICALL, and a series of patches were made where JNICALL was more or less arbitrarily added or removed, until things "worked". This should have been a warning sign, and I was increasingly uneasy about all these changes, but I hadn't spent enough time to realize what the core issue was or how to resolve it properly. > > That leaves those methods with JNIEXPORT only. > > That seems wrong to you because they are not "JNI methods", so you > want to replace with JDK_EXPORT to make it clear. [Ok - this seems > reasonable.] Yes. And given the fact that we have a bunch of "non-JNI methods" that use the JNIEXPORT...JNICALL pattern, another way to interpret the JDK_EXPORT semantics is that this function is exported for usage *in the JDK*, but not for public consumption. > > If JNICALL is removed from those native methods and they are currently > linked by external applications, those applications may stop working. > But this only affects win32 (as its the only platform where JNICALL is > different to the default C/C++ calling convention?) so your position > is this is acceptable breakage - and would only affect > unsupported/undocumented APIs. Yes. In fact, it's possible that at least some of the breakage that occurred was due to new *addition* of JNICALL, which was not present before. We might have had something like (I'm making this specific example up) a function "void ZIP_OpenFile(char* name)" with no decoration at all, and a "/export:ZIP_OpenFile" on the command line, and a ZIP_OpenFile entry in the unix map file. And I converted this to "JNIEXPORT void JNICALL ZIP_OpenFile(char* name)", which de facto -- although I didn't fully realize this at the time, changed the calling convention and name decoration on Windows 32. When something broke, perhaps because the user of ZIP_OpenFile did not include the proper header file with the JNICALL modifier, the solution was to remove the JNICALL part. And of all the bug reports I've seen on this, the issues has been internal linking only, i.e. problems building the JDK, not complaints that external tools tried to use internal interfaces and failed. So yes, my position is if this should break things, tough shit. That, of courses, requires that we do not change public APIs, so we need to be careful not to mess with those. > > Does that sum it up? Yep, I think so. > > We still need to be sure that we're only changing functions of type > (b) above. Yes, definitely. > > And I note that this has been driven by the choice to remove JNICALL > where there was a discrepancy - that leads to the visibility of the > two kinds of methods. If it had been chosen to add JNICALL where > missing instead, then we may not have been having this conversation. Actually, as I said, this has more been the result of a lot of added JNICALL where previously there was none. An alternative course of action is the make sure that *all* calls use the JNIEXPORT...JNICALL pattern, even type b ones, and that we fix all parts of code to actually accept the decorated names on Windows 32. This will lead to a lot more code changes, like the fix for JDK-8214122 (JDWP is broken on 32 bit Windows: transport library missing onLoad entry). And all this special case handling will be needed only on Windows 32, which is a platform we do not want to spend to much time or effort on. And finally, I think doing so would make us miss out on an opportunity to make the semantics clearer. > > This will also need a CSR request due to the change in linking behaviour. I'm not sure what you mean by this..? My entire intention here is that we should make no changes to documented interfaces; this is strictly an internal change, so no CSR should be needed. Also, I don't understand what you mean by "linking behavior"? /Magnus > > Cheers, > David > ----- > > On 12/12/2018 9:03 pm, Magnus Ihse Bursie wrote: >> >> >> On 2018-12-11 23:47, David Holmes wrote: >>> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >>>> >>>> >>>> On 2018-12-11 00:23, David Holmes wrote: >>>>> Hi Magnus, >>>>> >>>>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>>>> I propose that we introduce a new define, available to all JDK >>>>>> native files (Hotspot included), called JDK_EXPORT. The behavior >>>>>> of this symbol will be very similar (as of now, in fact >>>>>> identical) to JNIEXPORT; however, the semantics will not. >>>>>> >>>>>> Currently, we "mis-use" the JNIEXPORT define to mark a function >>>>>> for exporting from the library. The problem with this is that >>>>>> JNIEXPORT is part of the JNI interface, and is supposed to be >>>>>> used when C programs interact with Java. And, when doing this, >>>>>> the function should be fully decorated like this: "JNIEXPORT foo >>>>>> JNICALL". >>>>> >>>>> I've seen a lot of the emails on this issue and I don't fully >>>>> understand what has been going wrong. But the intent is obviously >>>>> the JNIEXPORT represents what is needed to export this function >>>>> for use by JNI, while JNICALL defines the calling convention. I >>>>> agree there may be some mistmatch when functions are actually not >>>>> intended for general export outside the JDK but are only for >>>>> internal JDK use. >>>>> >>>>>> We do have many such JNI exports in our native libraries, but we >>>>>> also have a lot of other, non-JNI exports, where one native >>>>>> library just provides an interface to other libraries. In these >>>>>> cases, we have still used JNIEXPORT for the functionality of >>>>>> getting the function exported, but we have not been consistent in >>>>>> our use of JNICALL. This has caused us way too much trouble for >>>>>> something that should Just Work. >>>>> >>>>> Are you suggesting that the interface between different libraries >>>>> in the JDK should not be a JNI interface? Is this because you >>>>> think the functions in these libraries are only for JDK internal >>>>> use or ... ?? >>>>> >>>>>> I therefore propose that we define "JDK_EXPORT", with the same >>>>>> behavior as JNIEXPORT (that is, flagging the function for >>>>>> external visibility in the resulting native library), but which >>>>>> is *not* supposed to be exported to Java code using JNI, nor >>>>>> supposed to be decorated with >>>>> >>>>> Just a clarification there. JNI functions are not exported to Java >>>>> code, they are exported to native code. Java code can declare >>>>> native methods and those native methods must be written as JNI >>>>> functions, but that's not what we are discussing. Libraries expose >>>>> a JNI interface (a set of functions in the library) that can be >>>>> called by application native code, using JNI. >>>> We're apparently looking at "JNI" and "exporting" from two opposite >>>> sides here. :-) Just to make everything clear: If I have a Java class >>>> class MyClass { >>>> public static void native myNativeFunc(); >>>> } >>>> >>>> then I have one half of the JNI function, the Java half. This must >>>> be matched by a corresponding implementation in native, like this: >>>> JNIEXPORT void JNICALL >>>> Java_MyClass_myNativeFunc(void) { >>>> // ... do stuff >>>> } >>>> >>>> And this is the native half of the JNI function. Right? Let's leave >>>> aside which side is "exporting" to the other for now. :-) >>>> >>>> This way of setting up native functions that can be called from >>>> Java is what I refer to as JNI. And when you declare a native JNI >>>> function, you *must* use both JNIEXPORT and JNICALL. Alright? >>>> >>>> We do have a lot of those functions in our native libraries. And >>>> they are correct just the way they are. >>> >>> Yep all well and good. A function declared native in Java must have >>> an implementation as you describe. But not all native functions >>> exist to provide the native-half of a Java native function! >>> >>>> However, we also have *other* native functions, that are exported, >>>> not as JNI functions that should be called from Java, but as normal >>>> native library functions that should be called by other native >>>> code. Okay so far? And *those* functions have been problematic in >>>> how we decorate >>> >>> But there are again two cases. Those functions exported from a >>> library that are expected to be called from external code using the >>> JNI interface mechanism - such as all the JNI functions and JVM TI >>> functions we export from the JVM - and those "exported" for access >>> between libraries within the JDK (such as all the JVM_* functions in >>> libjvm). >>> >>> I think it is only the second group that should be addressed by your >>> JDK_EXPORT proposal - though I'm not completely clear exactly how to >>> identify them. >> Alright, now I think we're on the same page again. :) >> >> Yes, this is what I'm saying. I'm not proposing to modify public APIs. >> >> You identify external APIs by the fact that they are documented. And >> if you are unsure, you ask Jon. ;-) >> >>> >>>> them. My proposal is that we *refrain* from using JNIEXPORT for >>>> those functions, and instead use JDK_EXPORT as name for the macro >>>> that decorates them as exported. That way, we can clearly see that >>>> a function like this: >>>> >>>> JDK_EXPORT void >>>> JLI_ReadEnv(char* env); >>>> >>>> is correctly declared, and will be exported to other native >>>> libraries, but not to Java. >>> >>> The issue is not whether it is "exported to Java"** but whether it >>> is exported using the JNI mechanism such that other native code >>> calls it using the JNI mechanism. >>> >>> ** There is no way to write a native method declaration in Java such >>> that it would be linked to the JLI_ReadEnv function. The naming is >>> all wrong, as is the signature. >>> >>>> Just to clarify, this has nothing to do with if this is a >>>> officially supported API or not. In general though, I assume that >>>> most (if not all?) of our exported functions (apart from the JNI_* >>>> stuff) is supposed to be consumed by other libraries in the JDK, >>>> and is not a public API. >>> >>> I think it varies library by library. You may need native >>> application code that can call directly into native JDK libraries. >>> JLI is the Java Launcher Interface - I think it was introduced to >>> make it easier for other launchers to be created. Native agents may >>> need access to libmanagement or libjdwp functions. Native graphics >>> code may need access to the JDK graphics library. Some of these >>> accesses may be unsupported and undocumented, but I don't think you >>> can just cut them all off. >> If they are unsupported and undocumented, I sure as h*ll can cut them >> all off! :-) >> >> Besides, and let me re-iterate this, the only change that will happen >> by removing JNICALL, is on Windows 32. No other platform will be >> affected. There is no official support for Windows 32 anymore. >> There's some low-effort community work done on keeping Windows 32 >> alive, but it's not backed by any major player. Right now, they are >> taking a lot of hits *due to the fact* that we have not sorted this >> out, and waste a lot of their precious effort trying to fix this >> piecemeal. If we do fix things according to this proposal, then it's >> at least clear how things *should* work. If it turns out that there's >> some code out there in the wild, running on Windows 32, that uses an >> undocumented and unsupported function call, and it breaks -- well, >> sucks to be them. They'll just have to do what everyone does who uses >> an undocumented interface that suddenly changes: update their code. >> >> /Magnus >> >>> >>> David >>> >>>> >>>> /Magnus >>>> >>>> >>>> >>>>> >>>>>> JNICALL. All current instances of JNIEXPORT which is not pure JNI >>>>>> native functions should be changed to use JDK_EXPORT instead. >>>>>> >>>>>> I further propose that this macro should reside in a new file >>>>>> "jdk.h", placed in the new directory >>>>>> src/java.base/share/native/include/internal. This header file >>>>>> path will automatically be provided to all native libraries, but >>>>>> not copied to the JDK being built. (The existence of a >>>>>> "include/internal" directory with this behavior has been >>>>>> discussed before. There are more files that ought to be moved >>>>>> there, if/when it is created.) I believe in many cases the >>>>>> #include "jni.h" can be just modified to #include "#jdk.h", since >>>>>> most native code will not require "jni.h" unless actually doing >>>>>> JNI calls -- most have included this file to get the JNIEXPORT >>>>>> macro, which would explain the pervasive use of #include "jni.h" >>>>>> in our code base. >>>>> >>>>> jni.h also defines all of the types used by the JNI. Those types >>>>> are pervsive to the native code used throughout the JDK. >>>>> >>>>>> Thoughts? >>>>> >>>>> I think we need to understand the problems on Windows that >>>>> prompted all this. Then I think we need to look at exactly how >>>>> jni.h and JNIEXPORT etc are being used and understand whether this >>>>> is truly an exported interface or not. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> /Magnus >>>>>> >>>> >> From tobias.hartmann at oracle.com Thu Dec 13 10:47:52 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 13 Dec 2018 11:47:52 +0100 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes Best regards, Tobias On 12.12.18 23:58, jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From ChrisPhi at LGonQn.Org Thu Dec 13 13:23:13 2018 From: ChrisPhi at LGonQn.Org (Chris Phillips) Date: Thu, 13 Dec 2018 08:23:13 -0500 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes On 2018-12-12 5:58 p.m., jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > > > > Cheers Chris From shade at redhat.com Thu Dec 13 15:49:03 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 13 Dec 2018 16:49:03 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> <21d2347b-86b9-6333-eb3a-4a249061ee0c@oracle.com> Message-ID: <995210ac-fa2c-841e-18e2-a8f166c5199b@redhat.com> Thanks! Pushed. If there are any follow-ups, we can do them in a separate RFE. Cheers, -Aleksey On 12/12/18 11:41 PM, David Holmes wrote: > LGTM! > > Thanks, > David > > On 13/12/2018 8:29 am, Aleksey Shipilev wrote: >> On 12/12/18 11:23 PM, David Holmes wrote: >>> Sorry Aleksey the existing test directory is runtime/Safepoint not Safepoints. Please place the new >>> test there. >> >> /me facepalms >> >> Updated in-place: >> ?? http://cr.openjdk.java.net/~shade/8181143/webrev.05/ >> >> -Aleksey >> From thomas.schatzl at oracle.com Thu Dec 13 16:02:42 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 13 Dec 2018 17:02:42 +0100 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> Message-ID: <0466c060d2f354fd8e17e50eb24e2c1faf481614.camel@oracle.com> Hi again, On Thu, 2018-12-13 at 11:33 +0100, Thomas Schatzl wrote: > Hi Amit, > On Thu, 2018-12-13 at 15:11 +0530, amith pawar wrote: > > Hi Thomas, > > > > Please find the attached patch updated as per your suggestion. > > If everything OK then can you please commit this to repo ? > > looks good. We will need a second reviewer though, I am going to > ask around. Testing showed that there are (new) crashes in conjunction with the NUMAInterleaveGranularity option in the runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test. I am going to look into this later. Thanks, Thomas > > Latest webrev: > http://cr.openjdk.java.net/~tschatzl/8213827/webrev.3/ > > Thanks, > Thomas > > From yumin.qi at gmail.com Thu Dec 13 16:44:17 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 13 Dec 2018 08:44:17 -0800 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: vote: yes On Wed, Dec 12, 2018 at 3:01 PM wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to > Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the > HotSpot project for over 8 years and is a Reviewer in the JDK Project with > 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote > as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this > nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > > From adinn at redhat.com Thu Dec 13 17:25:22 2018 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 13 Dec 2018 17:25:22 +0000 Subject: 8212043: Request for help with Submit Failure In-Reply-To: References: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> <48685738-ad99-f48d-72a7-ff87d20cfea9@oracle.com> Message-ID: Hi Pengfei, On 13/12/2018 09:41, Pengfei Li (Arm Technology China) wrote: > Hi Andrew Dinn, > >> Pengfei, I think a portable way to define +/-INFINITY will be to use the values >> 1.0D/0.0D or -1.0D/0.0D. >> >> We might want to introduce some local defines for them. >> >> I'm just checking that this definition works on AArch64 (it ought to as it uses >> gcc) and will then resubmit to see if sparc also accepts it. > > This is my rebased webrev with the INFINITY issue fixed. > http://cr.openjdk.java.net/~pli/rfr/8212043/webrev.02/ > > The +/-INFINITY definitions for float/double are added into type.cpp/hpp. > I've tested the build on x86 and AArch64 and it's good. Hope other platforms wouldn't be broken this time. Bad news again, Pengfei :-/. I pushed your latest patch and got the errors below (see after sig :-). I don't know for sure what the actual errors are but I suspect that the relevant compilers will not bite on the expression (1.0/0.0). It seems that defining +/-INFINITY in a portable way is going to require some work. One option would be to replicate the approach employed in src/java.base/share/native/libfdlibm which clearly must work on all platforms. File fdlibm.h defines macros HI and LO (also HIp and LOp) for accessing and updating floats and doubles as one or two 32 bit ints. HI can be used to read or set float bits or the 'top' bits of doubles. LO is used to read or set the 'bottom' bits of doubles. fdlibm.h includes the relevant version of endian.h to ensure it respects respects endianness. So, you could use these macros to initialise the relevant float and double constants to the correct bit patterns. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- Build Details: 2018-12-13-1216576.adinn.source 0 Failed Tests Mach5 Tasks Results Summary FAILED: 0 EXECUTED_WITH_FAILURE: 10 NA: 0 KILLED: 0 PASSED: 27 UNABLE_TO_RUN: 39 Build 10 Executed with failure macosx-x64-macosx-x64-build-5 error while building, return value: 2 macosx-x64-debug-macosx-x64-build-6 error while building, return value: 2 macosx-x64-open-macosx-x64-build-7 error while building, return value: 2 macosx-x64-open-debug-macosx-x64-build-8 error while building, return value: 2 solaris-sparcv9-solaris-sparcv9-build-9 error while building, return value: 2 solaris-sparcv9-debug-solaris-sparcv9-build-10 error while building, return value: 2 windows-x64-windows-x64-build-11 error while building, return value: 2 windows-x64-debug-windows-x64-build-12 error while building, return value: 2 windows-x64-open-windows-x64-build-13 error while building, return value: 2 windows-x64-open-debug-windows-x64-build-14 error while building, return value: 2 3 Not run macosx-x64-install-macosx-x64-build-16 Dependency task failed: mach5...1213-1219-14490-macosx-x64-macosx-x64-build-5 solaris-sparcv9-install-solaris-sparcv9-build-17 Dependency task failed: mach5...14490-solaris-sparcv9-solaris-sparcv9-build-9 windows-x64-install-windows-x64-build-18 Dependency task failed: sCTahmBZKo Test 36 Not run tier1-product-open_test_hotspot_jtreg_tier1_common-macosx-x64-20 Dependency task failed: mach5...1213-1219-14490-macosx-x64-macosx-x64-build-5 tier1-debug-open_test_hotspot_jtreg_tier1_common-macosx-x64-debug-26 Dependency task failed: mach5...219-14490-macosx-x64-debug-macosx-x64-build-6 tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-58 Dependency task failed: mach5...14490-solaris-sparcv9-solaris-sparcv9-build-9 tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-59 Dependency task failed: mach5...olaris-sparcv9-debug-solaris-sparcv9-build-10 tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-21 Dependency task failed: sCTahmBZKo tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-27 Dependency task failed: sCTahmB0Ko tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-macosx-x64-debug-29 Dependency task failed: mach5...219-14490-macosx-x64-debug-macosx-x64-build-6 tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-30 Dependency task failed: sCTahmB0Ko tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-macosx-x64-debug-32 Dependency task failed: mach5...219-14490-macosx-x64-debug-macosx-x64-build-6 tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-33 Dependency task failed: sCTahmB0Ko See all 36... From john.r.rose at oracle.com Thu Dec 13 19:22:55 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 13 Dec 2018 11:22:55 -0800 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: Vote: yes From thomas.schatzl at oracle.com Thu Dec 13 20:44:15 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 13 Dec 2018 21:44:15 +0100 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: <0466c060d2f354fd8e17e50eb24e2c1faf481614.camel@oracle.com> References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> <99164b92f47f264978339ed327da9d41098a7e1d.camel@oracle.com> <0466c060d2f354fd8e17e50eb24e2c1faf481614.camel@oracle.com> Message-ID: Hi, On Thu, 2018-12-13 at 17:02 +0100, Thomas Schatzl wrote: > Hi again, > > On Thu, 2018-12-13 at 11:33 +0100, Thomas Schatzl wrote: > > Hi Amit, > > On Thu, 2018-12-13 at 15:11 +0530, amith pawar wrote: > > > Hi Thomas, > > > > > > Please find the attached patch updated as per your suggestion. > > > If everything OK then can you please commit this to repo ? > > > > looks good. We will need a second reviewer though, I am going to > > ask around. > > Testing showed that there are (new) crashes in conjunction with the > NUMAInterleaveGranularity option in the > runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java > test. Everything is fine. This is JDK-8204478. Thanks, Thomas From leonid.mesnik at oracle.com Thu Dec 13 20:57:13 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 13 Dec 2018 12:57:13 -0800 Subject: RFR: 8215369: Jcstress pollute /var/tmp with temporary files. Message-ID: <96812CDD-ACE0-4CCE-9584-4FCEDFBBDEC3@oracle.com> Hi Could you please review following fix which set property "java.io.tmpdir" for jctress harness and it's forked VMs . So any temporary files are created in test directory and not in system temp. webrev: http://cr.openjdk.java.net/~lmesnik/8215369/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8215369 Leonid From igor.ignatyev at oracle.com Thu Dec 13 21:24:07 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 13 Dec 2018 13:24:07 -0800 Subject: RFR: 8215369: Jcstress pollute /var/tmp with temporary files. In-Reply-To: <96812CDD-ACE0-4CCE-9584-4FCEDFBBDEC3@oracle.com> References: <96812CDD-ACE0-4CCE-9584-4FCEDFBBDEC3@oracle.com> Message-ID: <07659D3B-E3ED-4258-AB25-A53F9C9A7C13@oracle.com> Hi Leonid, shouldn't 2nd -Djava.io.tmpdir go after '--jvmArgs' ? > 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); the rest looks good. since this is test-only fix, I'd prefer to get it integrated into jdk/jdk12 rather than into jdk/jdk. Cheers, -- Igor > On Dec 13, 2018, at 12:57 PM, Leonid Mesnik wrote: > > Hi > > Could you please review following fix which set property "java.io.tmpdir" for jctress harness and it's forked VMs . So any temporary files are created in test directory and not in system temp. > > webrev: http://cr.openjdk.java.net/~lmesnik/8215369/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8215369 > > Leonid From leonid.mesnik at oracle.com Thu Dec 13 21:36:11 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 13 Dec 2018 13:36:11 -0800 Subject: RFR: 8215369: Jcstress pollute /var/tmp with temporary files. In-Reply-To: <07659D3B-E3ED-4258-AB25-A53F9C9A7C13@oracle.com> References: <96812CDD-ACE0-4CCE-9584-4FCEDFBBDEC3@oracle.com> <07659D3B-E3ED-4258-AB25-A53F9C9A7C13@oracle.com> Message-ID: <4A05EA09-17CE-4DBE-911F-2A0F5B1F0BD6@oracle.com> > On Dec 13, 2018, at 1:24 PM, Igor Ignatyev wrote: > > Hi Leonid, > > shouldn't 2nd -Djava.io.tmpdir go after '--jvmArgs' ? >> 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); > the rest looks good. It is right after -jvmArgs. 105 extraFlags.add("--jvmArgs"); 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); then more --jvmArgs are added with other options in for loop. > > since this is test-only fix, I'd prefer to get it integrated into jdk/jdk12 rather than into jdk/jdk. > ok. Leonid > Cheers, > -- Igor > >> On Dec 13, 2018, at 12:57 PM, Leonid Mesnik > wrote: >> >> Hi >> >> Could you please review following fix which set property "java.io.tmpdir" for jctress harness and it's forked VMs . So any temporary files are created in test directory and not in system temp. >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8215369/webrev.00/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215369 > >> >> Leonid > From shade at redhat.com Thu Dec 13 21:42:16 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 13 Dec 2018 22:42:16 +0100 Subject: RFR (XS) [12] 8215374: 32-bit build failures after JDK-8181143 (Introduce diagnostic flag to abort VM on too long VM operations) Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8215374 With all the excitement around fixing x86_32 issues before jdk12 fork today, I forgot to test my own patch still builds on x86_32! It does not, because format specifiers are wrong. Fix: diff -r 408cdb799358 -r 1fb17b0a42c2 src/hotspot/share/runtime/vmThread.cpp --- a/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:38 2018 +0100 +++ b/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:39 2018 +0100 @@ -206,7 +206,7 @@ if (is_armed()) { jlong delay = (os::javaTimeMillis() - _arm_time); if (delay > AbortVMOnVMOperationTimeoutDelay) { - fatal("VM operation took too long: " SIZE_FORMAT " ms (timeout: " SIZE_FORMAT " ms)", + fatal("VM operation took too long: " JLONG_FORMAT " ms (timeout: " INTX_FORMAT " ms)", delay, AbortVMOnVMOperationTimeoutDelay); } } This goes to jdk/jdk12, because the bug is there. This build failure is currently masked by another x86_32 build failure, fixed by JDK-8215353, which would be pushed before this fix. Testing: {x86_32, x86_64} build, AbortVMOnVMOperationTimeout test Thanks, -Aleksey From igor.ignatyev at oracle.com Thu Dec 13 21:43:10 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 13 Dec 2018 13:43:10 -0800 Subject: RFR: 8215369: Jcstress pollute /var/tmp with temporary files. In-Reply-To: <4A05EA09-17CE-4DBE-911F-2A0F5B1F0BD6@oracle.com> References: <96812CDD-ACE0-4CCE-9584-4FCEDFBBDEC3@oracle.com> <07659D3B-E3ED-4258-AB25-A53F9C9A7C13@oracle.com> <4A05EA09-17CE-4DBE-911F-2A0F5B1F0BD6@oracle.com> Message-ID: > On Dec 13, 2018, at 1:36 PM, Leonid Mesnik wrote: > > > >> On Dec 13, 2018, at 1:24 PM, Igor Ignatyev > wrote: >> >> Hi Leonid, >> >> shouldn't 2nd -Djava.io.tmpdir go after '--jvmArgs' ? >>> 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); >> the rest looks good. > > It is right after -jvmArgs. ok, it was just me misreading udiff. looks good and trivial to me, ship it. > > 105 extraFlags.add("--jvmArgs"); > 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); > then more --jvmArgs are added with other options in for loop. > >> >> since this is test-only fix, I'd prefer to get it integrated into jdk/jdk12 rather than into jdk/jdk. >> > ok. > > Leonid > >> Cheers, >> -- Igor >> >>> On Dec 13, 2018, at 12:57 PM, Leonid Mesnik > wrote: >>> >>> Hi >>> >>> Could you please review following fix which set property "java.io.tmpdir" for jctress harness and it's forked VMs . So any temporary files are created in test directory and not in system temp. >>> >>> webrev: http://cr.openjdk.java.net/~lmesnik/8215369/webrev.00/ > >>> bug: https://bugs.openjdk.java.net/browse/JDK-8215369 > >>> >>> Leonid >> > From mikhailo.seledtsov at oracle.com Thu Dec 13 21:46:00 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Thu, 13 Dec 2018 13:46:00 -0800 Subject: RFR: 8215369: Jcstress pollute /var/tmp with temporary files. In-Reply-To: References: <96812CDD-ACE0-4CCE-9584-4FCEDFBBDEC3@oracle.com> <07659D3B-E3ED-4258-AB25-A53F9C9A7C13@oracle.com> <4A05EA09-17CE-4DBE-911F-2A0F5B1F0BD6@oracle.com> Message-ID: <5C12D318.4000507@oracle.com> +1 On 12/13/18, 1:43 PM, Igor Ignatyev wrote: > >> On Dec 13, 2018, at 1:36 PM, Leonid Mesnik wrote: >> >> >> >>> On Dec 13, 2018, at 1:24 PM, Igor Ignatyev> wrote: >>> >>> Hi Leonid, >>> >>> shouldn't 2nd -Djava.io.tmpdir go after '--jvmArgs' ? >>>> 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); >>> the rest looks good. >> It is right after -jvmArgs. > ok, it was just me misreading udiff. looks good and trivial to me, ship it. >> 105 extraFlags.add("--jvmArgs"); >> 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); >> then more --jvmArgs are added with other options in for loop. >> >>> since this is test-only fix, I'd prefer to get it integrated into jdk/jdk12 rather than into jdk/jdk. >>> >> ok. >> >> Leonid >> >>> Cheers, >>> -- Igor >>> >>>> On Dec 13, 2018, at 12:57 PM, Leonid Mesnik> wrote: >>>> >>>> Hi >>>> >>>> Could you please review following fix which set property "java.io.tmpdir" for jctress harness and it's forked VMs . So any temporary files are created in test directory and not in system temp. >>>> >>>> webrev: http://cr.openjdk.java.net/~lmesnik/8215369/webrev.00/ > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215369 > >>>> >>>> Leonid From leonid.mesnik at oracle.com Thu Dec 13 22:10:54 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 13 Dec 2018 14:10:54 -0800 Subject: RFR: 8215369: Jcstress pollute /var/tmp with temporary files. In-Reply-To: <5C12D318.4000507@oracle.com> References: <96812CDD-ACE0-4CCE-9584-4FCEDFBBDEC3@oracle.com> <07659D3B-E3ED-4258-AB25-A53F9C9A7C13@oracle.com> <4A05EA09-17CE-4DBE-911F-2A0F5B1F0BD6@oracle.com> <5C12D318.4000507@oracle.com> Message-ID: Igor, Misha Thank you for review. I'm pushing fix in jdk/jdk12 now. Leonid > On Dec 13, 2018, at 1:46 PM, Mikhailo Seledtsov wrote: > > +1 > > On 12/13/18, 1:43 PM, Igor Ignatyev wrote: >> >>> On Dec 13, 2018, at 1:36 PM, Leonid Mesnik wrote: >>> >>> >>> >>>> On Dec 13, 2018, at 1:24 PM, Igor Ignatyev> wrote: >>>> >>>> Hi Leonid, >>>> >>>> shouldn't 2nd -Djava.io.tmpdir go after '--jvmArgs' ? >>>>> 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); >>>> the rest looks good. >>> It is right after -jvmArgs. >> ok, it was just me misreading udiff. looks good and trivial to me, ship it. >>> 105 extraFlags.add("--jvmArgs"); >>> 106 extraFlags.add("-Djava.io.tmpdir=" + System.getProperty("user.dir")); >>> then more --jvmArgs are added with other options in for loop. >>> >>>> since this is test-only fix, I'd prefer to get it integrated into jdk/jdk12 rather than into jdk/jdk. >>>> >>> ok. >>> >>> Leonid >>> >>>> Cheers, >>>> -- Igor >>>> >>>>> On Dec 13, 2018, at 12:57 PM, Leonid Mesnik> wrote: >>>>> >>>>> Hi >>>>> >>>>> Could you please review following fix which set property "java.io.tmpdir" for jctress harness and it's forked VMs . So any temporary files are created in test directory and not in system temp. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8215369/webrev.00/ > >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215369 > >>>>> >>>>> Leonid From coleen.phillimore at oracle.com Thu Dec 13 23:49:03 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Dec 2018 18:49:03 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <4CEA4338-5D3D-4537-A80B-F7AC5420AFC7@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> <396B4688-DF16-47D5-9F53-31FC2524C4F3@oracle.com> <4f10af7b-a18a-ac95-6371-7b2c5c6f3a89@oracle.com> <4CEA4338-5D3D-4537-A80B-F7AC5420AFC7@oracle.com> Message-ID: <862b4cf5-3b33-b860-467b-097e78d45591@oracle.com> This last version is fine. Coleen On 12/6/18 5:04 PM, Kim Barrett wrote: >> On Dec 3, 2018, at 2:33 AM, David Holmes wrote: >> >> Hi Kim, >> >> On 3/12/2018 5:03 pm, Kim Barrett wrote: >>>> On Nov 27, 2018, at 4:34 PM, David Holmes wrote: >>>> >>>> On 28/11/2018 4:12 am, Kim Barrett wrote: >>>>> The "natural result" is that the derived class copy-assign code does >>>>> what it does. There are things that can go badly wrong here regarding >>>>> lifetimes of embedded object references. But the asserted restriction >>>>> on the allocated location of the object being copy-assigned to does >>>>> nothing that I can see to prevent those potential problems. >>>>> Can you provide a use-case where the assertion actually does something >>>>> useful? I removed it because I don?t think such a thing exists. >>>> AFAICS the only time it makes any sense to use assignment is with stack/embedded ResourceObj. The assert verifies that is the case. >>>> >>>> This seems a very clear cut use of an assert to ensure an API is not being misused. Does it prevent all possible misuse - no. But that's not a reason to remove what it does do. >>>> >>>> Do we ever need the assignment semantics? Maybe we should just prohibit it altogether? >>>> >>>> David >>> I don't understand why you might think that, and don't agree. Maybe >>> there are additional unstated and unenforced restrictions? >>> But part of my rejection of such a position is that a commonly >>> understood idiomatic operation like assignment should either just >>> work, or it shouldn't be allowed at all. Having a strange restriction >>> like this makes code much harder to understand. Make a new operation >>> for this restricted assignment. >> How do you make a special assignment operator? AFAIK there is only one op= > You spell it differently and forbid op=, e.g. something like > ResourceObj& assign_living_dangerously(const ResourceObj&); > ResourceObj& operator=(const ResourceObj&) = delete; // C++11 > >> ResourceObj is not a nice clean, well-defined class > I can certainly agree with that! > >> - it is one class that embodies the possibility of being one of three different kinds of "resource obj" (embedded/stack, arena-allocated, C-heap allocated) and the operations of copying and assignment don't make sense for all of these - hence the restriction. > A ResourceObj cannot have ownership of any contained allocated objects > (ownership in the sense of having responsibility for deallocation), > because the resource and arena mechanisms by design do not call the > destructor of relevant objects. (Such destructor avoidance is allowed > by C++ 3.8/4, so long as the destructor doesn't have any side effects > the application depends on.) > > So copy and copy-assign for a class derived from ResourceObj (and so > possibly resource allocated) can't really do anything more than a > shallow copy unless it can somehow know that it was not resource > allocated. The relevant API provided by ResourceObj is DEBUG_ONLY, so > can't be used for conditionalization, only assertion. > > Some classes (GrowableArray, for example), have their own mechanism > for determining allocation location and ResourceMark nesting when > relevant. GrowableArray is actually a pretty good example of how > things can go awry right now. It has the default copy constructor and > copy assignment operator, which are both wrong and better not be used > in some cases. > > Both copy and copy-assign when the copy source is on_C_heap will > result in double-free. (The ResourceObj copy-assign assert doesn't > help here.) > > Copy-assign to an on_C_heap object will leak the old data array. > (The ResourceObj copy-assign assert doesn't help very much here > either. The array can be stack/member allocated but configured to > allocate its data from the C heap; that's an entirely reasonable > configuration.) > > Both of those could be addressed by providing the appropriate > definition (similar to what, for example, std::vector does). > > Of course, resource/arena allocating a growable array configured to > use the C heap for its data is a mistake, since the destructor won't > be called to free the data, so leaking it. > > There are other cases that could be okay or not, and may or may not be > consistent with the ResourceObj copy-assign assertion. I think the > debug-only nesting checking done by GrowableArray is likely more > useful and correct than that ResourceObj assertion. > > I looked at a few actual uses. None were fatally wrong, but none were > good either. Some looked accidental. > >>> Unfortunately, there are already many uses of ResourceObj assignment. >>> I spent a bit of time exploring the idea of removing or replacing it, >>> but that seems way too hard for the intended scope of this cleanup. >>> (Note that for some of the existing uses it is not obvious whether the >>> restriction applies. Presumably it doesn't, since we're not hitting >>> the assert.) While I like the idea of prohibiting assignment (and >>> possibly copy too, which has many of the same issues), I'm not going >>> to pursue that beyond perhaps an RFE for future work. >>> BTW, as part of that exploration I found such gems as this, from >>> opto/cfgnode.cpp:962: >>> Node_Array node_map = new Node_Array(a); >> I can certainly understand people being confused about C++ allocation, construction and assignment. So I think the above acts as: >> >> Node_Array node_map; // Stack object using default constructor >> node_map.operator=(new Node_Array(a)); >> >> Which means the whole "new Node_Array(a)" is completely useless (does it cause a leak?) as operator= ignores it. > Nope. It is equivalent to > > Node_Array node_map(new Node_Array(a)); > > No operator= involved at all. Rather, node_map is a stack-allocated > Node_Array constructed via the conversion (not copy!) constructor > Node_Array(Node_Array*). That constructor default constructs the > ResourceObj base class object and initializes the Node_Array members > from the corresponding members of the argument object. > > The result of "new Node_Array(a)" is then dropped. Since it's a > ResourceObj, it was allocated in the resource area and will be > reclaimed when the innermost enclosing ResourceMark is exited. > > A better way to write this would be > > Node_Array node_map(a); > > >>> Anyway, even though I think the assert prevents some things that would >>> work (but should perhaps be avoided anyway), and doesn't prevent some >>> things that won't work, I'm willing to reinstate it in order to make >>> progress. >>> New webrevs: >>> full: http://cr.openjdk.java.net/~kbarrett/8213481/open.01/ >>> incr: http://cr.openjdk.java.net/~kbarrett/8213481/open.01.inc/ >> Thanks for doing that. > Anyway, this has gone somewhat astray from this particular change. > I'm not sure where I am, review-wise. I *think* both David and > Vladimir are okay with the latest version? > From coleen.phillimore at oracle.com Fri Dec 14 00:29:48 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Dec 2018 19:29:48 -0500 Subject: CFV: New hotspot Group Member: Chris Plummer In-Reply-To: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> References: <4C2EDFCF-6F0B-4BE3-A2DC-B1EDB2DE71C9@oracle.com> Message-ID: <9678e066-0766-172f-b46c-2a0230439040@oracle.com> Vote: yes On 12/12/18 5:58 PM, jesper.wilhelmsson at oracle.com wrote: > I hereby nominate Chris Plummer (OpenJDK user name: cjplummer) to Membership in the HotSpot Group. > > With a background in the embedded area, Chris has been working on the HotSpot project for over 8 years and is a Reviewer in the JDK Project with 76 changes. > > Votes are due by Wednesday, December 26, 2019 - yes, you can give a vote as your special holiday gift ;-) > > Only current Members of the HotSpot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Thanks, > /Jesper > > [1]http://openjdk.java.net/census#hotspot > [2]http://openjdk.java.net/groups/#member-vote > From Pengfei.Li at arm.com Fri Dec 14 07:43:02 2018 From: Pengfei.Li at arm.com (Pengfei Li (Arm Technology China)) Date: Fri, 14 Dec 2018 07:43:02 +0000 Subject: 8212043: Request for help with Submit Failure In-Reply-To: References: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> <48685738-ad99-f48d-72a7-ff87d20cfea9@oracle.com> Message-ID: Hi Andrew Dinn, Hope this is the last trial. http://cr.openjdk.java.net/~pli/rfr/8212043/webrev.03/ > I don't know for sure what the actual errors are but I suspect that the > relevant compilers will not bite on the expression (1.0/0.0). It seems that > defining +/-INFINITY in a portable way is going to require some work. Actually, the expression (1.0/0.0) is good but (1.0d/0.0d) is not. I've reproduced the failure by building jdk on MacBook. I got below error messages. ... src/hotspot/share/opto/type.cpp:453:35: error: invalid suffix 'd' on floating constant TypeD::POS_INF = TypeD::make(1.0d/0.0d); // Double positive infinity ... ISO/IEC 9899:1999[1] Chapter 6.4.4.2 Paragraph 4 said: "An unsuffixed floating constant has type double. If suffixed by the letter f or F, it has type float. If suffixed by the letter l or L, it has type long double." And there's no definition for floating constant suffixed by letter d or D. So this is probably a feature only in GCC but not supported by other compilers in solaris/macOS/Windows. I've also tried to compile a simple C program with 1.0D by AArch64 LLVM and it fails. In the new webrev, the suffix letter d is removed. And to be consistent with other definitions above like "TypeF::ONE", the suffix letter f is removed as well. This patch passed build test on Intel/Arm servers and MacBook. [1] ISO/IEC 9899:1999 http://www.dii.uchile.cl/~daespino/files/Iso_C_1999_definition.pdf -- Thanks, Pengfei From adinn at redhat.com Fri Dec 14 09:36:22 2018 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 14 Dec 2018 09:36:22 +0000 Subject: 8212043: Request for help with Submit Failure In-Reply-To: References: <840917749.554.1544553675668.JavaMail.root@sca00lvh.us.oracle.com> <48685738-ad99-f48d-72a7-ff87d20cfea9@oracle.com> Message-ID: Hi Pengfei, On 14/12/2018 07:43, Pengfei Li (Arm Technology China) wrote: > Hope this is the last trial. > http://cr.openjdk.java.net/~pli/rfr/8212043/webrev.03/ > >> I don't know for sure what the actual errors are but I suspect that the >> relevant compilers will not bite on the expression (1.0/0.0). It seems that >> defining +/-INFINITY in a portable way is going to require some work. > > Actually, the expression (1.0/0.0) is good but (1.0d/0.0d) is not. > I've reproduced the failure by building jdk on MacBook. I got below error messages. > ... > src/hotspot/share/opto/type.cpp:453:35: error: invalid suffix 'd' on floating constant > TypeD::POS_INF = TypeD::make(1.0d/0.0d); // Double positive infinity > ... > ISO/IEC 9899:1999[1] Chapter 6.4.4.2 Paragraph 4 said: "An unsuffixed floating constant has type double. If suffixed by the letter f or F, it has type float. If suffixed by the letter l or L, it has type long double." And there's no definition for floating constant suffixed by letter d or D. So this is probably a feature only in GCC but not supported by other compilers in solaris/macOS/Windows. I've also tried to compile a simple C program with 1.0D by AArch64 LLVM and it fails. > > In the new webrev, the suffix letter d is removed. And to be consistent with other definitions above like "TypeF::ONE", the suffix letter f is removed as well. This patch passed build test on Intel/Arm servers and MacBook. > > [1] ISO/IEC 9899:1999 http://www.dii.uchile.cl/~daespino/files/Iso_C_1999_definition.pdf That sounds pretty convincing to me. Thanks for doing such thorough research and testing. I'll try the latest patch and keep my fingers crossed that it works this time :-) regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From shade at redhat.com Fri Dec 14 11:21:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 14 Dec 2018 12:21:47 +0100 Subject: RFR (XS) [12] 8215374: 32-bit build failures after JDK-8181143 (Introduce diagnostic flag to abort VM on too long VM operations) In-Reply-To: References: Message-ID: Ping. I'd like to push this trivial (?) fix soon. -Aleksey On 12/13/18 10:42 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8215374 > > With all the excitement around fixing x86_32 issues before jdk12 fork today, I forgot to test my own > patch still builds on x86_32! It does not, because format specifiers are wrong. Fix: > > diff -r 408cdb799358 -r 1fb17b0a42c2 src/hotspot/share/runtime/vmThread.cpp > --- a/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:38 2018 +0100 > +++ b/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:39 2018 +0100 > @@ -206,7 +206,7 @@ > if (is_armed()) { > jlong delay = (os::javaTimeMillis() - _arm_time); > if (delay > AbortVMOnVMOperationTimeoutDelay) { > - fatal("VM operation took too long: " SIZE_FORMAT " ms (timeout: " SIZE_FORMAT " ms)", > + fatal("VM operation took too long: " JLONG_FORMAT " ms (timeout: " INTX_FORMAT " ms)", > delay, AbortVMOnVMOperationTimeoutDelay); > } > } > > This goes to jdk/jdk12, because the bug is there. This build failure is currently masked by another > x86_32 build failure, fixed by JDK-8215353, which would be pushed before this fix. > > Testing: {x86_32, x86_64} build, AbortVMOnVMOperationTimeout test > > Thanks, > -Aleksey > From rwestrel at redhat.com Fri Dec 14 11:28:54 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 14 Dec 2018 12:28:54 +0100 Subject: RFR (XS) [12] 8215374: 32-bit build failures after JDK-8181143 (Introduce diagnostic flag to abort VM on too long VM operations) In-Reply-To: References: Message-ID: <87zht8fitl.fsf@redhat.com> > With all the excitement around fixing x86_32 issues before jdk12 fork today, I forgot to test my own > patch still builds on x86_32! It does not, because format specifiers are wrong. Fix: > > diff -r 408cdb799358 -r 1fb17b0a42c2 src/hotspot/share/runtime/vmThread.cpp > --- a/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:38 2018 +0100 > +++ b/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:39 2018 +0100 > @@ -206,7 +206,7 @@ > if (is_armed()) { > jlong delay = (os::javaTimeMillis() - _arm_time); > if (delay > AbortVMOnVMOperationTimeoutDelay) { > - fatal("VM operation took too long: " SIZE_FORMAT " ms (timeout: " SIZE_FORMAT " ms)", > + fatal("VM operation took too long: " JLONG_FORMAT " ms (timeout: " INTX_FORMAT " ms)", > delay, AbortVMOnVMOperationTimeoutDelay); > } > } That looks good and trivial to me. Roland. From shade at redhat.com Fri Dec 14 11:58:24 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 14 Dec 2018 12:58:24 +0100 Subject: RFR (XS) [12] 8215374: 32-bit build failures after JDK-8181143 (Introduce diagnostic flag to abort VM on too long VM operations) In-Reply-To: <87zht8fitl.fsf@redhat.com> References: <87zht8fitl.fsf@redhat.com> Message-ID: On 12/14/18 12:28 PM, Roland Westrelin wrote: > >> With all the excitement around fixing x86_32 issues before jdk12 fork today, I forgot to test my own >> patch still builds on x86_32! It does not, because format specifiers are wrong. Fix: >> >> diff -r 408cdb799358 -r 1fb17b0a42c2 src/hotspot/share/runtime/vmThread.cpp >> --- a/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:38 2018 +0100 >> +++ b/src/hotspot/share/runtime/vmThread.cpp Thu Dec 13 22:32:39 2018 +0100 >> @@ -206,7 +206,7 @@ >> if (is_armed()) { >> jlong delay = (os::javaTimeMillis() - _arm_time); >> if (delay > AbortVMOnVMOperationTimeoutDelay) { >> - fatal("VM operation took too long: " SIZE_FORMAT " ms (timeout: " SIZE_FORMAT " ms)", >> + fatal("VM operation took too long: " JLONG_FORMAT " ms (timeout: " INTX_FORMAT " ms)", >> delay, AbortVMOnVMOperationTimeoutDelay); >> } >> } > > That looks good and trivial to me. Thank you! Pushed. -Aleksey From dean.long at oracle.com Sat Dec 15 00:59:33 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 14 Dec 2018 16:59:33 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 Message-ID: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8214583 http://cr.openjdk.java.net/~dlong/8214583/webrev This change includes two new regression test that demonstrate the problem, and a fix that allows the tests to pass. The problem happens when the JIT compiler's escape analysis eliminates the allocation of the AccessControlContext object passed to doPrivileged.? The compiler thinks this is safe because it does not see that the object "escapes".? However, getContext needs to be able to find the object using a stack walk, so we need a way to tell the compiler that it does indeed escape.? To do this we pass the value to a native method that does nothing. Microbenchmark results: jdk12-b18: Benchmark??????????????? Mode? Cnt??? Score?? Error? Units DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op jdk12-b19: Benchmark??????????????? Mode? Cnt? Score??? Error? Units DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op this fix: Benchmark??????????????? Mode? Cnt? Score??? Error? Units DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op dl From claes.redestad at oracle.com Sat Dec 15 14:48:38 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 15 Dec 2018 15:48:38 +0100 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> Message-ID: Hi Dean, to avoid escape analysis-eliminated allocations in the past @DontInline has been sufficient. This means a simpler patch (no changes to native code needed - added assertions notwithstanding) and passes your tests with C2 (it'd concern me if Graal's EA sees through this trick, as it might break some existing places where DontInline is used to this effect): /** * The value needs to be physically located in the frame, so that it * can be found by a stack walk. */ @Hidden @DontInline private static void ensureMaterializedForStackWalk(Object o) {} Thanks! /Claes On 2018-12-15 01:59, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8214583 > http://cr.openjdk.java.net/~dlong/8214583/webrev > > This change includes two new regression test that demonstrate the > problem, and a fix that allows the tests > to pass. > > The problem happens when the JIT compiler's escape analysis eliminates > the allocation of the AccessControlContext object passed to > doPrivileged.? The compiler thinks this is safe because it does not see > that the object "escapes".? However, getContext needs to be able to find > the object using a stack walk, so we need a way to tell the compiler > that it does indeed escape.? To do this we pass the value to a native > method that does nothing. > > Microbenchmark results: > > jdk12-b18: > > Benchmark??????????????? Mode? Cnt??? Score?? Error? Units > DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op > DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op > > > jdk12-b19: > > Benchmark??????????????? Mode? Cnt? Score??? Error? Units > DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op > DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op > > this fix: > > Benchmark??????????????? Mode? Cnt? Score??? Error? Units > DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op > DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op > > > dl From david.holmes at oracle.com Mon Dec 17 00:06:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Dec 2018 10:06:22 +1000 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> Message-ID: <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> On 15/12/2018 10:59 am, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8214583 > http://cr.openjdk.java.net/~dlong/8214583/webrev > > This change includes two new regression test that demonstrate the > problem, and a fix that allows the tests > to pass. > > The problem happens when the JIT compiler's escape analysis eliminates > the allocation of the AccessControlContext object passed to > doPrivileged.? The compiler thinks this is safe because it does not see > that the object "escapes". Then surely the compiler's notion of "escapes" needs to be updated! David ----- ? However, getContext needs to be able to find > the object using a stack walk, so we need a way to tell the compiler > that it does indeed escape.? To do this we pass the value to a native > method that does nothing. > > Microbenchmark results: > > jdk12-b18: > > Benchmark??????????????? Mode? Cnt??? Score?? Error? Units > DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op > DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op > > > jdk12-b19: > > Benchmark??????????????? Mode? Cnt? Score??? Error? Units > DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op > DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op > > this fix: > > Benchmark??????????????? Mode? Cnt? Score??? Error? Units > DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op > DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op > > > dl From david.holmes at oracle.com Mon Dec 17 00:57:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Dec 2018 10:57:34 +1000 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> <236420f2-cd56-75a7-db52-59a7ef9f9be3@oracle.com> Message-ID: Hi Magnus, Thanks for explaining how addition of JNIEXPORT may have started this problem. One follow up: >> This will also need a CSR request due to the change in linking behaviour. > I'm not sure what you mean by this..? My entire intention here is that we should make no changes to documented interfaces; this is strictly an internal change, so no CSR should be needed. Also, I don't understand what you mean by "linking behavior"? A CSR request is also required for behavioural changes - which this is. Someone linking an "internal only" function today may get an error if JNICALL is removed tomorrow. This may be acceptable but that is what the CSR request establishes. Thanks, David On 13/12/2018 8:37 pm, Magnus Ihse Bursie wrote: > On 2018-12-12 13:17, David Holmes wrote: >> Okay I went away and did some homework ... >> >> Let me back up a bit and see if I have the evolution of this correctly. >> >> The native implementation of Java methods should be declared and >> defined with JNIEXPORT and JNICALL. >> >> JNIEXPORT controls the export visibility so they can be linked. >> >> JNICALL controls the calling convention and is needed so that the >> JVM's calling convention matches the native code. [This part was >> unclear to me.] > Yes. And JNICALL is empty on all platforms except Windows 32, that's why > we're only seeing issues about mismatch there. >> >> Other native methods exported from the same (or different) native >> libraries may also be decorated with JNIEXPORT and JNICALL. But in >> places there is a mismatch between the declaration in the header and >> the definition leading to errors. > Yes; this mismatch has typically happened when the linking has not been > done by simply including the relevant header file, but either > duplicating the definition, or looking up the symbol dynamically. > Otherwise things should basically work out. >> >> There are two main types of such native functions: >> >> a) those publicly defined in the various native APIs: JNI itself >> (jni.h), JVM TI (jvmti.h), AWT (jawt.h) ... >> >> b) those intended only for use by other native code within the JDK >> libraries (JLI_* - though I note Alan's comment re javafxpackager, >> others??) >> >> and a possible third type are callback functions like the JPLISAgent >> event handlers (e.g. eventHandlerVMInit). > > I'm not sure I understand what the third type is, but if it is a > publically defined API (which, at a first look, the JPLISAgent API seems > to be), then it's part of catagory a, otherwise it's part of category b. > > I must also state that my initial proposal didn't separate these two > cases. I had in mind only the b cases -- I have no intention of changing > public specifications, but I did not express this in my proposal. That > might have been one source of confusion. I apologize for this omission. >> >> There is a view that no native method that isn't the native-half of a >> Java method should use JNICALL. [Okay I can see how that makes sense - >> regardless of the actual calling convention used marking such methods >> as "must use the same calling convention as the VM native method call" >> is somewhat meaningless. Yet obviously the public native APIs do >> specify JNICALL so this is not a hard and fast rule. Further the >> callbacks must also follow a convention known to the VM doing the >> calling!] > Yes, or rather the rule is "only native half Java methods should use > JNICALL, and also all public APIs where so is specified". > >> >> Where we have (introduced?) a discrepancy between the definition and >> declaration the approach has been (because of the previous view) to >> remove JNICALL. [This should only relate to methods of kind (b) above.] > Actually, it's not so much as we have *removed* JNICALL, as that we have > *introduced* JNIEXPORT. When I removed the map files, I also removed the > .def files and /export command lines for Windows. As a result, I needed > to add JNIEXPORT to a lot of places. Initially, I followed the rule of > adding JNICALL to those calls -- but I can surely have missed a couple > of places, since things will work fine anyway, as long as caller and > callee agree on the calling convention; and a mismatch can only happen > on Windows 32, which we do not build and test. So there is a risk that > even with the initial patch, not all newly added JNIEXPORTs had JNICALL. > > Then, it turned out, that a lot of this code did not compile with > Windows 32. With no deeper analysis of the flaw, the blame was put on > the absence or presence of JNICALL, and a series of patches were made > where JNICALL was more or less arbitrarily added or removed, until > things "worked". This should have been a warning sign, and I was > increasingly uneasy about all these changes, but I hadn't spent enough > time to realize what the core issue was or how to resolve it properly. > >> >> That leaves those methods with JNIEXPORT only. >> >> That seems wrong to you because they are not "JNI methods", so you >> want to replace with JDK_EXPORT to make it clear. [Ok - this seems >> reasonable.] > Yes. And given the fact that we have a bunch of "non-JNI methods" that > use the JNIEXPORT...JNICALL pattern, another way to interpret the > JDK_EXPORT semantics is that this function is exported for usage *in the > JDK*, but not for public consumption. >> >> If JNICALL is removed from those native methods and they are currently >> linked by external applications, those applications may stop working. >> But this only affects win32 (as its the only platform where JNICALL is >> different to the default C/C++ calling convention?) so your position >> is this is acceptable breakage - and would only affect >> unsupported/undocumented APIs. > Yes. In fact, it's possible that at least some of the breakage that > occurred was due to new *addition* of JNICALL, which was not present > before. We might have had something like (I'm making this specific > example up) a function "void ZIP_OpenFile(char* name)" with no > decoration at all, and a "/export:ZIP_OpenFile" on the command line, and > a ZIP_OpenFile entry in the unix map file. And I converted this to > "JNIEXPORT void JNICALL ZIP_OpenFile(char* name)", which de facto -- > although I didn't fully realize this at the time, changed the calling > convention and name decoration on Windows 32. When something broke, > perhaps because the user of ZIP_OpenFile did not include the proper > header file with the JNICALL modifier, the solution was to remove the > JNICALL part. > > And of all the bug reports I've seen on this, the issues has been > internal linking only, i.e. problems building the JDK, not complaints > that external tools tried to use internal interfaces and failed. So yes, > my position is if this should break things, tough shit. That, of > courses, requires that we do not change public APIs, so we need to be > careful not to mess with those. >> >> Does that sum it up? > Yep, I think so. >> >> We still need to be sure that we're only changing functions of type >> (b) above. > Yes, definitely. >> >> And I note that this has been driven by the choice to remove JNICALL >> where there was a discrepancy - that leads to the visibility of the >> two kinds of methods. If it had been chosen to add JNICALL where >> missing instead, then we may not have been having this conversation. > Actually, as I said, this has more been the result of a lot of added > JNICALL where previously there was none. > > An alternative course of action is the make sure that *all* calls use > the JNIEXPORT...JNICALL pattern, even type b ones, and that we fix all > parts of code to actually accept the decorated names on Windows 32. This > will lead to a lot more code changes, like the fix for JDK-8214122 (JDWP > is broken on 32 bit Windows: transport library missing onLoad entry). > And all this special case handling will be needed only on Windows 32, > which is a platform we do not want to spend to much time or effort on. > And finally, I think doing so would make us miss out on an opportunity > to make the semantics clearer. >> >> This will also need a CSR request due to the change in linking behaviour. > I'm not sure what you mean by this..? My entire intention here is that > we should make no changes to documented interfaces; this is strictly an > internal change, so no CSR should be needed. Also, I don't understand > what you mean by "linking behavior"? > > /Magnus >> >> Cheers, >> David >> ----- >> >> On 12/12/2018 9:03 pm, Magnus Ihse Bursie wrote: >>> >>> >>> On 2018-12-11 23:47, David Holmes wrote: >>>> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >>>>> >>>>> >>>>> On 2018-12-11 00:23, David Holmes wrote: >>>>>> Hi Magnus, >>>>>> >>>>>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>>>>> I propose that we introduce a new define, available to all JDK >>>>>>> native files (Hotspot included), called JDK_EXPORT. The behavior >>>>>>> of this symbol will be very similar (as of now, in fact >>>>>>> identical) to JNIEXPORT; however, the semantics will not. >>>>>>> >>>>>>> Currently, we "mis-use" the JNIEXPORT define to mark a function >>>>>>> for exporting from the library. The problem with this is that >>>>>>> JNIEXPORT is part of the JNI interface, and is supposed to be >>>>>>> used when C programs interact with Java. And, when doing this, >>>>>>> the function should be fully decorated like this: "JNIEXPORT foo >>>>>>> JNICALL". >>>>>> >>>>>> I've seen a lot of the emails on this issue and I don't fully >>>>>> understand what has been going wrong. But the intent is obviously >>>>>> the JNIEXPORT represents what is needed to export this function >>>>>> for use by JNI, while JNICALL defines the calling convention. I >>>>>> agree there may be some mistmatch when functions are actually not >>>>>> intended for general export outside the JDK but are only for >>>>>> internal JDK use. >>>>>> >>>>>>> We do have many such JNI exports in our native libraries, but we >>>>>>> also have a lot of other, non-JNI exports, where one native >>>>>>> library just provides an interface to other libraries. In these >>>>>>> cases, we have still used JNIEXPORT for the functionality of >>>>>>> getting the function exported, but we have not been consistent in >>>>>>> our use of JNICALL. This has caused us way too much trouble for >>>>>>> something that should Just Work. >>>>>> >>>>>> Are you suggesting that the interface between different libraries >>>>>> in the JDK should not be a JNI interface? Is this because you >>>>>> think the functions in these libraries are only for JDK internal >>>>>> use or ... ?? >>>>>> >>>>>>> I therefore propose that we define "JDK_EXPORT", with the same >>>>>>> behavior as JNIEXPORT (that is, flagging the function for >>>>>>> external visibility in the resulting native library), but which >>>>>>> is *not* supposed to be exported to Java code using JNI, nor >>>>>>> supposed to be decorated with >>>>>> >>>>>> Just a clarification there. JNI functions are not exported to Java >>>>>> code, they are exported to native code. Java code can declare >>>>>> native methods and those native methods must be written as JNI >>>>>> functions, but that's not what we are discussing. Libraries expose >>>>>> a JNI interface (a set of functions in the library) that can be >>>>>> called by application native code, using JNI. >>>>> We're apparently looking at "JNI" and "exporting" from two opposite >>>>> sides here. :-) Just to make everything clear: If I have a Java class >>>>> class MyClass { >>>>> ?? public static void native myNativeFunc(); >>>>> } >>>>> >>>>> then I have one half of the JNI function, the Java half. This must >>>>> be matched by a corresponding implementation in native, like this: >>>>> JNIEXPORT void JNICALL >>>>> Java_MyClass_myNativeFunc(void) { >>>>> // ... do stuff >>>>> } >>>>> >>>>> And this is the native half of the JNI function. Right? Let's leave >>>>> aside which side is "exporting" to the other for now. :-) >>>>> >>>>> This way of setting up native functions that can be called from >>>>> Java is what I refer to as JNI. And when you declare a native JNI >>>>> function, you *must* use both JNIEXPORT and JNICALL. Alright? >>>>> >>>>> We do have a lot of those functions in our native libraries. And >>>>> they are correct just the way they are. >>>> >>>> Yep all well and good. A function declared native in Java must have >>>> an implementation as you describe. But not all native functions >>>> exist to provide the native-half of a Java native function! >>>> >>>>> However, we also have *other* native functions, that are exported, >>>>> not as JNI functions that should be called from Java, but as normal >>>>> native library functions that should be called by other native >>>>> code. Okay so far? And *those* functions have been problematic in >>>>> how we decorate >>>> >>>> But there are again two cases. Those functions exported from a >>>> library that are expected to be called from external code using the >>>> JNI interface mechanism - such as all the JNI functions and JVM TI >>>> functions we export from the JVM - and those "exported" for access >>>> between libraries within the JDK (such as all the JVM_* functions in >>>> libjvm). >>>> >>>> I think it is only the second group that should be addressed by your >>>> JDK_EXPORT proposal - though I'm not completely clear exactly how to >>>> identify them. >>> Alright, now I think we're on the same page again. :) >>> >>> Yes, this is what I'm saying. I'm not proposing to modify public APIs. >>> >>> You identify external APIs by the fact that they are documented. And >>> if you are unsure, you ask Jon. ;-) >>> >>>> >>>>> them. My proposal is that we *refrain* from using JNIEXPORT for >>>>> those functions, and instead use JDK_EXPORT as name for the macro >>>>> that decorates them as exported. That way, we can clearly see that >>>>> a function like this: >>>>> >>>>> JDK_EXPORT void >>>>> JLI_ReadEnv(char* env); >>>>> >>>>> is correctly declared, and will be exported to other native >>>>> libraries, but not to Java. >>>> >>>> The issue is not whether it is "exported to Java"** but whether it >>>> is exported using the JNI mechanism such that other native code >>>> calls it using the JNI mechanism. >>>> >>>> ** There is no way to write a native method declaration in Java such >>>> that it would be linked to the JLI_ReadEnv function. The naming is >>>> all wrong, as is the signature. >>>> >>>>> Just to clarify, this has nothing to do with if this is a >>>>> officially supported API or not. In general though, I assume that >>>>> most (if not all?) of our exported functions (apart from the JNI_* >>>>> stuff) is supposed to be consumed by other libraries in the JDK, >>>>> and is not a public API. >>>> >>>> I think it varies library by library. You may need native >>>> application code that can call directly into native JDK libraries. >>>> JLI is the Java Launcher Interface - I think it was introduced to >>>> make it easier for other launchers to be created. Native agents may >>>> need access to libmanagement or libjdwp functions. Native graphics >>>> code may need access to the JDK graphics library. Some of these >>>> accesses may be unsupported and undocumented, but I don't think you >>>> can just cut them all off. >>> If they are unsupported and undocumented, I sure as h*ll can cut them >>> all off! :-) >>> >>> Besides, and let me re-iterate this, the only change that will happen >>> by removing JNICALL, is on Windows 32. No other platform will be >>> affected. There is no official support for Windows 32 anymore. >>> There's some low-effort community work done on keeping Windows 32 >>> alive, but it's not backed by any major player. Right now, they are >>> taking a lot of hits *due to the fact* that we have not sorted this >>> out, and waste a lot of their precious effort trying to fix this >>> piecemeal. If we do fix things according to this proposal, then it's >>> at least clear how things *should* work. If it turns out that there's >>> some code out there in the wild, running on Windows 32, that uses an >>> undocumented and unsupported function call, and it breaks -- well, >>> sucks to be them. They'll just have to do what everyone does who uses >>> an undocumented interface that suddenly changes: update their code. >>> >>> /Magnus >>> >>>> >>>> David >>>> >>>>> >>>>> /Magnus >>>>> >>>>> >>>>> >>>>>> >>>>>>> JNICALL. All current instances of JNIEXPORT which is not pure JNI >>>>>>> native functions should be changed to use JDK_EXPORT instead. >>>>>>> >>>>>>> I further propose that this macro should reside in a new file >>>>>>> "jdk.h", placed in the new directory >>>>>>> src/java.base/share/native/include/internal. This header file >>>>>>> path will automatically be provided to all native libraries, but >>>>>>> not copied to the JDK being built. (The existence of a >>>>>>> "include/internal" directory with this behavior has been >>>>>>> discussed before. There are more files that ought to be moved >>>>>>> there, if/when it is created.) I believe in many cases the >>>>>>> #include "jni.h" can be just modified to #include "#jdk.h", since >>>>>>> most native code will not require "jni.h" unless actually doing >>>>>>> JNI calls -- most have included this file to get the JNIEXPORT >>>>>>> macro, which would explain the pervasive use of #include "jni.h" >>>>>>> in our code base. >>>>>> >>>>>> jni.h also defines all of the types used by the JNI. Those types >>>>>> are pervsive to the native code used throughout the JDK. >>>>>> >>>>>>> Thoughts? >>>>>> >>>>>> I think we need to understand the problems on Windows that >>>>>> prompted all this. Then I think we need to look at exactly how >>>>>> jni.h and JNIEXPORT etc are being used and understand whether this >>>>>> is truly an exported interface or not. >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>>> /Magnus >>>>>>> >>>>> >>> > From Nick.Gasson at arm.com Mon Dec 17 02:00:26 2018 From: Nick.Gasson at arm.com (Nick Gasson (Arm Technology China)) Date: Mon, 17 Dec 2018 02:00:26 +0000 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> Message-ID: Hi Dean, Sorry for the delay. This is submitted on behalf of ARM Ltd. Thanks, Nick From: dean.long at oracle.com Sent: 13 December 2018 12:01 To: Nick Gasson (Arm Technology China) ; hotspot-dev at openjdk.java.net; Boris Ulasevich Cc: hotspot-compiler-dev at openjdk.java.net compiler ; nd ; aarch32-port-dev at openjdk.java.net Subject: Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 Nick, can you please confirm that you are covered under the ? ARM Ltd. - OpenJDK, MySQL OCA agreement? I'm guessing Arm Technology China is under ARM Ltd., judging by your email domain, but please confirm. dl On 12/11/18 5:30 PM, dean.long at oracle.com wrote: I can help you commit it, but first I think another review would be good. dl On 12/11/18 5:15 PM, Nick Gasson (Arm Technology China) wrote: Hi Nick. This change looks good to me. Thanks Dean. Could you help me to commit this patch if it doesn't need other reviews? Nick -----Original Message----- From: dean.long at oracle.com Sent: 12 December 2018 06:05 To: Nick Gasson (Arm Technology China) ; hotspot- dev at openjdk.java.net; Boris Ulasevich Cc: nd ; hotspot-compiler-dev at openjdk.java.net compiler ; aarch32-port- dev at openjdk.java.net Subject: Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 Hi Nick. This change looks good to me. dl On 12/2/18 10:06 PM, Nick Gasson wrote: Hi, Could someone please help me review this fix for a test failure on ARM32: Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 Webrev: http://cr.openjdk.java.net/~njian/8214512/webrev.0/ This fixes two assertion failures related to biased locking when C2's bias inlining feature is disabled (-XX:-OptoBiasInlining): MacroAssembler::atomic_cas_bool takes three register arguments plus a temporary register for use in the CAS loop. If the caller passes `noreg' as this temporary register (which happens in the !OptoBiasInlining call path from MacroAssembler::fast_lock) then MacroAssembler::atomic_cas_bool will default to using LR as a temporary. But it's possible with C2 that LR is one of the other three arguments which then triggers an assert_different_registers assertion failure. Fix this by supplying an additional scratch register to MacroAssembler::fast_lock if !OptoBiasInlining. In the !OptoBiasInlining case MacroAssembler::fast_lock calls MacroAssembler::biased_locking_enter to handle the case where the mark word contains the biased lock pattern which fast_lock wouldn't otherwise see if OptoBiasInlining was true. However in the case that we fail to acquire the biased lock this falls through to label `failed' and runs the normal fast_lock code that implicitly assumes the mark word does not have the bias pattern. This means we can end up with a BasicLock where _displaced_header contains the biased lock pattern which is an illegal state and subsequently triggers an assertion failure in ObjectSynchronizer::fast_exit. The right thing to do here is branch to `done' whenever the bias lock pattern is present and let the runtime handle the failure case. Also edited the comment describing MacroAssembler::biased_locking_enter to more accurately describe what it does (the comment about `slow_case' being optional is wrong). Jtreg test compiler/c2/Test8062950.java now passes on ARM32 with this patch. Thanks, Nick From dean.long at oracle.com Mon Dec 17 02:49:49 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 18:49:49 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> Message-ID: <61f4a254-be9d-b0b2-3525-8caa247d9556@oracle.com> On 12/16/18 4:06 PM, David Holmes wrote: > On 15/12/2018 10:59 am, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8214583 >> http://cr.openjdk.java.net/~dlong/8214583/webrev >> >> This change includes two new regression test that demonstrate the >> problem, and a fix that allows the tests >> to pass. >> >> The problem happens when the JIT compiler's escape analysis >> eliminates the allocation of the AccessControlContext object passed >> to doPrivileged.? The compiler thinks this is safe because it does >> not see that the object "escapes". > > Then surely the compiler's notion of "escapes" needs to be updated! > The compiler can inline the callee method and see that the value doesn't escape.? This is a valid optimization in cases where the callee method is known. dl > David > ----- > > ? However, getContext needs to be able to find >> the object using a stack walk, so we need a way to tell the compiler >> that it does indeed escape.? To do this we pass the value to a native >> method that does nothing. >> >> Microbenchmark results: >> >> jdk12-b18: >> >> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >> >> >> jdk12-b19: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >> >> this fix: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >> >> >> dl From dean.long at oracle.com Mon Dec 17 02:51:48 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 18:51:48 -0800 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> <236420f2-cd56-75a7-db52-59a7ef9f9be3@oracle.com> Message-ID: If we are adding JDK_EXPORT, does it make sense to add JDK_CALL as well? dl On 12/16/18 4:57 PM, David Holmes wrote: > Hi Magnus, > > Thanks for explaining how addition of JNIEXPORT may have started this > problem. > > One follow up: > >>> This will also need a CSR request due to the change in linking >>> behaviour. >> I'm not sure what you mean by this..? My entire intention here is >> that we should make no changes to documented interfaces; this is >> strictly an internal change, so no CSR should be needed. Also, I >> don't understand what you mean by "linking behavior"? > > A CSR request is also required for behavioural changes - which this > is. Someone linking an "internal only" function today may get an error > if JNICALL is removed tomorrow. This may be acceptable but that is > what the CSR request establishes. > > Thanks, > David > > > On 13/12/2018 8:37 pm, Magnus Ihse Bursie wrote: >> On 2018-12-12 13:17, David Holmes wrote: >>> Okay I went away and did some homework ... >>> >>> Let me back up a bit and see if I have the evolution of this correctly. >>> >>> The native implementation of Java methods should be declared and >>> defined with JNIEXPORT and JNICALL. >>> >>> JNIEXPORT controls the export visibility so they can be linked. >>> >>> JNICALL controls the calling convention and is needed so that the >>> JVM's calling convention matches the native code. [This part was >>> unclear to me.] >> Yes. And JNICALL is empty on all platforms except Windows 32, that's >> why we're only seeing issues about mismatch there. >>> >>> Other native methods exported from the same (or different) native >>> libraries may also be decorated with JNIEXPORT and JNICALL. But in >>> places there is a mismatch between the declaration in the header and >>> the definition leading to errors. >> Yes; this mismatch has typically happened when the linking has not >> been done by simply including the relevant header file, but either >> duplicating the definition, or looking up the symbol dynamically. >> Otherwise things should basically work out. >>> >>> There are two main types of such native functions: >>> >>> a) those publicly defined in the various native APIs: JNI itself >>> (jni.h), JVM TI (jvmti.h), AWT (jawt.h) ... >>> >>> b) those intended only for use by other native code within the JDK >>> libraries (JLI_* - though I note Alan's comment re javafxpackager, >>> others??) >>> >>> and a possible third type are callback functions like the JPLISAgent >>> event handlers (e.g. eventHandlerVMInit). >> >> I'm not sure I understand what the third type is, but if it is a >> publically defined API (which, at a first look, the JPLISAgent API >> seems to be), then it's part of catagory a, otherwise it's part of >> category b. >> >> I must also state that my initial proposal didn't separate these two >> cases. I had in mind only the b cases -- I have no intention of >> changing public specifications, but I did not express this in my >> proposal. That might have been one source of confusion. I apologize >> for this omission. >>> >>> There is a view that no native method that isn't the native-half of >>> a Java method should use JNICALL. [Okay I can see how that makes >>> sense - regardless of the actual calling convention used marking >>> such methods as "must use the same calling convention as the VM >>> native method call" is somewhat meaningless. Yet obviously the >>> public native APIs do specify JNICALL so this is not a hard and fast >>> rule. Further the callbacks must also follow a convention known to >>> the VM doing the calling!] >> Yes, or rather the rule is "only native half Java methods should use >> JNICALL, and also all public APIs where so is specified". >> >>> >>> Where we have (introduced?) a discrepancy between the definition and >>> declaration the approach has been (because of the previous view) to >>> remove JNICALL. [This should only relate to methods of kind (b) above.] >> Actually, it's not so much as we have *removed* JNICALL, as that we >> have *introduced* JNIEXPORT. When I removed the map files, I also >> removed the .def files and /export command lines for Windows. As a >> result, I needed to add JNIEXPORT to a lot of places. Initially, I >> followed the rule of adding JNICALL to those calls -- but I can >> surely have missed a couple of places, since things will work fine >> anyway, as long as caller and callee agree on the calling convention; >> and a mismatch can only happen on Windows 32, which we do not build >> and test. So there is a risk that even with the initial patch, not >> all newly added JNIEXPORTs had JNICALL. >> >> Then, it turned out, that a lot of this code did not compile with >> Windows 32. With no deeper analysis of the flaw, the blame was put on >> the absence or presence of JNICALL, and a series of patches were made >> where JNICALL was more or less arbitrarily added or removed, until >> things "worked". This should have been a warning sign, and I was >> increasingly uneasy about all these changes, but I hadn't spent >> enough time to realize what the core issue was or how to resolve it >> properly. >> >>> >>> That leaves those methods with JNIEXPORT only. >>> >>> That seems wrong to you because they are not "JNI methods", so you >>> want to replace with JDK_EXPORT to make it clear. [Ok - this seems >>> reasonable.] >> Yes. And given the fact that we have a bunch of "non-JNI methods" >> that use the JNIEXPORT...JNICALL pattern, another way to interpret >> the JDK_EXPORT semantics is that this function is exported for usage >> *in the JDK*, but not for public consumption. >>> >>> If JNICALL is removed from those native methods and they are >>> currently linked by external applications, those applications may >>> stop working. But this only affects win32 (as its the only platform >>> where JNICALL is different to the default C/C++ calling convention?) >>> so your position is this is acceptable breakage - and would only >>> affect unsupported/undocumented APIs. >> Yes. In fact, it's possible that at least some of the breakage that >> occurred was due to new *addition* of JNICALL, which was not present >> before. We might have had something like (I'm making this specific >> example up) a function "void ZIP_OpenFile(char* name)" with no >> decoration at all, and a "/export:ZIP_OpenFile" on the command line, >> and a ZIP_OpenFile entry in the unix map file. And I converted this >> to "JNIEXPORT void JNICALL ZIP_OpenFile(char* name)", which de facto >> -- although I didn't fully realize this at the time, changed the >> calling convention and name decoration on Windows 32. When something >> broke, perhaps because the user of ZIP_OpenFile did not include the >> proper header file with the JNICALL modifier, the solution was to >> remove the JNICALL part. >> >> And of all the bug reports I've seen on this, the issues has been >> internal linking only, i.e. problems building the JDK, not complaints >> that external tools tried to use internal interfaces and failed. So >> yes, my position is if this should break things, tough shit. That, of >> courses, requires that we do not change public APIs, so we need to be >> careful not to mess with those. >>> >>> Does that sum it up? >> Yep, I think so. >>> >>> We still need to be sure that we're only changing functions of type >>> (b) above. >> Yes, definitely. >>> >>> And I note that this has been driven by the choice to remove JNICALL >>> where there was a discrepancy - that leads to the visibility of the >>> two kinds of methods. If it had been chosen to add JNICALL where >>> missing instead, then we may not have been having this conversation. >> Actually, as I said, this has more been the result of a lot of added >> JNICALL where previously there was none. >> >> An alternative course of action is the make sure that *all* calls use >> the JNIEXPORT...JNICALL pattern, even type b ones, and that we fix >> all parts of code to actually accept the decorated names on Windows >> 32. This will lead to a lot more code changes, like the fix for >> JDK-8214122 (JDWP is broken on 32 bit Windows: transport library >> missing onLoad entry). And all this special case handling will be >> needed only on Windows 32, which is a platform we do not want to >> spend to much time or effort on. And finally, I think doing so would >> make us miss out on an opportunity to make the semantics clearer. >>> >>> This will also need a CSR request due to the change in linking >>> behaviour. >> I'm not sure what you mean by this..? My entire intention here is >> that we should make no changes to documented interfaces; this is >> strictly an internal change, so no CSR should be needed. Also, I >> don't understand what you mean by "linking behavior"? >> >> /Magnus >>> >>> Cheers, >>> David >>> ----- >>> >>> On 12/12/2018 9:03 pm, Magnus Ihse Bursie wrote: >>>> >>>> >>>> On 2018-12-11 23:47, David Holmes wrote: >>>>> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >>>>>> >>>>>> >>>>>> On 2018-12-11 00:23, David Holmes wrote: >>>>>>> Hi Magnus, >>>>>>> >>>>>>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>>>>>> I propose that we introduce a new define, available to all JDK >>>>>>>> native files (Hotspot included), called JDK_EXPORT. The >>>>>>>> behavior of this symbol will be very similar (as of now, in >>>>>>>> fact identical) to JNIEXPORT; however, the semantics will not. >>>>>>>> >>>>>>>> Currently, we "mis-use" the JNIEXPORT define to mark a function >>>>>>>> for exporting from the library. The problem with this is that >>>>>>>> JNIEXPORT is part of the JNI interface, and is supposed to be >>>>>>>> used when C programs interact with Java. And, when doing this, >>>>>>>> the function should be fully decorated like this: "JNIEXPORT >>>>>>>> foo JNICALL". >>>>>>> >>>>>>> I've seen a lot of the emails on this issue and I don't fully >>>>>>> understand what has been going wrong. But the intent is >>>>>>> obviously the JNIEXPORT represents what is needed to export this >>>>>>> function for use by JNI, while JNICALL defines the calling >>>>>>> convention. I agree there may be some mistmatch when functions >>>>>>> are actually not intended for general export outside the JDK but >>>>>>> are only for internal JDK use. >>>>>>> >>>>>>>> We do have many such JNI exports in our native libraries, but >>>>>>>> we also have a lot of other, non-JNI exports, where one native >>>>>>>> library just provides an interface to other libraries. In these >>>>>>>> cases, we have still used JNIEXPORT for the functionality of >>>>>>>> getting the function exported, but we have not been consistent >>>>>>>> in our use of JNICALL. This has caused us way too much trouble >>>>>>>> for something that should Just Work. >>>>>>> >>>>>>> Are you suggesting that the interface between different >>>>>>> libraries in the JDK should not be a JNI interface? Is this >>>>>>> because you think the functions in these libraries are only for >>>>>>> JDK internal use or ... ?? >>>>>>> >>>>>>>> I therefore propose that we define "JDK_EXPORT", with the same >>>>>>>> behavior as JNIEXPORT (that is, flagging the function for >>>>>>>> external visibility in the resulting native library), but which >>>>>>>> is *not* supposed to be exported to Java code using JNI, nor >>>>>>>> supposed to be decorated with >>>>>>> >>>>>>> Just a clarification there. JNI functions are not exported to >>>>>>> Java code, they are exported to native code. Java code can >>>>>>> declare native methods and those native methods must be written >>>>>>> as JNI functions, but that's not what we are discussing. >>>>>>> Libraries expose a JNI interface (a set of functions in the >>>>>>> library) that can be called by application native code, using JNI. >>>>>> We're apparently looking at "JNI" and "exporting" from two >>>>>> opposite sides here. :-) Just to make everything clear: If I have >>>>>> a Java class >>>>>> class MyClass { >>>>>> ?? public static void native myNativeFunc(); >>>>>> } >>>>>> >>>>>> then I have one half of the JNI function, the Java half. This >>>>>> must be matched by a corresponding implementation in native, like >>>>>> this: >>>>>> JNIEXPORT void JNICALL >>>>>> Java_MyClass_myNativeFunc(void) { >>>>>> // ... do stuff >>>>>> } >>>>>> >>>>>> And this is the native half of the JNI function. Right? Let's >>>>>> leave aside which side is "exporting" to the other for now. :-) >>>>>> >>>>>> This way of setting up native functions that can be called from >>>>>> Java is what I refer to as JNI. And when you declare a native JNI >>>>>> function, you *must* use both JNIEXPORT and JNICALL. Alright? >>>>>> >>>>>> We do have a lot of those functions in our native libraries. And >>>>>> they are correct just the way they are. >>>>> >>>>> Yep all well and good. A function declared native in Java must >>>>> have an implementation as you describe. But not all native >>>>> functions exist to provide the native-half of a Java native function! >>>>> >>>>>> However, we also have *other* native functions, that are >>>>>> exported, not as JNI functions that should be called from Java, >>>>>> but as normal native library functions that should be called by >>>>>> other native code. Okay so far? And *those* functions have been >>>>>> problematic in how we decorate >>>>> >>>>> But there are again two cases. Those functions exported from a >>>>> library that are expected to be called from external code using >>>>> the JNI interface mechanism - such as all the JNI functions and >>>>> JVM TI functions we export from the JVM - and those "exported" for >>>>> access between libraries within the JDK (such as all the JVM_* >>>>> functions in libjvm). >>>>> >>>>> I think it is only the second group that should be addressed by >>>>> your JDK_EXPORT proposal - though I'm not completely clear exactly >>>>> how to identify them. >>>> Alright, now I think we're on the same page again. :) >>>> >>>> Yes, this is what I'm saying. I'm not proposing to modify public APIs. >>>> >>>> You identify external APIs by the fact that they are documented. >>>> And if you are unsure, you ask Jon. ;-) >>>> >>>>> >>>>>> them. My proposal is that we *refrain* from using JNIEXPORT for >>>>>> those functions, and instead use JDK_EXPORT as name for the macro >>>>>> that decorates them as exported. That way, we can clearly see >>>>>> that a function like this: >>>>>> >>>>>> JDK_EXPORT void >>>>>> JLI_ReadEnv(char* env); >>>>>> >>>>>> is correctly declared, and will be exported to other native >>>>>> libraries, but not to Java. >>>>> >>>>> The issue is not whether it is "exported to Java"** but whether it >>>>> is exported using the JNI mechanism such that other native code >>>>> calls it using the JNI mechanism. >>>>> >>>>> ** There is no way to write a native method declaration in Java >>>>> such that it would be linked to the JLI_ReadEnv function. The >>>>> naming is all wrong, as is the signature. >>>>> >>>>>> Just to clarify, this has nothing to do with if this is a >>>>>> officially supported API or not. In general though, I assume that >>>>>> most (if not all?) of our exported functions (apart from the >>>>>> JNI_* stuff) is supposed to be consumed by other libraries in the >>>>>> JDK, and is not a public API. >>>>> >>>>> I think it varies library by library. You may need native >>>>> application code that can call directly into native JDK libraries. >>>>> JLI is the Java Launcher Interface - I think it was introduced to >>>>> make it easier for other launchers to be created. Native agents >>>>> may need access to libmanagement or libjdwp functions. Native >>>>> graphics code may need access to the JDK graphics library. Some of >>>>> these accesses may be unsupported and undocumented, but I don't >>>>> think you can just cut them all off. >>>> If they are unsupported and undocumented, I sure as h*ll can cut >>>> them all off! :-) >>>> >>>> Besides, and let me re-iterate this, the only change that will >>>> happen by removing JNICALL, is on Windows 32. No other platform >>>> will be affected. There is no official support for Windows 32 >>>> anymore. There's some low-effort community work done on keeping >>>> Windows 32 alive, but it's not backed by any major player. Right >>>> now, they are taking a lot of hits *due to the fact* that we have >>>> not sorted this out, and waste a lot of their precious effort >>>> trying to fix this piecemeal. If we do fix things according to this >>>> proposal, then it's at least clear how things *should* work. If it >>>> turns out that there's some code out there in the wild, running on >>>> Windows 32, that uses an undocumented and unsupported function >>>> call, and it breaks -- well, sucks to be them. They'll just have to >>>> do what everyone does who uses an undocumented interface that >>>> suddenly changes: update their code. >>>> >>>> /Magnus >>>> >>>>> >>>>> David >>>>> >>>>>> >>>>>> /Magnus >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>>> JNICALL. All current instances of JNIEXPORT which is not pure >>>>>>>> JNI native functions should be changed to use JDK_EXPORT instead. >>>>>>>> >>>>>>>> I further propose that this macro should reside in a new file >>>>>>>> "jdk.h", placed in the new directory >>>>>>>> src/java.base/share/native/include/internal. This header file >>>>>>>> path will automatically be provided to all native libraries, >>>>>>>> but not copied to the JDK being built. (The existence of a >>>>>>>> "include/internal" directory with this behavior has been >>>>>>>> discussed before. There are more files that ought to be moved >>>>>>>> there, if/when it is created.) I believe in many cases the >>>>>>>> #include "jni.h" can be just modified to #include "#jdk.h", >>>>>>>> since most native code will not require "jni.h" unless actually >>>>>>>> doing JNI calls -- most have included this file to get the >>>>>>>> JNIEXPORT macro, which would explain the pervasive use of >>>>>>>> #include "jni.h" in our code base. >>>>>>> >>>>>>> jni.h also defines all of the types used by the JNI. Those types >>>>>>> are pervsive to the native code used throughout the JDK. >>>>>>> >>>>>>>> Thoughts? >>>>>>> >>>>>>> I think we need to understand the problems on Windows that >>>>>>> prompted all this. Then I think we need to look at exactly how >>>>>>> jni.h and JNIEXPORT etc are being used and understand whether >>>>>>> this is truly an exported interface or not. >>>>>>> >>>>>>> Cheers, >>>>>>> David >>>>>>> >>>>>>>> /Magnus >>>>>>>> >>>>>> >>>> >> From david.holmes at oracle.com Mon Dec 17 03:03:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Dec 2018 13:03:30 +1000 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <61f4a254-be9d-b0b2-3525-8caa247d9556@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> <61f4a254-be9d-b0b2-3525-8caa247d9556@oracle.com> Message-ID: <3de758a3-e990-384a-5267-265bda302cbc@oracle.com> On 17/12/2018 12:49 pm, dean.long at oracle.com wrote: > On 12/16/18 4:06 PM, David Holmes wrote: >> On 15/12/2018 10:59 am, dean.long at oracle.com wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>> >>> This change includes two new regression test that demonstrate the >>> problem, and a fix that allows the tests >>> to pass. >>> >>> The problem happens when the JIT compiler's escape analysis >>> eliminates the allocation of the AccessControlContext object passed >>> to doPrivileged.? The compiler thinks this is safe because it does >>> not see that the object "escapes". >> >> Then surely the compiler's notion of "escapes" needs to be updated! >> > > The compiler can inline the callee method and see that the value doesn't > escape.? This is a valid optimization in cases where the callee method > is known. But it's not a valid optimization in this case, so my comment stands. Is this stack walking something this is guaranteed by the spec to be always valid (and hence the JIT is violating the rules), or is the stack walking code making assumptions about whether it will find the context object in the stack? If we have to hack around this with an annotation I'd rather see a specific annotation that addresses the problematic usecase than a generic "don't inline" one. E.g. @StackVisible or something like that. Cheers, David > > dl > >> David >> ----- >> >> ? However, getContext needs to be able to find >>> the object using a stack walk, so we need a way to tell the compiler >>> that it does indeed escape.? To do this we pass the value to a native >>> method that does nothing. >>> >>> Microbenchmark results: >>> >>> jdk12-b18: >>> >>> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >>> >>> >>> jdk12-b19: >>> >>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >>> >>> this fix: >>> >>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >>> >>> >>> dl > From david.holmes at oracle.com Mon Dec 17 03:07:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Dec 2018 13:07:48 +1000 Subject: Proposal: Use new JDK_EXPORT decorator instead of JNIEXPORT In-Reply-To: References: <43fe99fa-ed6f-69d5-c9bf-e56cb0321b69@oracle.com> <80b9c17e-f351-34e5-8853-b8b8ccc42727@oracle.com> <8e376189-6857-4db2-8f33-fc02ffbb2b71@oracle.com> <236420f2-cd56-75a7-db52-59a7ef9f9be3@oracle.com> Message-ID: <3d7cd0eb-9694-840f-1e1e-4abc71877461@oracle.com> On 17/12/2018 12:51 pm, dean.long at oracle.com wrote: > If we are adding JDK_EXPORT, does it make sense to add JDK_CALL as well? It should never be necessary. JNICALL is only needed to ensure caller and callee use the same calling convention. For JDK_EXPORT the caller and callee are part of the same thing - the JDK - and will always use the same (platform default) calling convention. David > dl > > On 12/16/18 4:57 PM, David Holmes wrote: >> Hi Magnus, >> >> Thanks for explaining how addition of JNIEXPORT may have started this >> problem. >> >> One follow up: >> >>>> This will also need a CSR request due to the change in linking >>>> behaviour. >>> I'm not sure what you mean by this..? My entire intention here is >>> that we should make no changes to documented interfaces; this is >>> strictly an internal change, so no CSR should be needed. Also, I >>> don't understand what you mean by "linking behavior"? >> >> A CSR request is also required for behavioural changes - which this >> is. Someone linking an "internal only" function today may get an error >> if JNICALL is removed tomorrow. This may be acceptable but that is >> what the CSR request establishes. >> >> Thanks, >> David >> >> >> On 13/12/2018 8:37 pm, Magnus Ihse Bursie wrote: >>> On 2018-12-12 13:17, David Holmes wrote: >>>> Okay I went away and did some homework ... >>>> >>>> Let me back up a bit and see if I have the evolution of this correctly. >>>> >>>> The native implementation of Java methods should be declared and >>>> defined with JNIEXPORT and JNICALL. >>>> >>>> JNIEXPORT controls the export visibility so they can be linked. >>>> >>>> JNICALL controls the calling convention and is needed so that the >>>> JVM's calling convention matches the native code. [This part was >>>> unclear to me.] >>> Yes. And JNICALL is empty on all platforms except Windows 32, that's >>> why we're only seeing issues about mismatch there. >>>> >>>> Other native methods exported from the same (or different) native >>>> libraries may also be decorated with JNIEXPORT and JNICALL. But in >>>> places there is a mismatch between the declaration in the header and >>>> the definition leading to errors. >>> Yes; this mismatch has typically happened when the linking has not >>> been done by simply including the relevant header file, but either >>> duplicating the definition, or looking up the symbol dynamically. >>> Otherwise things should basically work out. >>>> >>>> There are two main types of such native functions: >>>> >>>> a) those publicly defined in the various native APIs: JNI itself >>>> (jni.h), JVM TI (jvmti.h), AWT (jawt.h) ... >>>> >>>> b) those intended only for use by other native code within the JDK >>>> libraries (JLI_* - though I note Alan's comment re javafxpackager, >>>> others??) >>>> >>>> and a possible third type are callback functions like the JPLISAgent >>>> event handlers (e.g. eventHandlerVMInit). >>> >>> I'm not sure I understand what the third type is, but if it is a >>> publically defined API (which, at a first look, the JPLISAgent API >>> seems to be), then it's part of catagory a, otherwise it's part of >>> category b. >>> >>> I must also state that my initial proposal didn't separate these two >>> cases. I had in mind only the b cases -- I have no intention of >>> changing public specifications, but I did not express this in my >>> proposal. That might have been one source of confusion. I apologize >>> for this omission. >>>> >>>> There is a view that no native method that isn't the native-half of >>>> a Java method should use JNICALL. [Okay I can see how that makes >>>> sense - regardless of the actual calling convention used marking >>>> such methods as "must use the same calling convention as the VM >>>> native method call" is somewhat meaningless. Yet obviously the >>>> public native APIs do specify JNICALL so this is not a hard and fast >>>> rule. Further the callbacks must also follow a convention known to >>>> the VM doing the calling!] >>> Yes, or rather the rule is "only native half Java methods should use >>> JNICALL, and also all public APIs where so is specified". >>> >>>> >>>> Where we have (introduced?) a discrepancy between the definition and >>>> declaration the approach has been (because of the previous view) to >>>> remove JNICALL. [This should only relate to methods of kind (b) above.] >>> Actually, it's not so much as we have *removed* JNICALL, as that we >>> have *introduced* JNIEXPORT. When I removed the map files, I also >>> removed the .def files and /export command lines for Windows. As a >>> result, I needed to add JNIEXPORT to a lot of places. Initially, I >>> followed the rule of adding JNICALL to those calls -- but I can >>> surely have missed a couple of places, since things will work fine >>> anyway, as long as caller and callee agree on the calling convention; >>> and a mismatch can only happen on Windows 32, which we do not build >>> and test. So there is a risk that even with the initial patch, not >>> all newly added JNIEXPORTs had JNICALL. >>> >>> Then, it turned out, that a lot of this code did not compile with >>> Windows 32. With no deeper analysis of the flaw, the blame was put on >>> the absence or presence of JNICALL, and a series of patches were made >>> where JNICALL was more or less arbitrarily added or removed, until >>> things "worked". This should have been a warning sign, and I was >>> increasingly uneasy about all these changes, but I hadn't spent >>> enough time to realize what the core issue was or how to resolve it >>> properly. >>> >>>> >>>> That leaves those methods with JNIEXPORT only. >>>> >>>> That seems wrong to you because they are not "JNI methods", so you >>>> want to replace with JDK_EXPORT to make it clear. [Ok - this seems >>>> reasonable.] >>> Yes. And given the fact that we have a bunch of "non-JNI methods" >>> that use the JNIEXPORT...JNICALL pattern, another way to interpret >>> the JDK_EXPORT semantics is that this function is exported for usage >>> *in the JDK*, but not for public consumption. >>>> >>>> If JNICALL is removed from those native methods and they are >>>> currently linked by external applications, those applications may >>>> stop working. But this only affects win32 (as its the only platform >>>> where JNICALL is different to the default C/C++ calling convention?) >>>> so your position is this is acceptable breakage - and would only >>>> affect unsupported/undocumented APIs. >>> Yes. In fact, it's possible that at least some of the breakage that >>> occurred was due to new *addition* of JNICALL, which was not present >>> before. We might have had something like (I'm making this specific >>> example up) a function "void ZIP_OpenFile(char* name)" with no >>> decoration at all, and a "/export:ZIP_OpenFile" on the command line, >>> and a ZIP_OpenFile entry in the unix map file. And I converted this >>> to "JNIEXPORT void JNICALL ZIP_OpenFile(char* name)", which de facto >>> -- although I didn't fully realize this at the time, changed the >>> calling convention and name decoration on Windows 32. When something >>> broke, perhaps because the user of ZIP_OpenFile did not include the >>> proper header file with the JNICALL modifier, the solution was to >>> remove the JNICALL part. >>> >>> And of all the bug reports I've seen on this, the issues has been >>> internal linking only, i.e. problems building the JDK, not complaints >>> that external tools tried to use internal interfaces and failed. So >>> yes, my position is if this should break things, tough shit. That, of >>> courses, requires that we do not change public APIs, so we need to be >>> careful not to mess with those. >>>> >>>> Does that sum it up? >>> Yep, I think so. >>>> >>>> We still need to be sure that we're only changing functions of type >>>> (b) above. >>> Yes, definitely. >>>> >>>> And I note that this has been driven by the choice to remove JNICALL >>>> where there was a discrepancy - that leads to the visibility of the >>>> two kinds of methods. If it had been chosen to add JNICALL where >>>> missing instead, then we may not have been having this conversation. >>> Actually, as I said, this has more been the result of a lot of added >>> JNICALL where previously there was none. >>> >>> An alternative course of action is the make sure that *all* calls use >>> the JNIEXPORT...JNICALL pattern, even type b ones, and that we fix >>> all parts of code to actually accept the decorated names on Windows >>> 32. This will lead to a lot more code changes, like the fix for >>> JDK-8214122 (JDWP is broken on 32 bit Windows: transport library >>> missing onLoad entry). And all this special case handling will be >>> needed only on Windows 32, which is a platform we do not want to >>> spend to much time or effort on. And finally, I think doing so would >>> make us miss out on an opportunity to make the semantics clearer. >>>> >>>> This will also need a CSR request due to the change in linking >>>> behaviour. >>> I'm not sure what you mean by this..? My entire intention here is >>> that we should make no changes to documented interfaces; this is >>> strictly an internal change, so no CSR should be needed. Also, I >>> don't understand what you mean by "linking behavior"? >>> >>> /Magnus >>>> >>>> Cheers, >>>> David >>>> ----- >>>> >>>> On 12/12/2018 9:03 pm, Magnus Ihse Bursie wrote: >>>>> >>>>> >>>>> On 2018-12-11 23:47, David Holmes wrote: >>>>>> On 12/12/2018 12:34 am, Magnus Ihse Bursie wrote: >>>>>>> >>>>>>> >>>>>>> On 2018-12-11 00:23, David Holmes wrote: >>>>>>>> Hi Magnus, >>>>>>>> >>>>>>>> On 10/12/2018 11:19 pm, Magnus Ihse Bursie wrote: >>>>>>>>> I propose that we introduce a new define, available to all JDK >>>>>>>>> native files (Hotspot included), called JDK_EXPORT. The >>>>>>>>> behavior of this symbol will be very similar (as of now, in >>>>>>>>> fact identical) to JNIEXPORT; however, the semantics will not. >>>>>>>>> >>>>>>>>> Currently, we "mis-use" the JNIEXPORT define to mark a function >>>>>>>>> for exporting from the library. The problem with this is that >>>>>>>>> JNIEXPORT is part of the JNI interface, and is supposed to be >>>>>>>>> used when C programs interact with Java. And, when doing this, >>>>>>>>> the function should be fully decorated like this: "JNIEXPORT >>>>>>>>> foo JNICALL". >>>>>>>> >>>>>>>> I've seen a lot of the emails on this issue and I don't fully >>>>>>>> understand what has been going wrong. But the intent is >>>>>>>> obviously the JNIEXPORT represents what is needed to export this >>>>>>>> function for use by JNI, while JNICALL defines the calling >>>>>>>> convention. I agree there may be some mistmatch when functions >>>>>>>> are actually not intended for general export outside the JDK but >>>>>>>> are only for internal JDK use. >>>>>>>> >>>>>>>>> We do have many such JNI exports in our native libraries, but >>>>>>>>> we also have a lot of other, non-JNI exports, where one native >>>>>>>>> library just provides an interface to other libraries. In these >>>>>>>>> cases, we have still used JNIEXPORT for the functionality of >>>>>>>>> getting the function exported, but we have not been consistent >>>>>>>>> in our use of JNICALL. This has caused us way too much trouble >>>>>>>>> for something that should Just Work. >>>>>>>> >>>>>>>> Are you suggesting that the interface between different >>>>>>>> libraries in the JDK should not be a JNI interface? Is this >>>>>>>> because you think the functions in these libraries are only for >>>>>>>> JDK internal use or ... ?? >>>>>>>> >>>>>>>>> I therefore propose that we define "JDK_EXPORT", with the same >>>>>>>>> behavior as JNIEXPORT (that is, flagging the function for >>>>>>>>> external visibility in the resulting native library), but which >>>>>>>>> is *not* supposed to be exported to Java code using JNI, nor >>>>>>>>> supposed to be decorated with >>>>>>>> >>>>>>>> Just a clarification there. JNI functions are not exported to >>>>>>>> Java code, they are exported to native code. Java code can >>>>>>>> declare native methods and those native methods must be written >>>>>>>> as JNI functions, but that's not what we are discussing. >>>>>>>> Libraries expose a JNI interface (a set of functions in the >>>>>>>> library) that can be called by application native code, using JNI. >>>>>>> We're apparently looking at "JNI" and "exporting" from two >>>>>>> opposite sides here. :-) Just to make everything clear: If I have >>>>>>> a Java class >>>>>>> class MyClass { >>>>>>> ?? public static void native myNativeFunc(); >>>>>>> } >>>>>>> >>>>>>> then I have one half of the JNI function, the Java half. This >>>>>>> must be matched by a corresponding implementation in native, like >>>>>>> this: >>>>>>> JNIEXPORT void JNICALL >>>>>>> Java_MyClass_myNativeFunc(void) { >>>>>>> // ... do stuff >>>>>>> } >>>>>>> >>>>>>> And this is the native half of the JNI function. Right? Let's >>>>>>> leave aside which side is "exporting" to the other for now. :-) >>>>>>> >>>>>>> This way of setting up native functions that can be called from >>>>>>> Java is what I refer to as JNI. And when you declare a native JNI >>>>>>> function, you *must* use both JNIEXPORT and JNICALL. Alright? >>>>>>> >>>>>>> We do have a lot of those functions in our native libraries. And >>>>>>> they are correct just the way they are. >>>>>> >>>>>> Yep all well and good. A function declared native in Java must >>>>>> have an implementation as you describe. But not all native >>>>>> functions exist to provide the native-half of a Java native function! >>>>>> >>>>>>> However, we also have *other* native functions, that are >>>>>>> exported, not as JNI functions that should be called from Java, >>>>>>> but as normal native library functions that should be called by >>>>>>> other native code. Okay so far? And *those* functions have been >>>>>>> problematic in how we decorate >>>>>> >>>>>> But there are again two cases. Those functions exported from a >>>>>> library that are expected to be called from external code using >>>>>> the JNI interface mechanism - such as all the JNI functions and >>>>>> JVM TI functions we export from the JVM - and those "exported" for >>>>>> access between libraries within the JDK (such as all the JVM_* >>>>>> functions in libjvm). >>>>>> >>>>>> I think it is only the second group that should be addressed by >>>>>> your JDK_EXPORT proposal - though I'm not completely clear exactly >>>>>> how to identify them. >>>>> Alright, now I think we're on the same page again. :) >>>>> >>>>> Yes, this is what I'm saying. I'm not proposing to modify public APIs. >>>>> >>>>> You identify external APIs by the fact that they are documented. >>>>> And if you are unsure, you ask Jon. ;-) >>>>> >>>>>> >>>>>>> them. My proposal is that we *refrain* from using JNIEXPORT for >>>>>>> those functions, and instead use JDK_EXPORT as name for the macro >>>>>>> that decorates them as exported. That way, we can clearly see >>>>>>> that a function like this: >>>>>>> >>>>>>> JDK_EXPORT void >>>>>>> JLI_ReadEnv(char* env); >>>>>>> >>>>>>> is correctly declared, and will be exported to other native >>>>>>> libraries, but not to Java. >>>>>> >>>>>> The issue is not whether it is "exported to Java"** but whether it >>>>>> is exported using the JNI mechanism such that other native code >>>>>> calls it using the JNI mechanism. >>>>>> >>>>>> ** There is no way to write a native method declaration in Java >>>>>> such that it would be linked to the JLI_ReadEnv function. The >>>>>> naming is all wrong, as is the signature. >>>>>> >>>>>>> Just to clarify, this has nothing to do with if this is a >>>>>>> officially supported API or not. In general though, I assume that >>>>>>> most (if not all?) of our exported functions (apart from the >>>>>>> JNI_* stuff) is supposed to be consumed by other libraries in the >>>>>>> JDK, and is not a public API. >>>>>> >>>>>> I think it varies library by library. You may need native >>>>>> application code that can call directly into native JDK libraries. >>>>>> JLI is the Java Launcher Interface - I think it was introduced to >>>>>> make it easier for other launchers to be created. Native agents >>>>>> may need access to libmanagement or libjdwp functions. Native >>>>>> graphics code may need access to the JDK graphics library. Some of >>>>>> these accesses may be unsupported and undocumented, but I don't >>>>>> think you can just cut them all off. >>>>> If they are unsupported and undocumented, I sure as h*ll can cut >>>>> them all off! :-) >>>>> >>>>> Besides, and let me re-iterate this, the only change that will >>>>> happen by removing JNICALL, is on Windows 32. No other platform >>>>> will be affected. There is no official support for Windows 32 >>>>> anymore. There's some low-effort community work done on keeping >>>>> Windows 32 alive, but it's not backed by any major player. Right >>>>> now, they are taking a lot of hits *due to the fact* that we have >>>>> not sorted this out, and waste a lot of their precious effort >>>>> trying to fix this piecemeal. If we do fix things according to this >>>>> proposal, then it's at least clear how things *should* work. If it >>>>> turns out that there's some code out there in the wild, running on >>>>> Windows 32, that uses an undocumented and unsupported function >>>>> call, and it breaks -- well, sucks to be them. They'll just have to >>>>> do what everyone does who uses an undocumented interface that >>>>> suddenly changes: update their code. >>>>> >>>>> /Magnus >>>>> >>>>>> >>>>>> David >>>>>> >>>>>>> >>>>>>> /Magnus >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> JNICALL. All current instances of JNIEXPORT which is not pure >>>>>>>>> JNI native functions should be changed to use JDK_EXPORT instead. >>>>>>>>> >>>>>>>>> I further propose that this macro should reside in a new file >>>>>>>>> "jdk.h", placed in the new directory >>>>>>>>> src/java.base/share/native/include/internal. This header file >>>>>>>>> path will automatically be provided to all native libraries, >>>>>>>>> but not copied to the JDK being built. (The existence of a >>>>>>>>> "include/internal" directory with this behavior has been >>>>>>>>> discussed before. There are more files that ought to be moved >>>>>>>>> there, if/when it is created.) I believe in many cases the >>>>>>>>> #include "jni.h" can be just modified to #include "#jdk.h", >>>>>>>>> since most native code will not require "jni.h" unless actually >>>>>>>>> doing JNI calls -- most have included this file to get the >>>>>>>>> JNIEXPORT macro, which would explain the pervasive use of >>>>>>>>> #include "jni.h" in our code base. >>>>>>>> >>>>>>>> jni.h also defines all of the types used by the JNI. Those types >>>>>>>> are pervsive to the native code used throughout the JDK. >>>>>>>> >>>>>>>>> Thoughts? >>>>>>>> >>>>>>>> I think we need to understand the problems on Windows that >>>>>>>> prompted all this. Then I think we need to look at exactly how >>>>>>>> jni.h and JNIEXPORT etc are being used and understand whether >>>>>>>> this is truly an exported interface or not. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> David >>>>>>>> >>>>>>>>> /Magnus >>>>>>>>> >>>>>>> >>>>> >>> > From dean.long at oracle.com Mon Dec 17 03:39:31 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 19:39:31 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <3de758a3-e990-384a-5267-265bda302cbc@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> <61f4a254-be9d-b0b2-3525-8caa247d9556@oracle.com> <3de758a3-e990-384a-5267-265bda302cbc@oracle.com> Message-ID: On 12/16/18 7:03 PM, David Holmes wrote: > On 17/12/2018 12:49 pm, dean.long at oracle.com wrote: >> On 12/16/18 4:06 PM, David Holmes wrote: >>> On 15/12/2018 10:59 am, dean.long at oracle.com wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>>> >>>> This change includes two new regression test that demonstrate the >>>> problem, and a fix that allows the tests >>>> to pass. >>>> >>>> The problem happens when the JIT compiler's escape analysis >>>> eliminates the allocation of the AccessControlContext object passed >>>> to doPrivileged.? The compiler thinks this is safe because it does >>>> not see that the object "escapes". >>> >>> Then surely the compiler's notion of "escapes" needs to be updated! >>> >> >> The compiler can inline the callee method and see that the value >> doesn't escape.? This is a valid optimization in cases where the >> callee method is known. > > But it's not a valid optimization in this case, so my comment stands. > > Is this stack walking something this is guaranteed by the spec to be > always valid (and hence the JIT is violating the rules), or is the > stack walking code making assumptions about whether it will find the > context object in the stack? > The stack walking is in the VM and is an internal implementation detail, not part of the AccessController API spec.? A different thread running normal Java code would never be able to see a non-escaping value.? The stack walking code does need to find the context object in the stack.? Non-escaping objects won't show up in the stack. > If we have to hack around this with an annotation I'd rather see a > specific annotation that addresses the problematic usecase than a > generic "don't inline" one. E.g. @StackVisible or something like that. > That sounds like a good idea for 13, but would require changes to both C2 and Graal, and it seems a little risky compared to using existing mechanisms. dl > Cheers, > David > >> >> dl >> >>> David >>> ----- >>> >>> ? However, getContext needs to be able to find >>>> the object using a stack walk, so we need a way to tell the >>>> compiler that it does indeed escape. To do this we pass the value >>>> to a native method that does nothing. >>>> >>>> Microbenchmark results: >>>> >>>> jdk12-b18: >>>> >>>> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >>>> >>>> >>>> jdk12-b19: >>>> >>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >>>> >>>> this fix: >>>> >>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >>>> >>>> >>>> dl >> From dean.long at oracle.com Mon Dec 17 04:45:02 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 20:45:02 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> Message-ID: <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> Unfortunately, I don't think @DontInline on an empty method is sufficient here.? If other code is relying on @DontInline for the same purpose then we might need to reexamine that code.? My understanding from discussing with other compiler engineers is that using a native method is the safest technique that the compilers can't see through.? The problem with @DontInline is that C2 looks at the bytecodes of the target method, even if it isn't inlined (see BCEscapeAnalyzer and the EstimateArgEscape flag). There may be a way to make it work, but that would require more investigation, and I'm not sure the benefit outweighs the risk. dl On 12/15/18 6:48 AM, Claes Redestad wrote: > Hi Dean, > > to avoid escape analysis-eliminated allocations in the past @DontInline > has been sufficient. This means a simpler patch (no changes to native > code needed - added assertions notwithstanding) and passes your tests > with C2 (it'd concern me if Graal's EA sees through this trick, as it > might break some existing places where DontInline is used to this > effect): > > ??? /** > ???? * The value needs to be physically located in the frame, so that it > ???? * can be found by a stack walk. > ???? */ > ??? @Hidden > ??? @DontInline > ??? private static void ensureMaterializedForStackWalk(Object o) {} > > Thanks! > > /Claes > > On 2018-12-15 01:59, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8214583 >> http://cr.openjdk.java.net/~dlong/8214583/webrev >> >> This change includes two new regression test that demonstrate the >> problem, and a fix that allows the tests >> to pass. >> >> The problem happens when the JIT compiler's escape analysis >> eliminates the allocation of the AccessControlContext object passed >> to doPrivileged.? The compiler thinks this is safe because it does >> not see that the object "escapes".? However, getContext needs to be >> able to find the object using a stack walk, so we need a way to tell >> the compiler that it does indeed escape.? To do this we pass the >> value to a native method that does nothing. >> >> Microbenchmark results: >> >> jdk12-b18: >> >> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >> >> >> jdk12-b19: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >> >> this fix: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >> >> >> dl From dean.long at oracle.com Mon Dec 17 04:47:59 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 20:47:59 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> <61f4a254-be9d-b0b2-3525-8caa247d9556@oracle.com> <3de758a3-e990-384a-5267-265bda302cbc@oracle.com> Message-ID: On 12/16/18 7:39 PM, dean.long at oracle.com wrote: > On 12/16/18 7:03 PM, David Holmes wrote: >> On 17/12/2018 12:49 pm, dean.long at oracle.com wrote: >>> On 12/16/18 4:06 PM, David Holmes wrote: >>>> On 15/12/2018 10:59 am, dean.long at oracle.com wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>>>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>>>> >>>>> This change includes two new regression test that demonstrate the >>>>> problem, and a fix that allows the tests >>>>> to pass. >>>>> >>>>> The problem happens when the JIT compiler's escape analysis >>>>> eliminates the allocation of the AccessControlContext object >>>>> passed to doPrivileged.? The compiler thinks this is safe because >>>>> it does not see that the object "escapes". >>>> >>>> Then surely the compiler's notion of "escapes" needs to be updated! >>>> >>> >>> The compiler can inline the callee method and see that the value >>> doesn't escape.? This is a valid optimization in cases where the >>> callee method is known. >> >> But it's not a valid optimization in this case, so my comment stands. >> >> Is this stack walking something this is guaranteed by the spec to be >> always valid (and hence the JIT is violating the rules), or is the >> stack walking code making assumptions about whether it will find the >> context object in the stack? >> > > The stack walking is in the VM and is an internal implementation > detail, not part of the AccessController API spec.? A different thread > running normal Java code would never be able to see a non-escaping > value.? The stack walking code does need to find the context object in > the stack.? Non-escaping objects won't show up in the stack. > >> If we have to hack around this with an annotation I'd rather see a >> specific annotation that addresses the problematic usecase than a >> generic "don't inline" one. E.g. @StackVisible or something like that. >> > > That sounds like a good idea for 13, but would require changes to both > C2 and Graal, and it seems a little risky compared to using existing > mechanisms. > I forgot to address this in my last reply, but I'm not suggesting a @DontInline annotation.? That was Claes.? My fixes uses a native method. dl > dl > >> Cheers, >> David >> >>> >>> dl >>> >>>> David >>>> ----- >>>> >>>> ? However, getContext needs to be able to find >>>>> the object using a stack walk, so we need a way to tell the >>>>> compiler that it does indeed escape. To do this we pass the value >>>>> to a native method that does nothing. >>>>> >>>>> Microbenchmark results: >>>>> >>>>> jdk12-b18: >>>>> >>>>> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >>>>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >>>>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >>>>> >>>>> >>>>> jdk12-b19: >>>>> >>>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >>>>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >>>>> >>>>> this fix: >>>>> >>>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >>>>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >>>>> >>>>> >>>>> dl >>> > From dean.long at oracle.com Mon Dec 17 04:59:07 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 20:59:07 -0800 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> Message-ID: <98be78d0-d49d-cf4a-fda8-34cd33d5c813@oracle.com> Thanks.? Would you like to add comments as suggested by Boris?? I think it's a good idea. dl On 12/16/18 6:00 PM, Nick Gasson (Arm Technology China) wrote: > > Hi Dean, > > Sorry for the delay. This is submitted on behalf of ARM Ltd. > > Thanks, > > Nick > > *From:* dean.long at oracle.com > *Sent:* 13 December 2018 12:01 > *To:* Nick Gasson (Arm Technology China) ; > hotspot-dev at openjdk.java.net; Boris Ulasevich > > *Cc:* hotspot-compiler-dev at openjdk.java.net compiler > ; nd ; > aarch32-port-dev at openjdk.java.net > *Subject:* Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java > fails on ARM32 > > Nick, can you please confirm that you are covered under the > > ? ARM Ltd. - OpenJDK, MySQL > > > OCA agreement?? I'm guessing Arm Technology China is under ARM Ltd., > judging > by your email domain, but please confirm. > > dl > > On 12/11/18 5:30 PM, dean.long at oracle.com > wrote: > > I can help you commit it, but first I think another review would > be good. > > dl > > On 12/11/18 5:15 PM, Nick Gasson (Arm Technology China) wrote: > > Hi Nick.? This change looks good to me. > > Thanks Dean. Could you help me to commit this patch if it > doesn't need other reviews? > > Nick > > > > -----Original Message----- > From: dean.long at oracle.com > > Sent: 12 December 2018 06:05 > To: Nick Gasson (Arm Technology China) > ; hotspot- > dev at openjdk.java.net ; Boris > Ulasevich > > Cc: nd ; > hotspot-compiler-dev at openjdk.java.net > compiler > > ; aarch32-port- > dev at openjdk.java.net > Subject: Re: RFR: 8214512: Jtreg test > compiler/c2/Test8062950.java fails on > ARM32 > > Hi Nick.? This change looks good to me. > > dl > > On 12/2/18 10:06 PM, Nick Gasson wrote: > > Hi, > > Could someone please help me review this fix for a > test failure on ARM32: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 > Webrev: > http://cr.openjdk.java.net/~njian/8214512/webrev.0/ > > This fixes two assertion failures related to biased > locking when C2's > bias inlining feature is disabled > (-XX:-OptoBiasInlining): > > MacroAssembler::atomic_cas_bool takes three register > arguments plus a > temporary register for use in the CAS loop. If the > caller passes > `noreg' as this temporary register (which happens in the > !OptoBiasInlining call path from > MacroAssembler::fast_lock) then > MacroAssembler::atomic_cas_bool will default to using > LR as a > temporary. But it's possible with C2 that LR is one of > the other three > arguments which then triggers an > assert_different_registers assertion > failure. Fix this by supplying an additional scratch > register to > MacroAssembler::fast_lock if !OptoBiasInlining. > > In the !OptoBiasInlining case > MacroAssembler::fast_lock calls > MacroAssembler::biased_locking_enter to handle the > case where the mark > word contains the biased lock pattern which fast_lock > wouldn't > otherwise see if OptoBiasInlining was true. However in > the case that > we fail to acquire the biased lock this falls through > to label > `failed' and runs the normal fast_lock code that > implicitly assumes > the mark word does not have the bias pattern. This > means we can end up > with a BasicLock where _displaced_header contains the > biased lock > pattern which is an illegal state and subsequently > triggers an > assertion failure in ObjectSynchronizer::fast_exit. > > The right thing to do here is branch to `done' > whenever the bias lock > pattern is present and let the runtime handle the > failure case.? Also > edited the comment describing > MacroAssembler::biased_locking_enter to > more accurately describe what it does (the comment > about `slow_case' > being optional is wrong). > > Jtreg test compiler/c2/Test8062950.java now passes on > ARM32 with this > patch. > > Thanks, > Nick > From claes.redestad at oracle.com Mon Dec 17 05:52:22 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 17 Dec 2018 06:52:22 +0100 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> Message-ID: <790736fc-836c-d116-28c2-ffdc02c1e2ca@oracle.com> Fair enough, I wasn't aware EA was looking beyond the inlined code like this (which means it can't be "dead" or the JIT might see through the trick at some point). I've skimmed the existing usages in the JDK and can't find anything that seems to be dependent on DontInline defeating EA - maybe there was in the past, or I simply misremembered. Either way, it might be nice to have a more explicit facility for this in a future release, as David suggested. Say an @Escaping local variable/parameter annotation. /Claes On 2018-12-17 05:45, dean.long at oracle.com wrote: > Unfortunately, I don't think @DontInline on an empty method is sufficient > here.? If other code is relying on @DontInline for the same purpose then > we might need to reexamine that code.? My understanding from discussing > with other compiler engineers is that using a native method is the safest > technique that the compilers can't see through.? The problem with > @DontInline is that C2 looks at the bytecodes of the target method, even > if it isn't inlined (see BCEscapeAnalyzer and the EstimateArgEscape flag). > There may be a way to make it work, but that would require more > investigation, and I'm not sure the benefit outweighs the risk. > > dl > > On 12/15/18 6:48 AM, Claes Redestad wrote: >> Hi Dean, >> >> to avoid escape analysis-eliminated allocations in the past @DontInline >> has been sufficient. This means a simpler patch (no changes to native >> code needed - added assertions notwithstanding) and passes your tests >> with C2 (it'd concern me if Graal's EA sees through this trick, as it >> might break some existing places where DontInline is used to this >> effect): >> >> ??? /** >> ???? * The value needs to be physically located in the frame, so that it >> ???? * can be found by a stack walk. >> ???? */ >> ??? @Hidden >> ??? @DontInline >> ??? private static void ensureMaterializedForStackWalk(Object o) {} >> >> Thanks! >> >> /Claes >> >> On 2018-12-15 01:59, dean.long at oracle.com wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>> >>> This change includes two new regression test that demonstrate the >>> problem, and a fix that allows the tests >>> to pass. >>> >>> The problem happens when the JIT compiler's escape analysis >>> eliminates the allocation of the AccessControlContext object passed >>> to doPrivileged.? The compiler thinks this is safe because it does >>> not see that the object "escapes".? However, getContext needs to be >>> able to find the object using a stack walk, so we need a way to tell >>> the compiler that it does indeed escape.? To do this we pass the >>> value to a native method that does nothing. >>> >>> Microbenchmark results: >>> >>> jdk12-b18: >>> >>> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >>> >>> >>> jdk12-b19: >>> >>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >>> >>> this fix: >>> >>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >>> >>> >>> dl > From dean.long at oracle.com Mon Dec 17 05:54:57 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 21:54:57 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <790736fc-836c-d116-28c2-ffdc02c1e2ca@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> <790736fc-836c-d116-28c2-ffdc02c1e2ca@oracle.com> Message-ID: <8fcf6de6-4cfa-ebbe-84ac-6025924047c4@oracle.com> On 12/16/18 9:52 PM, Claes Redestad wrote: > > Either way, it might be nice to have a more explicit facility for this > in a future release, as David suggested. Say an @Escaping local > variable/parameter annotation. > I agree.? I suggested @Escapes or @StackWalkable in JDK-8214585. @Escaping or @StackVisible sound just as reasonable. dl > /Claes > > On 2018-12-17 05:45, dean.long at oracle.com wrote: >> Unfortunately, I don't think @DontInline on an empty method is >> sufficient >> here.? If other code is relying on @DontInline for the same purpose then >> we might need to reexamine that code.? My understanding from discussing >> with other compiler engineers is that using a native method is the >> safest >> technique that the compilers can't see through.? The problem with >> @DontInline is that C2 looks at the bytecodes of the target method, even >> if it isn't inlined (see BCEscapeAnalyzer and the EstimateArgEscape >> flag). >> There may be a way to make it work, but that would require more >> investigation, and I'm not sure the benefit outweighs the risk. >> >> dl >> >> On 12/15/18 6:48 AM, Claes Redestad wrote: >>> Hi Dean, >>> >>> to avoid escape analysis-eliminated allocations in the past @DontInline >>> has been sufficient. This means a simpler patch (no changes to native >>> code needed - added assertions notwithstanding) and passes your tests >>> with C2 (it'd concern me if Graal's EA sees through this trick, as it >>> might break some existing places where DontInline is used to this >>> effect): >>> >>> ??? /** >>> ???? * The value needs to be physically located in the frame, so >>> that it >>> ???? * can be found by a stack walk. >>> ???? */ >>> ??? @Hidden >>> ??? @DontInline >>> ??? private static void ensureMaterializedForStackWalk(Object o) {} >>> >>> Thanks! >>> >>> /Claes >>> >>> On 2018-12-15 01:59, dean.long at oracle.com wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>>> >>>> This change includes two new regression test that demonstrate the >>>> problem, and a fix that allows the tests >>>> to pass. >>>> >>>> The problem happens when the JIT compiler's escape analysis >>>> eliminates the allocation of the AccessControlContext object passed >>>> to doPrivileged.? The compiler thinks this is safe because it does >>>> not see that the object "escapes".? However, getContext needs to be >>>> able to find the object using a stack walk, so we need a way to >>>> tell the compiler that it does indeed escape.? To do this we pass >>>> the value to a native method that does nothing. >>>> >>>> Microbenchmark results: >>>> >>>> jdk12-b18: >>>> >>>> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >>>> >>>> >>>> jdk12-b19: >>>> >>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >>>> >>>> this fix: >>>> >>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >>>> >>>> >>>> dl >> From dean.long at oracle.com Mon Dec 17 06:28:40 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sun, 16 Dec 2018 22:28:40 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <790736fc-836c-d116-28c2-ffdc02c1e2ca@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> <790736fc-836c-d116-28c2-ffdc02c1e2ca@oracle.com> Message-ID: <30ef0a6a-68cf-d8a8-9c98-f83d220a52d5@oracle.com> On 12/16/18 9:52 PM, Claes Redestad wrote: > Fair enough, I wasn't aware EA was looking beyond the inlined code like > this (which means it can't be "dead" or the JIT might see through the > trick at some point). I've skimmed the existing usages in the JDK and > can't find anything that seems to be dependent on DontInline defeating > EA - maybe there was in the past, or I simply misremembered. Were you thinking of the pre-jdk11 implementation of Reference.reachabilityFence() perhaps? dl From Nick.Gasson at arm.com Mon Dec 17 08:26:43 2018 From: Nick.Gasson at arm.com (Nick Gasson (Arm Technology China)) Date: Mon, 17 Dec 2018 08:26:43 +0000 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: <98be78d0-d49d-cf4a-fda8-34cd33d5c813@oracle.com> References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> <98be78d0-d49d-cf4a-fda8-34cd33d5c813@oracle.com> Message-ID: <4f6d2937-e60c-78c8-844c-b54d0205e719@arm.com> Hi Dean, > Would you like to add comments as suggested by Boris? I think > it's a good idea. Sure, I've updated the webrev here: http://cr.openjdk.java.net/~njian/8214512/webrev.1/ Thanks, Nick On 17/12/2018 12:59, dean.long at oracle.com wrote: > Thanks.? Would you like to add comments as suggested by Boris?? I think > it's a good idea. > > dl > > On 12/16/18 6:00 PM, Nick Gasson (Arm Technology China) wrote: >> >> Hi Dean, >> >> Sorry for the delay. This is submitted on behalf of ARM Ltd. >> >> Thanks, >> >> Nick >> >> *From:* dean.long at oracle.com >> *Sent:* 13 December 2018 12:01 >> *To:* Nick Gasson (Arm Technology China) ; >> hotspot-dev at openjdk.java.net; Boris Ulasevich >> >> *Cc:* hotspot-compiler-dev at openjdk.java.net compiler >> ; nd ; >> aarch32-port-dev at openjdk.java.net >> *Subject:* Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java >> fails on ARM32 >> >> Nick, can you please confirm that you are covered under the >> >> ? ARM Ltd. - OpenJDK, MySQL >> >> >> OCA agreement?? I'm guessing Arm Technology China is under ARM Ltd., >> judging >> by your email domain, but please confirm. >> >> dl >> >> On 12/11/18 5:30 PM, dean.long at oracle.com >> wrote: >> >> I can help you commit it, but first I think another review would >> be good. >> >> dl >> >> On 12/11/18 5:15 PM, Nick Gasson (Arm Technology China) wrote: >> >> Hi Nick.? This change looks good to me. >> >> Thanks Dean. Could you help me to commit this patch if it >> doesn't need other reviews? >> >> Nick >> >> >> >> -----Original Message----- >> From: dean.long at oracle.com >> >> Sent: 12 December 2018 06:05 >> To: Nick Gasson (Arm Technology China) >> ; hotspot- >> dev at openjdk.java.net ; Boris >> Ulasevich >> >> Cc: nd ; >> hotspot-compiler-dev at openjdk.java.net >> compiler >> >> ; aarch32-port- >> dev at openjdk.java.net >> Subject: Re: RFR: 8214512: Jtreg test >> compiler/c2/Test8062950.java fails on >> ARM32 >> >> Hi Nick.? This change looks good to me. >> >> dl >> >> On 12/2/18 10:06 PM, Nick Gasson wrote: >> >> Hi, >> >> Could someone please help me review this fix for a >> test failure on ARM32: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 >> Webrev: >> http://cr.openjdk.java.net/~njian/8214512/webrev.0/ >> >> This fixes two assertion failures related to biased >> locking when C2's >> bias inlining feature is disabled >> (-XX:-OptoBiasInlining): >> >> MacroAssembler::atomic_cas_bool takes three register >> arguments plus a >> temporary register for use in the CAS loop. If the >> caller passes >> `noreg' as this temporary register (which happens in the >> !OptoBiasInlining call path from >> MacroAssembler::fast_lock) then >> MacroAssembler::atomic_cas_bool will default to using >> LR as a >> temporary. But it's possible with C2 that LR is one of >> the other three >> arguments which then triggers an >> assert_different_registers assertion >> failure. Fix this by supplying an additional scratch >> register to >> MacroAssembler::fast_lock if !OptoBiasInlining. >> >> In the !OptoBiasInlining case >> MacroAssembler::fast_lock calls >> MacroAssembler::biased_locking_enter to handle the >> case where the mark >> word contains the biased lock pattern which fast_lock >> wouldn't >> otherwise see if OptoBiasInlining was true. However in >> the case that >> we fail to acquire the biased lock this falls through >> to label >> `failed' and runs the normal fast_lock code that >> implicitly assumes >> the mark word does not have the bias pattern. This >> means we can end up >> with a BasicLock where _displaced_header contains the >> biased lock >> pattern which is an illegal state and subsequently >> triggers an >> assertion failure in ObjectSynchronizer::fast_exit. >> >> The right thing to do here is branch to `done' >> whenever the bias lock >> pattern is present and let the runtime handle the >> failure case.? Also >> edited the comment describing >> MacroAssembler::biased_locking_enter to >> more accurately describe what it does (the comment >> about `slow_case' >> being optional is wrong). >> >> Jtreg test compiler/c2/Test8062950.java now passes on >> ARM32 with this >> patch. >> >> Thanks, >> Nick >> > From matthias.baesken at sap.com Mon Dec 17 08:59:18 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 17 Dec 2018 08:59:18 +0000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release Message-ID: Hello, please review the following change. I noticed that we miss at some places (for example in case of early returns) where GetByteArrayElements is used, the corresponding ReleaseByteArrayElements call. In VirtualMachineImpl.c I also removed a check for isCopy (is the returned byte array a copy ?) because from what I read at https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html the ReleaseByteArrayElements routine should always be called. bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8215411 http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ Thanks, Matthias From robbin.ehn at oracle.com Mon Dec 17 10:39:01 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 17 Dec 2018 11:39:01 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> Message-ID: <15bf0187-fb9e-0724-d1a6-c4b4030fe3b9@oracle.com> Hi David, On 11/26/18 7:12 AM, David Holmes wrote: > Hi Robbin, > > On 24/11/2018 2:55 am, Robbin Ehn wrote: >> Forgot RFR in subject. > > Yep and now you have two different review threads happening in parallel > unfortunately :( > > - src/hotspot/share/utilities/waitBarrier.hpp > > I'm studying just the WaitBarrierType API. Is this inherently tied to safepoint > usage or intended as a general synchronization tool? As a general tool the API > does not have clear semantics on how it should be used: There is two potential uses-cases in G1, which might require some changes. > > - How do you communicate the current tag between the arming thread and the > waiting threads? There would seem to be inherent races between arm(tag) and > wait(tag) unless access to the tag itself is synchronized via another mechanism. As seen in gtest: - arm tag - publish tag > > - What happens if wake() is called before disarm()? Should it be disallowed? > (For other readers disarm() and wake() are distinct operations so that they fit > better into the existing safepoint protocol which disarms the safepoint polling > page at one spot and wakes blocked threads at another.) Yes, wake() have no effect if you have not disarmed. There are asserts checking this. > > - Should there be constraints that the same thread must arm/disarm/wake? It > doesn't really make sense to allow these operations to happen in arbitrary order > from multiple threads. Maybe, *thinking* if there is nice way to get the arming/waking thread into an argument. > > The semantics for re-arming with the same tag should be clearly set out not > "implementation-defined". This should probably be a usage error IMHO - but it > comes back to how the tag is expected to be used. If you have a way to detect that all threads have left the waitbarrier, there is no problem with re-using the same tag. So yes, re-arming before that with the same tag should be an error. But since the waitbarrier can't tell the difference and do not know which tag it was last armed for I have no assert. > > As Andrew mentioned there needs to be documentation regarding spurious wakeups > or other "interruptions" at the API level. And I assume a blocked wait() only > ever returns in response to a wake(). Yes, if it's disarmed before. Since these are handled internally there are no spurious wakeups, not sure why we want to document that, except inside the implementation. E.g. semaphore.hpp does not document that it do not have it. > > Nothwithstanding clarification of the above may I suggest the following rewrite > of the API documentation for further clarity: > > /* Platform independent WaitBarrier API. > ?? An armed WaitBarrier prevents threads from advancing until the > ?? barrier is disarmed and the waiting threads woken. The barrier is > ?? armed by setting a non-zero value - the tag. > > ?? Expected Usage: > ???? - Arming thread: > ???????? tag = ...;? // non-zero value > ???????? barrier.arm(tag); > ???????? > ???????? barrier.disarm(); > ???????? barrier.wake(); > > ?????? - After arm(tag) returns any thread calling wait(tag) will block > ?????? - After disarm() returns any subsequent calls to wait(tag) will not block > ?????? - After wake() returns all blocked threads are unblocked and eligible to > execute again > ?????? - After calling disarm() and wake() the barrier is ready to be re-armed > with a new tag > > ??? - Waiting threads > ??????? wait(tag); // don't execute following code unless 'safe' > ??????? > > ????? - A call to wait(tag) will block if the barrier is armed with the value > 'tag'; else it will return immediately. > ????? - A blocked thread is eligible to execute again once the barrier is > disarmed and wake() has been called. > > ?A primary goal of the WaitBarrier implementation is to wake all waiting > ?threads as fast, and as concurrently, as possible. > > */ Updated, added after barrier.arm(tag); > > Looking at the "implementation" in this file I'm unclear on the way the Linux > specialization is being handled here. Why do we need the template? Can't this > just be done the same way we do Semaphore? Because the gtest can test all implementations available on that platform. Meaning I can test both implementation locally and compare performance etc.. I'll post one update after I sorted out the other review parts also. Thanks, Robbin > > More to follow. > > Thanks, > David > ----- > >> /Robbin >> >> On 2018-11-23 17:51, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> When a safepoint is ended we need a way to get back to 100% utilization as fast >>> as possible. 100% utilization means no idle cpu in the system if there is a >>> JavaThread that could be executed. The traditional ways to wake many, e.g. >>> semaphore, pthread_cond, is not implemented with a single syscall instead they >>> typical do one syscall per thread to wake. >>> >>> This change-set contains that primitive, the WaitBarrier, and a gtest for it. >>> No actual users, which is in coming patches. >>> >>> The WaitBarrier solves by doing a cooperative semaphore posting, threads woken >>> will also post. On Linux we can instead directly use a futex and with one >>> syscall wake all. Depending on how many threads and cpus the performance vary, >>> but a good utilization of the machine, just on the edge of saturated, the >>> time to reach 100% utilization is around 3 times faster with the WaitBarrier >>> (where futex is faster than semaphore). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>> >>> Passes 100 iterations of gtest on our platforms, both fastdebug and release. >>> And have been stable when used in safepoints (t1-8) (coming patches). >>> >>> Thanks, Robbin From robbin.ehn at oracle.com Mon Dec 17 11:08:19 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 17 Dec 2018 12:08:19 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <5291cfa5-65ab-8dcf-44e4-e2253141343a@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <5291cfa5-65ab-8dcf-44e4-e2253141343a@oracle.com> Message-ID: <7f870ba3-1bac-2942-5bfa-71eb71bb4b49@oracle.com> Thanks, fixed. I'll send update once I sorted out part 3. On 11/28/18 5:17 AM, David Holmes wrote: > Hi Robbin, > > Part 2: > > I've looked at the generic waitBarrier implementation and it seems okay. A lot > of the usage constraints not clear in the API are more evident in the asserts in > the code. I still have some doubts about the benefit of helping with the wakeups > rather than it all being done by one thread, due to potential contention on the > semaphore - but your performance numbers show it can help (I just hope it > doesn't hurt on other platforms/systems - time will tell). I think once of the best features is if the VM thread gets swapped out there is a big chance that other threads keep posting. An alternative option is to create a new Semaphore for each arm and thus allow extra posts. /Robbin > > Some minor nits with comments: > > - src/hotspot/share/utilities/waitBarrier_generic.hpp > > 31 // Except for the barrier tag it self, it uses two counter to keep the semaphore > ?32 // count correct and not leave any late thread hanging. > > s/it self/itself/ > > s/counter/counters/ > > suggest s/hanging/waiting/ > > 38?? // Which means it can become a waiter. > > suggest -> // These threads can become waiters. > > --- > > -? src/hotspot/share/utilities/waitBarrier_generic.cpp > > 55?? // We need an exact count and never go below 0. > 56?? // Otherwise the semaphore might contain to many posts. > > suggest: > > // We need an exact count which never goes below zero, > // otherwise the semaphore may be signalled too many times. > > 74???? // We must loop here until there is no waiters or potential waiters. > > s/is/are/ > > More to follow. > > Thanks, > David > > On 24/11/2018 2:55 am, Robbin Ehn wrote: >> Forgot RFR in subject. >> >> /Robbin >> >> On 2018-11-23 17:51, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> When a safepoint is ended we need a way to get back to 100% utilization as fast >>> as possible. 100% utilization means no idle cpu in the system if there is a >>> JavaThread that could be executed. The traditional ways to wake many, e.g. >>> semaphore, pthread_cond, is not implemented with a single syscall instead they >>> typical do one syscall per thread to wake. >>> >>> This change-set contains that primitive, the WaitBarrier, and a gtest for it. >>> No actual users, which is in coming patches. >>> >>> The WaitBarrier solves by doing a cooperative semaphore posting, threads woken >>> will also post. On Linux we can instead directly use a futex and with one >>> syscall wake all. Depending on how many threads and cpus the performance vary, >>> but a good utilization of the machine, just on the edge of saturated, the >>> time to reach 100% utilization is around 3 times faster with the WaitBarrier >>> (where futex is faster than semaphore). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>> >>> Passes 100 iterations of gtest on our platforms, both fastdebug and release. >>> And have been stable when used in safepoints (t1-8) (coming patches). >>> >>> Thanks, Robbin From robbin.ehn at oracle.com Mon Dec 17 11:18:12 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 17 Dec 2018 12:18:12 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> Message-ID: <9e9c73ae-2c5e-400d-e541-0f69a401670f@oracle.com> Hi David, On 11/28/18 6:41 AM, David Holmes wrote: > Hi Robbin, > > Part 3 (and final part) > > This is looking at the futex implementation and the test. > > Overall looks good! Great! > > Shouldn't you be using the FUTEX_*_PRIVATE variants as this is a process-local > FUTEX? Yes > > Your main code would be simpler to read (and thus understand) if your futex > helper function handled the unused arguments e.g. instead of: > > static int futex(volatile int *uaddr, int futex_op, int val, > ???????????????? const struct timespec *timeout, int *uaddr2, int val3) > ?{ > ?? return syscall(SYS_futex, uaddr, futex_op, val, timeout, uaddr2, val3); > ?} > > use (with name changes): > > static int futex(volatile int *addr, int futex_op, int op_arg) { > ?? return syscall(SYS_futex, addr, futex_op, op_arg, NULL, NULL, 0); > } > Fixed > --- > > Minor suggestions on this comment block: > > ? 88???? // Return value 0, re-check in case of spurious wake-up. > ? 89???? // EINTR and re-check and go back to waiting. > ? 90???? // EAGAIN we already are disarmed, we should pass the check, > ? 91???? // if not re-armed with same tag. > > -> > > ? // Return value 0: woken up, but re-check in case of spurious wakeup > ? // Error EINTR: woken by signal, so re-check and re-wait if necessary. > ? // Error EAGAIN: we are already disarmed and so will pass the check > Fixed > As per initial review comments I think re-arming with same tag should be an > error. I think the tag semantics should be that the tag is a strictly increasing > value, and we can then store the previous tag and check that. Since the futex only have a 32-bit this would eventually wrap. But we could save previously armed tag make sure the new one is different. > > --- > > The test could do with at least an introductory comment block explaining the > basic operation of the test. Further inline comments wouldn't hurt either. Sure! Sending update with reply to first RFR mail, when tested and gtest commented. /Robbin > > Thanks, > David > > On 24/11/2018 2:55 am, Robbin Ehn wrote: >> Forgot RFR in subject. >> >> /Robbin >> >> On 2018-11-23 17:51, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> When a safepoint is ended we need a way to get back to 100% utilization as fast >>> as possible. 100% utilization means no idle cpu in the system if there is a >>> JavaThread that could be executed. The traditional ways to wake many, e.g. >>> semaphore, pthread_cond, is not implemented with a single syscall instead they >>> typical do one syscall per thread to wake. >>> >>> This change-set contains that primitive, the WaitBarrier, and a gtest for it. >>> No actual users, which is in coming patches. >>> >>> The WaitBarrier solves by doing a cooperative semaphore posting, threads woken >>> will also post. On Linux we can instead directly use a futex and with one >>> syscall wake all. Depending on how many threads and cpus the performance vary, >>> but a good utilization of the machine, just on the edge of saturated, the >>> time to reach 100% utilization is around 3 times faster with the WaitBarrier >>> (where futex is faster than semaphore). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>> >>> Passes 100 iterations of gtest on our platforms, both fastdebug and release. >>> And have been stable when used in safepoints (t1-8) (coming patches). >>> >>> Thanks, Robbin From jcbeyler at google.com Mon Dec 17 16:43:41 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 17 Dec 2018 08:43:41 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: References: <895ef766-9c96-7185-4222-178379629ce4@oracle.com> <04a464fa-c1c8-5d86-3633-0b532840561c@oracle.com> <7ef06464-a614-8941-bb51-ce1c467889b2@oracle.com> <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> <839301b7-c247-df3b-e485-283e8bb7388b@oracle.com> <95fe277d-ba6e-4fec-77aa-d1f1051751aa@oracle.com> <72bf2f4a-5bf7-98de-5f00-68485072923d@oracle.com> Message-ID: Hi all, I don't believe I got actual LGTM for this version: Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 It removed the namespaces and uses explicit static instead :) Thanks! Jc On Wed, Dec 12, 2018 at 8:06 PM JC Beyler wrote: > So did I Alexey but with David & Serguei preferring static, it seems more > reasonable to go down their route :-) > > So here is the latest webrev with static instead of an anonymous namespace: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > Let me know what you think, can I get a webrev 06 review? > > Thanks! > Jc > > On Wed, Dec 12, 2018 at 3:10 PM Alex Menkov > wrote: > >> Hm.. >> I considered unnamed namespaces "C++ style" (and static globals as "C >> style"). >> Static globals were deprecated in C++ (but some time ago the deprecation >> was reverted). >> >> --alex >> >> On 12/12/2018 13:55, serguei.spitsyn at oracle.com wrote: >> > Agreed. >> > >> > Thanks, >> > Serguei >> > >> > >> > On 12/12/18 13:52, David Holmes wrote: >> >> FWIW I think namespaces are overkill in all of this test code and just >> >> obfuscates things - the declaration is easily missed. A static >> >> variable in a .cpp is clearly a global variable to the file. >> >> >> >> Cheers, >> >> David >> >> >> >> >> >> >> >> On 13/12/2018 5:37 am, serguei.spitsyn at oracle.com wrote: >> >>> Hi Jc, >> >>> >> >>> >> >>> On 12/11/18 21:16, JC Beyler wrote: >> >>>> Hi all, >> >>>> >> >>>> Here is the new webrev with the TEST.groups change. Serguei, let me >> >>>> know if I convinced you with the static vs anonymous namespaces or >> >>>> if you'd still rather have a "static" for now :-) >> >>> >> >>> >> >>> What do you think about this post? : >> >>> >> https://stackoverflow.com/questions/11623451/static-vs-non-static-variables-in-namespace >> >>> >> >>> >> >>>> >> >>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ >> >>>> >> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> >>> >> >>> The update looks fine. >> >>> >> >>> Thanks, >> >>> Serguei >> >>> >> >>> >> >>> Thanks, >> >>> Serguei >> >>> >> >>>> >> >>>> Thanks again for the reviews! >> >>>> Jc >> >>>> >> >>>> On Mon, Dec 10, 2018 at 3:10 PM JC Beyler > >>>> > wrote: >> >>>> >> >>>> Hi Serguei, >> >>>> >> >>>> Yes basically it is equivalent :) I can put them in but they are >> >>>> not required. The norm actually wanted to deprecate it but then >> >>>> remembered that C compatibility would require the static key-word >> >>>> for this case [1] >> >>>> >> >>>> So, really, they are not required here and will amount to the >> same >> >>>> thing: only that file can refer to them and you cannot get to >> them >> >>>> without a globally available method to return a pointer to them >> >>>> (ie same as a static variable in C). >> >>>> >> >>>> I can put static if it makes it easier to see but, by being in an >> >>>> anonymous namespace they are only available for the file's >> >>>> translation unit. For example: >> >>>> >> >>>> $ cat main.cpp >> >>>> >> >>>> int totally_global; >> >>>> static int explictly_static; >> >>>> >> >>>> namespace { >> >>>> int implicitly_static; >> >>>> } >> >>>> >> >>>> void foo(); >> >>>> int main() { >> >>>> foo(); >> >>>> } >> >>>> >> >>>> $ g++ -O3 main.cpp -c >> >>>> $ nm main.o >> >>>> U _GLOBAL_OFFSET_TABLE_ >> >>>> 0000000000000000 T main >> >>>> 0000000000000000 B totally_global >> >>>> U _Z3foov >> >>>> >> >>>> As you can see, the static and anonymous namespace variables are >> >>>> not in the file due to not being used. If you were to use them, >> >>>> you'd see them show up as something like: >> >>>> 0000000000000008 b _ZL17explicitly_static >> >>>> 0000000000000004 b _ZN12_GLOBAL__N_117implicitly_staticE >> >>>> >> >>>> Where again, it shows that it is mangling the names so that no >> >>>> external usage can happen without tinkering. >> >>>> >> >>>> Hopefully that helps :-), >> >>>> Jc >> >>>> >> >>>> [1] >> >>>> http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 >> >>>> >> >>>> >> >>>> On Mon, Dec 10, 2018 at 2:04 PM serguei.spitsyn at oracle.com >> >>>> > >>>> > wrote: >> >>>> >> >>>> Hi Jc, >> >>>> >> >>>> I had little experience with the C++ namespaces. >> >>>> My understanding is that static in this context should mean >> >>>> internal linkage. >> >>>> >> >>>> Thanks, >> >>>> Serguei >> >>>> >> >>>> >> >>>> On 12/10/18 13:57, JC Beyler wrote: >> >>>>> Hi Serguei, >> >>>>> >> >>>>> The variables and functions are in a anonymous namespace; my >> >>>>> understanding of C++ is that this is equivalent to putting >> it >> >>>>> as static.Hence, I didn't add them there. Does that make >> >>>>> sense? >> >>>>> >> >>>>> Thanks! >> >>>>> Jc >> >>>>> >> >>>>> On Mon, Dec 10, 2018 at 1:33 PM serguei.spitsyn at oracle.com >> >>>>> >> >>>>> > >>>>> > wrote: >> >>>>> >> >>>>> Hi Jc, >> >>>>> >> >>>>> It looks good in general. >> >>>>> One question though. >> >>>>> >> >>>>> >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >> >>>>> >> >>>>> >> >>>>> I wonder if the variables and functions have to be >> static. >> >>>>> >> >>>>> Thanks, >> >>>>> Serguei >> >>>>> >> >>>>> >> >>>>> On 12/5/18 11:36, JC Beyler wrote: >> >>>>>> Hi all, >> >>>>>> >> >>>>>> My apologies to having to come back for another review >> >>>>>> for this change: I ran into a snag when trying to pull >> >>>>>> the latest changes compared to the base I was working >> >>>>>> on. I basically forgot that there was an issue with >> >>>>>> snprintf and that I had solved it via JDK-8213622. >> >>>>>> >> >>>>>> Could I have a new review of this webrev: >> >>>>>> Webrev: >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ >> >>>>>> >> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> >>>>>> Incremental from the port of webrev.03 that got LGTMs: >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ >> >>>>>> >> >>>>>> >> >>>>>> A few comments on this because it took me a while to >> get >> >>>>>> things in a state I thought was good: >> >>>>>> - I had to implement an itoa method, do we have >> >>>>>> something like that in the test base (remember that >> >>>>>> JDK-8213622 could not use sprintf due to being in the >> >>>>>> test code)? >> >>>>>> >> >>>>>> - The differences here compared to the one you all >> >>>>>> reviewed are: >> >>>>>> - I found that adding to the strlen/memcpy error >> >>>>>> prone and thought that I would try to make it less so. >> >>>>>> If you want to compare, I extended the strlen/memcpy >> >>>>>> with the new format to show you if you prefer [1] >> >>>>>> - Note that the diff between the "old >> >>>>>> extended way from [1]" to the webrev.04 can be found >> >>>>>> in [2] >> >>>>>> >> >>>>>> - I added a test to test the exception wrapper in >> >>>>>> tests :); I'm not sure it is deemed useful or not but >> >>>>>> helped me assure myself that I was not doing things >> >>>>>> wrong; you can find the base test file here [3]; should >> >>>>>> we have this or not? (I know that normally we don't add >> >>>>>> tests to vmTestbase but thought this might be an >> >>>>>> exception) >> >>>>>> >> >>>>>> Thanks for your help and my apologies for the snag, >> >>>>>> Jc >> >>>>>> >> >>>>>> [1]: >> >>>>>> >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >> >>>>>> >> >>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html> >> >> >>>>>> >> >>>>>> [2]: >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 >> >>>>>> >> >>>>>> [3] >> >>>>>> >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html >> >>>>>> >> >>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html> >> >> >>>>>> >> >>>>>> >> >>>>>> On Mon, Dec 3, 2018 at 11:29 PM David Holmes >> >>>>>> > >>>>>> > wrote: >> >>>>>> >> >>>>>> Looks fine to me. >> >>>>>> >> >>>>>> Thanks, >> >>>>>> David >> >>>>>> >> >>>>>> On 4/12/2018 4:04 pm, JC Beyler wrote: >> >>>>>> > Hi both, >> >>>>>> > >> >>>>>> > Thanks for the reviews! Since Serguei did not >> >>>>>> insist on get_basename, I >> >>>>>> > went for get_dirname since the method is a local >> >>>>>> static method and won't >> >>>>>> > have its name start spreading, I think it's ok >> too. >> >>>>>> > >> >>>>>> > For the naming of the local variable, the idea >> >>>>>> initially was to use the >> >>>>>> > same name as the local variable for JNIEnv >> already >> >>>>>> used to reduce the >> >>>>>> > code change. Since I'm now adding the line macro >> >>>>>> at the end anyway, this >> >>>>>> > does not matter anymore so I converged all local >> >>>>>> variables to "jni". >> >>>>>> > >> >>>>>> > So, without further ado, here is the new version: >> >>>>>> > Webrev: >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ >> >>>>>> >> >>>>>> > Bug: >> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> >>>>>> > >> >>>>>> > This passes the various tests changed by the >> >>>>>> webrev on my dev machine. >> >>>>>> > >> >>>>>> > Let me know what you think, >> >>>>>> > Jc >> >>>>>> > >> >>>>>> > On Mon, Dec 3, 2018 at 8:40 PM >> >>>>>> serguei.spitsyn at oracle.com >> >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>>>>> >> >>>>>> > > >>>>>> >> wrote: >> >>>>>> > >> >>>>>> > On 12/3/18 20:15, Chris Plummer wrote: >> >>>>>> > > Hi JC, >> >>>>>> > > >> >>>>>> > > Overall it looks good. A few naming nits >> >>>>>> thought: >> >>>>>> > > >> >>>>>> > > In bi01t001.cpp, why have you declared the >> >>>>>> > ExceptionCheckingJniEnvPtr >> >>>>>> > > using jni_env(jni). Elsewhere you use >> >>>>>> jni(jni_env) and rename the >> >>>>>> > > method argument passed in from jni to >> >>>>>> jni_env. >> >>>>>> > > >> >>>>>> > > Related to this, I also noticed in some >> >>>>>> files that already are using >> >>>>>> > > ExceptionCheckingJniEnvPtr, such as >> >>>>>> CharArrayCriticalLocker.cpp, you >> >>>>>> > > delcared it as env(jni_env). So that means >> >>>>>> there are 3 different >> >>>>>> > names >> >>>>>> > > you have used for the >> >>>>>> ExceptionCheckingJniEnvPtr local variable. >> >>>>>> > They >> >>>>>> > > should be consistent. >> >>>>>> > > >> >>>>>> > > Also, can you rename get_basename() to >> >>>>>> get_dirname()? I know Serguei >> >>>>>> > > suggested get_basename() a while back, but >> >>>>>> unless "basename" is >> >>>>>> > > commonly used for this purpose, I think >> >>>>>> "dirname" is more self >> >>>>>> > > explanatory. >> >>>>>> > >> >>>>>> > In general, I'm Okay with get_dirname(). >> >>>>>> > Just to mention dirname can be both short or >> >>>>>> full, so it is a little >> >>>>>> > confusing as well. >> >>>>>> > It is the reason why the get_basename() was >> >>>>>> suggested. >> >>>>>> > However, I do not insist on get_basename() >> nor >> >>>>>> get_full_dirname(). :) >> >>>>>> > >> >>>>>> > Thanks, >> >>>>>> > Serguei >> >>>>>> > >> >>>>>> > >> >>>>>> > > thanks, >> >>>>>> > > >> >>>>>> > > Chris >> >>>>>> > > >> >>>>>> > > On 12/2/18 10:29 PM, David Holmes wrote: >> >>>>>> > >> Hi Jc, >> >>>>>> > >> >> >>>>>> > >> I've been lurking on this one and have >> had >> >>>>>> a look through. I'm okay >> >>>>>> > >> with the FatalError approach for the >> tests >> >>>>>> - we don't expect >> >>>>>> > anything >> >>>>>> > >> to go wrong in a well written test in a >> >>>>>> correctly functioning VM. >> >>>>>> > >> >> >>>>>> > >> Thanks, >> >>>>>> > >> David >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> > >> On 3/12/2018 3:24 pm, JC Beyler wrote: >> >>>>>> > >>> Hi all, >> >>>>>> > >>> >> >>>>>> > >>> Would someone on the GC or runtime team >> >>>>>> be motivated to give >> >>>>>> > this a >> >>>>>> > >>> review? :) >> >>>>>> > >>> >> >>>>>> > >>> It would be much appreciated! >> >>>>>> > >>> >> >>>>>> > >>> Webrev: >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> >>>>>> >> >>>>>> > >>> Bug: >> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> >>>>>> > >>> >> >>>>>> > >>> Thanks for your help, >> >>>>>> > >>> Jc >> >>>>>> > >>> >> >>>>>> > >>> On Tue, Nov 27, 2018 at 4:36 PM JC >> Beyler >> >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>> > >>>>>> >> >>>>>> > >>>>>> >>> wrote: >> >>>>>> > >>> >> >>>>>> > >>> Hi Chris, >> >>>>>> > >>> >> >>>>>> > >>> Yes I was waiting for another review >> >>>>>> since you had explicitly >> >>>>>> > >>> asked :) >> >>>>>> > >>> >> >>>>>> > >>> And sounds good that when someone >> >>>>>> from GC or runtime gives a >> >>>>>> > >>> review, >> >>>>>> > >>> I'll wait for your full review on >> the >> >>>>>> webrev.02! >> >>>>>> > >>> >> >>>>>> > >>> Thanks again for your help, >> >>>>>> > >>> Jc >> >>>>>> > >>> >> >>>>>> > >>> >> >>>>>> > >>> On Tue, Nov 27, 2018 at 12:48 PM >> >>>>>> Chris Plummer >> >>>>>> > >>> > >>>>>> >> >>>>>> > >>>>>> > >> >>>>>> > > >>>>>> >> >>>>>> > >>>>>> >>> >> >>>>>> > wrote: >> >>>>>> > >>> >> >>>>>> > >>> Hi JC, >> >>>>>> > >>> >> >>>>>> > >>> I think it would be good to get >> a >> >>>>>> review from the gc or >> >>>>>> > runtime >> >>>>>> > >>> teams, since this also affects >> >>>>>> their tests. >> >>>>>> > >>> >> >>>>>> > >>> Also, once we are settled on >> this >> >>>>>> FatalError approach, >> >>>>>> > I still >> >>>>>> > >>> need to give your webrev-02 a >> >>>>>> full review. I only >> >>>>>> > skimmed over >> >>>>>> > >>> parts of it (I did look at all >> >>>>>> the changes in webrevo-01). >> >>>>>> > >>> >> >>>>>> > >>> thanks, >> >>>>>> > >>> >> >>>>>> > >>> Chris >> >>>>>> > >>> >> >>>>>> > >>> On 11/27/18 8:58 AM, >> >>>>>> serguei.spitsyn at oracle.com >> >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>> > >>>>>> >> >>>>>> > > >>>>>> >> wrote: >> >>>>>> > >>>> Hi Jc, >> >>>>>> > >>>> >> >>>>>> > >>>> I've already reviewed this too. >> >>>>>> > >>>> >> >>>>>> > >>>> Thanks, >> >>>>>> > >>>> Serguei >> >>>>>> > >>>> >> >>>>>> > >>>> >> >>>>>> > >>>> On 11/27/18 06:56, JC Beyler >> >>>>>> wrote: >> >>>>>> > >>>>> Thanks Chris, >> >>>>>> > >>>>> >> >>>>>> > >>>>> Anybody else motivated to look at this >> >>>>>> and review it? :) >> >>>>>> > >>>>> Jc >> >>>>>> > >>>>> >> >>>>>> > >>>>> On Mon, Nov 26, 2018 at 1:26 >> PM >> >>>>>> Chris Plummer >> >>>>>> > >>>>> > >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>>>>> >> >>>>>> > > >>>>>> >>> >> >>>>>> > >>>>> wrote: >> >>>>>> > >>>>> >> >>>>>> > >>>>> Hi JC, >> >>>>>> > >>>>> >> >>>>>> > >>>>> I'm ok with the FatalError approach, >> >>>>>> but would >> >>>>>> > like to >> >>>>>> > >>>>> hear opinions from others also. >> >>>>>> > >>>>> >> >>>>>> > >>>>> thanks, >> >>>>>> > >>>>> >> >>>>>> > >>>>> Chris >> >>>>>> > >>>>> >> >>>>>> > >>>>> On 11/21/18 8:19 AM, JC Beyler wrote: >> >>>>>> > >>>>>> Hi Chris, >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> Thanks for taking the >> time >> >>>>>> to look at it and yes you >> >>>>>> > >>>>>> have raised exactly why >> >>>>>> the webrev is between two >> >>>>>> > >>>>>> worlds: in cases where a >> >>>>>> fatal error on failure is >> >>>>>> > >>>>>> wanted, should we >> simplify >> >>>>>> the code to remove >> >>>>>> > the return >> >>>>>> > >>>>>> tests since we do them >> >>>>>> internally? Now that I've >> >>>>>> > looked >> >>>>>> > >>>>>> around for non-fatal >> >>>>>> cases, I think the answer >> >>>>>> > is yes, >> >>>>>> > >>>>>> it simplifies the code >> >>>>>> while maintaining the checks. >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> I looked a bit and it >> >>>>>> seems that I can't find >> >>>>>> > easily a >> >>>>>> > >>>>>> case where the test >> >>>>>> accepts a JNI failure to >> >>>>>> > then move >> >>>>>> > >>>>>> on. Therefore, perhaps, >> >>>>>> for now, the fail with a >> >>>>>> > Fatal >> >>>>>> > >>>>>> is enough and we can work >> >>>>>> on the tests to clean >> >>>>>> > them up? >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> That means that this is >> >>>>>> the new webrev with only >> >>>>>> > Fatal >> >>>>>> > >>>>>> and cleans up the tests >> so >> >>>>>> that it is no longer in >> >>>>>> > >>>>>> between two worlds: >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> Webrev: >> >>>>>> > >>>>>> >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> >>>>>> >> >>>>>> > >>>>>> >> >>>>>> >> >>>>>> > >>>>>> Bug: >> >>>>>> > https://bugs.openjdk.java.net/browse/JDK-8213501 >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> (This passes testing on >> my >> >>>>>> dev machine for all the >> >>>>>> > >>>>>> modified tests) >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> with the example you >> >>>>>> provided, it now looks like: >> >>>>>> > >>>>>> >> >>>>>> > >> >>>>>> >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> >>>>>> >> >>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >> >>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> Where it does, to me at >> >>>>>> least, seem cleaner and less >> >>>>>> > >>>>>> "noisy". >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> Let me know what you >> think, >> >>>>>> > >>>>>> Jc >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> On Tue, Nov 20, 2018 at >> >>>>>> 9:33 PM Chris Plummer >> >>>>>> > >>>>>> < >> chris.plummer at oracle.com >> >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>>>>> > >>>>>> >> >>>>>> > > >>>>>> >>> wrote: >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> Hi JC, >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> Sorry about the >> delay. >> >>>>>> I had to go back an >> >>>>>> > look at >> >>>>>> > >>>>>> the initial 8210842 >> >>>>>> webrev and RFR thread to see >> >>>>>> > >>>>>> what this was >> >>>>>> initially all about. >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> In general the >> changes >> >>>>>> look good. >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> I don't have a good >> >>>>>> answer to your >> >>>>>> > >>>>>> FatalError/NonFatalError question. It >> makes >> >>>>>> > the code >> >>>>>> > >>>>>> a lot cleaner to use >> >>>>>> FatalError, but then it >> >>>>>> > is a >> >>>>>> > >>>>>> behavior change, and >> >>>>>> you also need to deal with >> >>>>>> > >>>>>> tests that >> >>>>>> intentionally induce errors (do >> >>>>>> > you have >> >>>>>> > >>>>>> an example of that). >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> In any case, right >> now >> >>>>>> your webrev seems to be >> >>>>>> > >>>>>> between two worlds. >> >>>>>> You are producing >> >>>>>> > FatalError, >> >>>>>> > >>>>>> but still checking >> >>>>>> results. Here's a good >> >>>>>> > example: >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >> >>>>>> >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> >>>>>> >> >>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >> >>>>>> < >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html> >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> I'm not sure if this >> >>>>>> is just a temporary >> >>>>>> > state until >> >>>>>> > >>>>>> it was decided which >> >>>>>> approach to take. >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> thanks, >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> Chris >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> On 11/20/18 2:14 PM, >> >>>>>> JC Beyler wrote: >> >>>>>> > >>>>>>> Hi all, >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Chris thought it >> made >> >>>>>> sense to have more >> >>>>>> > eyes on >> >>>>>> > >>>>>>> this change than >> just >> >>>>>> serviceability as it will >> >>>>>> > >>>>>>> modify to tests that >> >>>>>> are not only >> >>>>>> > serviceability >> >>>>>> > >>>>>>> tests so I've moved >> >>>>>> this to conversation >> >>>>>> > here :) >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> For convenience, >> I've >> >>>>>> copy-pasted the >> >>>>>> > initial RFR: >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Could I have a >> review >> >>>>>> for the extension and >> >>>>>> > usage >> >>>>>> > >>>>>>> of the >> >>>>>> ExceptionJniWrapper. This adds lines and >> >>>>>> > >>>>>>> filenames to the end >> >>>>>> of the wrapper JNI >> >>>>>> > methods, >> >>>>>> > >>>>>>> adds tracing, and >> >>>>>> throws an error if need >> >>>>>> > be. I've >> >>>>>> > >>>>>>> ported the gc/lock >> >>>>>> files to use the new >> >>>>>> > >>>>>>> TRACE_JNI_CALL >> add-on >> >>>>>> and I've ported a few >> >>>>>> > of the >> >>>>>> > >>>>>>> tests that were >> >>>>>> already changed for the >> >>>>>> > assignment >> >>>>>> > >>>>>>> webrev for >> >>>>>> JDK-8212884. >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Webrev: >> >>>>>> > >>>>>>> >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >> >>>>>> >> >>>>>> > >>>>>>> >> >>>>>> >> >>>>>> > >>>>>>> Bug: >> >>>>>> > >>>>>>> >> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> For illustration, if >> >>>>>> I force an error to the >> >>>>>> > >>>>>>> AP04/ap04t03 test >> and >> >>>>>> set the verbosity on, >> >>>>>> > I get >> >>>>>> > >>>>>>> something like: >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> >> Calling JNI >> method >> >>>>>> FindClass from >> >>>>>> > >>>>>>> ap04t003.cpp:343 >> >>>>>> > >>>>>>> >> Calling with >> these >> >>>>>> parameter(s): >> >>>>>> > >>>>>>> java/lang/Threadd >> >>>>>> > >>>>>>> Wait for thread to >> >>>>>> finish >> >>>>>> > >>>>>>> << Called JNI method >> >>>>>> FindClass from >> >>>>>> > >>>>>>> ap04t003.cpp:343 >> >>>>>> > >>>>>>> Exception in thread >> >>>>>> "Thread-0" >> >>>>>> > >>>>>>> java.lang.NoClassDefFoundError: >> >>>>>> > java/lang/Threadd >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> >>>>>> >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Method) >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Caused by: >> >>>>>> java.lang.ClassNotFoundException: >> >>>>>> > >>>>>>> java.lang.Threadd >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> >>>>>> > >>>>>>> ... 3 more >> >>>>>> > >>>>>>> FATAL ERROR in >> native >> >>>>>> method: JNI method >> >>>>>> > FindClass >> >>>>>> > >>>>>>> : internal error >> from >> >>>>>> ap04t003.cpp:343 >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> >>>>>> >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Method) >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> at >> >>>>>> > >>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Questions/comments I >> >>>>>> have about this are: >> >>>>>> > >>>>>>> - Do we want to >> >>>>>> force fatal errors when a JNI >> >>>>>> > >>>>>>> call fails in >> >>>>>> general? Most of these tests >> >>>>>> > do the >> >>>>>> > >>>>>>> right thing and test >> >>>>>> the return of the JNI >> >>>>>> > calls, >> >>>>>> > >>>>>>> for example: >> >>>>>> > >>>>>>> thrClass = >> >>>>>> > jni->FindClass("java/lang/Threadd", >> >>>>>> > >>>>>>> TRACE_JNI_CALL); >> >>>>>> > >>>>>>> if (thrClass == >> >>>>>> NULL) { >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> but now the wrapper >> >>>>>> actually would do a >> >>>>>> > fatal if >> >>>>>> > >>>>>>> the FindClass call >> >>>>>> would return a nullptr, >> >>>>>> > so we >> >>>>>> > >>>>>>> could remove that >> >>>>>> test altogether. What do you >> >>>>>> > >>>>>>> think? >> >>>>>> > >>>>>>> - I prefer to >> >>>>>> leave them as the tests then >> >>>>>> > >>>>>>> become closer to >> what >> >>>>>> real users would have in >> >>>>>> > >>>>>>> their code and is >> the >> >>>>>> "recommended" way of >> >>>>>> > doing it >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> - The alternative >> >>>>>> is to use the >> >>>>>> > NonFatalError I >> >>>>>> > >>>>>>> added which then >> just >> >>>>>> prints out that something >> >>>>>> > >>>>>>> went wrong, letting >> >>>>>> the test continue. Question >> >>>>>> > >>>>>>> will be what should >> >>>>>> be the default? The >> >>>>>> > fatal or >> >>>>>> > >>>>>>> the non-fatal error >> >>>>>> handling? >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> On a different >> >>>>>> subject: >> >>>>>> > >>>>>>> - On the new >> tests, >> >>>>>> I've removed the >> >>>>>> > >>>>>>> NSK_JNI_VERIFY since >> >>>>>> the JNI wrapper >> >>>>>> > handles the >> >>>>>> > >>>>>>> tracing and the >> >>>>>> verify in almost the same >> >>>>>> > way; only >> >>>>>> > >>>>>>> difference I can >> >>>>>> really tell is that the >> >>>>>> > complain >> >>>>>> > >>>>>>> method from NSK has >> a >> >>>>>> max complain before >> >>>>>> > stopping >> >>>>>> > >>>>>>> to "complain"; I >> have >> >>>>>> not added that part >> >>>>>> > of the >> >>>>>> > >>>>>>> code in this webrev >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Once we decide on >> >>>>>> these, I can continue on the >> >>>>>> > >>>>>>> files from >> >>>>>> JDK-8212884 and then do both the >> >>>>>> > >>>>>>> assignment in an if >> >>>>>> extraction followed-by this >> >>>>>> > >>>>>>> type of webrev in an >> >>>>>> easier fashion. >> >>>>>> > Depending on >> >>>>>> > >>>>>>> decisions here, >> >>>>>> NSK*VERIFY can be deprecated as >> >>>>>> > >>>>>>> well as we go >> forward. >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Thanks! >> >>>>>> > >>>>>>> Jc >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> On Mon, Nov 19, 2018 >> >>>>>> at 11:34 AM Chris Plummer >> >>>>>> > >>>>>>> < >> chris.plummer at oracle.com >> >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>>>>>> > >>>>>> >> >>>>>> > > >>>>>> >>> wrote: >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> On 11/19/18 >> 10:07 >> >>>>>> AM, JC Beyler wrote: >> >>>>>> > >>>>>>>> Hi all, >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> @David/Chris: >> >>>>>> should I then push this >> >>>>>> > RFR to >> >>>>>> > >>>>>>>> the hotspot >> >>>>>> mailing or the runtime >> >>>>>> > one? For >> >>>>>> > >>>>>>>> what it's >> worth, >> >>>>>> a lot of the tests >> >>>>>> > under the >> >>>>>> > >>>>>>>> vmTestbase are >> >>>>>> jvmti so the review also >> >>>>>> > >>>>>>>> affects >> >>>>>> serviceability; it just turns >> >>>>>> > out I >> >>>>>> > >>>>>>>> started with >> the >> >>>>>> GC originally and >> >>>>>> > then hit >> >>>>>> > >>>>>>>> some other >> tests >> >>>>>> I had touched via the >> >>>>>> > >>>>>>>> assignment >> >>>>>> extraction. >> >>>>>> > >>>>>>> I think hotspot >> >>>>>> would be best. >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> Chris >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> @Serguei: Done >> >>>>>> for the method >> >>>>>> > renaming, for >> >>>>>> > >>>>>>>> the indent, are >> >>>>>> you talking about >> >>>>>> > going from >> >>>>>> > >>>>>>>> the 8-indent to >> >>>>>> 4-indent? If so, would >> >>>>>> > it not >> >>>>>> > >>>>>>>> just be better >> >>>>>> to do a new JBS bug and >> >>>>>> > do the >> >>>>>> > >>>>>>>> whole files in >> >>>>>> one go? I ask because >> >>>>>> > >>>>>>>> otherwise, it >> >>>>>> will look a bit weird to >> >>>>>> > have >> >>>>>> > >>>>>>>> parts of the >> >>>>>> file as 8-indent and others >> >>>>>> > >>>>>>>> 4-indent? >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> Thanks for >> >>>>>> looking at it! >> >>>>>> > >>>>>>>> Jc >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> On Mon, Nov 19, >> >>>>>> 2018 at 1:25 AM >> >>>>>> > >>>>>>>> serguei.spitsyn at oracle.com >> >>>>>> >> >>>>>> > >>>>>> > >> >>>>>> > >>>>>>>> > >>>>>> >> >>>>>> > > >>>>>> >> >> >>>>>> > >>>>>>>> > >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>>>>>>> > >>>>>> >> >>>>>> > > >>>>>> >>> wrote: >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> Hi Jc, >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> We have to >> >>>>>> start this review >> >>>>>> > anyway. :) >> >>>>>> > >>>>>>>> It looks >> >>>>>> good to me in general. >> >>>>>> > >>>>>>>> Thank you >> >>>>>> for your consistency in this >> >>>>>> > >>>>>>>> >> refactoring! >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> Some minor >> >>>>>> comments. >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> >> >>>>>> > >> >>>>>> >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> +static >> >>>>>> const char* >> >>>>>> > remove_folders(const >> >>>>>> > >>>>>>>> char* >> >>>>>> fullname) { I'd suggest to >> >>>>>> > rename >> >>>>>> > >>>>>>>> the >> function >> >>>>>> name to something >> >>>>>> > traditional >> >>>>>> > >>>>>>>> like >> >>>>>> get_basename. Otherwise, it >> >>>>>> > sounds >> >>>>>> > >>>>>>>> like this >> >>>>>> function has to really >> >>>>>> > remove >> >>>>>> > >>>>>>>> folders. :) >> >>>>>> Also, all *Locker.cpp have >> >>>>>> > >>>>>>>> wrong >> indent >> >>>>>> in the bodies of if >> >>>>>> > and while >> >>>>>> > >>>>>>>> statements. >> >>>>>> Could this be fixed >> >>>>>> > with the >> >>>>>> > >>>>>>>> >> refactoring? >> >>>>>> I did not look on how >> >>>>>> > this >> >>>>>> > >>>>>>>> impacts the >> >>>>>> tests other than >> >>>>>> > >>>>>>>> serviceability. Thanks, >> >>>>>> Serguei >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> On 11/16/18 >> >>>>>> 19:43, JC Beyler wrote: >> >>>>>> > >>>>>>>>> Hi all, >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Anybody >> >>>>>> motivated to review this? :) >> >>>>>> > >>>>>>>>> Jc >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> On Wed, Nov 7, >> >>>>>> 2018 at 9:53 PM JC >> >>>>>> > Beyler >> >>>>>> > >>>>>>>>> > >>>>>> >> >>>>>> > > >>>>>> > >> >>>>>> > >>>>>>>>> > >>>>>> >> >>>>>> > > >>>>>> >>> wrote: >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Hi all, >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Could I >> have >> >>>>>> a review for the >> >>>>>> > >>>>>>>>> extension >> >>>>>> and usage of the >> >>>>>> > >>>>>>>>> ExceptionJniWrapper. This >> >>>>>> > adds lines >> >>>>>> > >>>>>>>>> and >> >>>>>> filenames to the end of the >> >>>>>> > >>>>>>>>> wrapper JNI >> >>>>>> methods, adds >> >>>>>> > tracing, >> >>>>>> > >>>>>>>>> and throws >> >>>>>> an error if need >> >>>>>> > be. I've >> >>>>>> > >>>>>>>>> ported the >> >>>>>> gc/lock files to >> >>>>>> > use the >> >>>>>> > >>>>>>>>> new >> >>>>>> TRACE_JNI_CALL add-on and >> >>>>>> > I've >> >>>>>> > >>>>>>>>> ported a >> few >> >>>>>> of the tests >> >>>>>> > that were >> >>>>>> > >>>>>>>>> already >> >>>>>> changed for the >> >>>>>> > assignment >> >>>>>> > >>>>>>>>> webrev for >> >>>>>> JDK-8212884. >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Webrev: >> >>>>>> > >>>>>>>>> >> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >> >>>>>> >> >>>>>> > >>>>>>>>> >> >>>>>> >> >>>>>> > >>>>>>>>> Bug: >> >>>>>> > >>>>>>>>> >> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> For >> >>>>>> illustration, if I force >> >>>>>> > an error >> >>>>>> > >>>>>>>>> to the >> >>>>>> AP04/ap04t03 test and >> >>>>>> > set the >> >>>>>> > >>>>>>>>> verbosity >> >>>>>> on, I get something >> >>>>>> > like: >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> >> Calling >> >>>>>> JNI method >> >>>>>> > FindClass from >> >>>>>> > >>>>>>>>> ap04t003.cpp:343 >> >>>>>> > >>>>>>>>> >> Calling >> >>>>>> with these >> >>>>>> > parameter(s): >> >>>>>> > >>>>>>>>> java/lang/Threadd >> >>>>>> > >>>>>>>>> Wait for >> >>>>>> thread to finish >> >>>>>> > >>>>>>>>> << Called >> >>>>>> JNI method >> >>>>>> > FindClass from >> >>>>>> > >>>>>>>>> ap04t003.cpp:343 >> >>>>>> > >>>>>>>>> Exception >> in >> >>>>>> thread "Thread-0" >> >>>>>> > >>>>>>>>> java.lang.NoClassDefFoundError: >> >>>>>> > >>>>>>>>> java/lang/Threadd >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> >>>>>> >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Method) >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Caused by: >> >>>>>> > >>>>>>>>> java.lang.ClassNotFoundException: >> >>>>>> > >>>>>>>>> java.lang.Threadd >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> >>>>>> > >>>>>>>>> ... 3 more >> >>>>>> > >>>>>>>>> FATAL ERROR >> >>>>>> in native method: JNI >> >>>>>> > >>>>>>>>> method >> >>>>>> FindClass : internal error >> >>>>>> > >>>>>>>>> from >> >>>>>> ap04t003.cpp:343 >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> >>>>>> >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Method) >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> at >> >>>>>> > >>>>>>>>> >> >>>>>> > >> >>>>>> >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> >>>>>> >> >>>>>> > >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Questions/comments I have about >> >>>>>> > >>>>>>>>> this are: >> >>>>>> > >>>>>>>>> - Do we >> >>>>>> want to force fatal >> >>>>>> > errors >> >>>>>> > >>>>>>>>> when a JNI >> >>>>>> call fails in general? >> >>>>>> > >>>>>>>>> Most of >> >>>>>> these tests do the right >> >>>>>> > >>>>>>>>> thing and >> >>>>>> test the return of >> >>>>>> > the JNI >> >>>>>> > >>>>>>>>> calls, for >> >>>>>> example: >> >>>>>> > >>>>>>>>> thrClass = >> >>>>>> > >>>>>>>>> jni->FindClass("java/lang/Threadd", >> >>>>>> > >>>>>>>>> TRACE_JNI_CALL); >> >>>>>> > >>>>>>>>> if >> >>>>>> (thrClass == NULL) { >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> but now the >> >>>>>> wrapper actually >> >>>>>> > would do >> >>>>>> > >>>>>>>>> a fatal if >> >>>>>> the FindClass call >> >>>>>> > would >> >>>>>> > >>>>>>>>> return a >> >>>>>> nullptr, so we could >> >>>>>> > remove >> >>>>>> > >>>>>>>>> that test >> >>>>>> altogether. What do >> >>>>>> > you >> >>>>>> > >>>>>>>>> think? >> >>>>>> > >>>>>>>>> - I >> >>>>>> prefer to leave them >> >>>>>> > as the >> >>>>>> > >>>>>>>>> tests then >> >>>>>> become closer to >> >>>>>> > what real >> >>>>>> > >>>>>>>>> users would >> >>>>>> have in their >> >>>>>> > code and is >> >>>>>> > >>>>>>>>> the >> >>>>>> "recommended" way of doing it >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> - The >> >>>>>> alternative is to >> >>>>>> > use the >> >>>>>> > >>>>>>>>> NonFatalError I added >> >>>>>> which >> >>>>>> > then just >> >>>>>> > >>>>>>>>> prints out >> >>>>>> that something >> >>>>>> > went wrong, >> >>>>>> > >>>>>>>>> letting the >> >>>>>> test continue. >> >>>>>> > Question >> >>>>>> > >>>>>>>>> will be >> what >> >>>>>> should be the >> >>>>>> > default? >> >>>>>> > >>>>>>>>> The fatal >> or >> >>>>>> the non-fatal error >> >>>>>> > >>>>>>>>> handling? >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> On a >> >>>>>> different subject: >> >>>>>> > >>>>>>>>> - On the >> >>>>>> new tests, I've >> >>>>>> > removed >> >>>>>> > >>>>>>>>> the >> >>>>>> NSK_JNI_VERIFY since the JNI >> >>>>>> > >>>>>>>>> wrapper >> >>>>>> handles the tracing >> >>>>>> > and the >> >>>>>> > >>>>>>>>> verify in >> >>>>>> almost the same >> >>>>>> > way; only >> >>>>>> > >>>>>>>>> difference >> I >> >>>>>> can really tell >> >>>>>> > is that >> >>>>>> > >>>>>>>>> the >> complain >> >>>>>> method from NSK >> >>>>>> > has a >> >>>>>> > >>>>>>>>> max >> complain >> >>>>>> before stopping to >> >>>>>> > >>>>>>>>> "complain"; >> >>>>>> I have not added that >> >>>>>> > >>>>>>>>> part of the >> >>>>>> code in this webrev >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Once we >> >>>>>> decide on these, I can >> >>>>>> > >>>>>>>>> continue on >> >>>>>> the files from >> >>>>>> > >>>>>>>>> JDK-8212884 >> >>>>>> and then do both the >> >>>>>> > >>>>>>>>> assignment >> >>>>>> in an if extraction >> >>>>>> > >>>>>>>>> followed-by >> >>>>>> this type of >> >>>>>> > webrev in an >> >>>>>> > >>>>>>>>> easier >> >>>>>> fashion. Depending on >> >>>>>> > >>>>>>>>> decisions >> >>>>>> here, NSK*VERIFY can be >> >>>>>> > >>>>>>>>> deprecated >> >>>>>> as well as we go >> >>>>>> > forward. >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> Thank you >> >>>>>> for the >> >>>>>> > reviews/comments :) >> >>>>>> > >>>>>>>>> Jc >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> >> >>>>>> > >>>>>>>>> -- >> >>>>>> > >>>>>>>>> Thanks, >> >>>>>> > >>>>>>>>> Jc >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> >> >>>>>> > >>>>>>>> -- >> >>>>>> > >>>>>>>> Thanks, >> >>>>>> > >>>>>>>> Jc >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> >> >>>>>> > >>>>>>> -- >> >>>>>> > >>>>>>> Thanks, >> >>>>>> > >>>>>>> Jc >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> -- >> >>>>>> > >>>>>> Thanks, >> >>>>>> > >>>>>> Jc >> >>>>>> > >>>>> >> >>>>>> > >>>>> >> >>>>>> > >>>>> >> >>>>>> > >>>>> >> >>>>>> > >>>>> -- >> >>>>>> > >>>>> Thanks, >> >>>>>> > >>>>> Jc >> >>>>>> > >>>> >> >>>>>> > >>> >> >>>>>> > >>> >> >>>>>> > >>> >> >>>>>> > >>> -- >> >>>>>> > >>> Thanks, >> >>>>>> > >>> Jc >> >>>>>> > >>> >> >>>>>> > >>> >> >>>>>> > >>> >> >>>>>> > >>> -- >> >>>>>> > >>> >> >>>>>> > >>> Thanks, >> >>>>>> > >>> Jc >> >>>>>> > > >> >>>>>> > > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > -- >> >>>>>> > >> >>>>>> > Thanks, >> >>>>>> > Jc >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> -- >> >>>>>> Thanks, >> >>>>>> Jc >> >>>>> >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> Thanks, >> >>>>> Jc >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Thanks, >> >>>> Jc >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> >> >>>> Thanks, >> >>>> Jc >> >>> >> > >> > > > -- > > Thanks, > Jc > -- Thanks, Jc From erik.osterlund at oracle.com Mon Dec 17 16:56:55 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 17 Dec 2018 17:56:55 +0100 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading Message-ID: Hi, Today, when converting a not_entrant nmethod to zombie, the sweeper first clears the IC stubs of that nmethod, and then make_zombie(). With concurrent class unloading, this ordering is a bit unfortunate. Between clearing the IC stubs and making it zombie, the concurrent GC threads may hit a timing window where they will create IC stubs for concurrently cleaning the ICs of this still is_alive() nmethod. The result is that during safepoint cleanup, when we finalize the IC stubs, we find one associated with a zombie. The unregistering of the nmethod from the GC will block during concurrent nmethod unloading, so clearing the IC stubs after the nmethod has become zombie is a lot more sane as there can not be any such races then. Bug: https://bugs.openjdk.java.net/browse/JDK-8215491 Webrev: http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html Thanks, /Erik From jcbeyler at google.com Mon Dec 17 17:25:41 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 17 Dec 2018 09:25:41 -0800 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: Message-ID: Hi Matthias, Your webrev looks good to me (not a reviewer though ;-)). I saw a few things about the isCopy updates but that is out of scope of this webrev, I'll be creating a separate one in a bit :-) Thanks, Jc On Mon, Dec 17, 2018 at 1:00 AM Baesken, Matthias wrote: > Hello, please review the following change. > I noticed that we miss at some places (for example in case of early > returns) > where GetByteArrayElements is used, the corresponding > ReleaseByteArrayElements call. > > In VirtualMachineImpl.c I also removed a check for isCopy (is the > returned byte array a copy ?) > because from what I read at > > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html > > the ReleaseByteArrayElements routine should always be called. > > > bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8215411 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > > > Thanks, Matthias > -- Thanks, Jc From per.liden at oracle.com Mon Dec 17 18:02:10 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 17 Dec 2018 19:02:10 +0100 Subject: RFR: 8215492: Rename INTERNAL_EMPTY to something less "internal" Message-ID: The INTERNAL_EMPTY macro is sometimes useful outside of access*.[ch]pp. We should change the name of that macro and allow it to be used, for example, in JNIHandle code, which today passes a "0". I propose that we change it to DECORATORS_NONE, but I'm open to other suggestions. This patch also adjusts the JNI Handle code to make use of this macros instead of using "0". Bug: https://bugs.openjdk.java.net/browse/JDK-8215492 Webrev: http://cr.openjdk.java.net/~pliden/8215492/webrev.0 (Note! This patch is based on JDK-8215451, which has not yet propagated from jdk/jdk12 to jdk/jdk) /Per From coleen.phillimore at oracle.com Mon Dec 17 18:16:48 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 17 Dec 2018 13:16:48 -0500 Subject: RFR: 8215492: Rename INTERNAL_EMPTY to something less "internal" In-Reply-To: References: Message-ID: http://cr.openjdk.java.net/~pliden/8215492/webrev.0/src/hotspot/share/runtime/jniHandles.inline.hpp.udiff.html I like this a lot better than 0. thanks, Coleen On 12/17/18 1:02 PM, Per Liden wrote: > The INTERNAL_EMPTY macro is sometimes useful outside of > access*.[ch]pp. We should change the name of that macro and allow it > to be used, for example, in JNIHandle code, which today passes a "0". > > I propose that we change it to DECORATORS_NONE, but I'm open to other > suggestions. > > This patch also adjusts the JNI Handle code to make use of this macros > instead of using "0". > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215492 > Webrev: http://cr.openjdk.java.net/~pliden/8215492/webrev.0 > > (Note! This patch is based on JDK-8215451, which has not yet > propagated from jdk/jdk12 to jdk/jdk) > > /Per From dean.long at oracle.com Mon Dec 17 18:52:11 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 17 Dec 2018 10:52:11 -0800 Subject: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java fails on ARM32 In-Reply-To: <4f6d2937-e60c-78c8-844c-b54d0205e719@arm.com> References: <2f122afe-2d14-316d-63c4-df8113fa76d0@oracle.com> <98be78d0-d49d-cf4a-fda8-34cd33d5c813@oracle.com> <4f6d2937-e60c-78c8-844c-b54d0205e719@arm.com> Message-ID: This has been pushed now: http://hg.openjdk.java.net/jdk/jdk12/rev/5da72d7e0e80 dl On 12/17/18 12:26 AM, Nick Gasson (Arm Technology China) wrote: > Hi Dean, > >> Would you like to add comments as suggested by Boris? I think >> it's a good idea. > Sure, I've updated the webrev here: > > http://cr.openjdk.java.net/~njian/8214512/webrev.1/ > > Thanks, > Nick > > > On 17/12/2018 12:59, dean.long at oracle.com wrote: >> Thanks.? Would you like to add comments as suggested by Boris?? I think >> it's a good idea. >> >> dl >> >> On 12/16/18 6:00 PM, Nick Gasson (Arm Technology China) wrote: >>> Hi Dean, >>> >>> Sorry for the delay. This is submitted on behalf of ARM Ltd. >>> >>> Thanks, >>> >>> Nick >>> >>> *From:* dean.long at oracle.com >>> *Sent:* 13 December 2018 12:01 >>> *To:* Nick Gasson (Arm Technology China) ; >>> hotspot-dev at openjdk.java.net; Boris Ulasevich >>> >>> *Cc:* hotspot-compiler-dev at openjdk.java.net compiler >>> ; nd ; >>> aarch32-port-dev at openjdk.java.net >>> *Subject:* Re: RFR: 8214512: Jtreg test compiler/c2/Test8062950.java >>> fails on ARM32 >>> >>> Nick, can you please confirm that you are covered under the >>> >>> ? ARM Ltd. - OpenJDK, MySQL >>> >>> >>> OCA agreement?? I'm guessing Arm Technology China is under ARM Ltd., >>> judging >>> by your email domain, but please confirm. >>> >>> dl >>> >>> On 12/11/18 5:30 PM, dean.long at oracle.com >>> wrote: >>> >>> I can help you commit it, but first I think another review would >>> be good. >>> >>> dl >>> >>> On 12/11/18 5:15 PM, Nick Gasson (Arm Technology China) wrote: >>> >>> Hi Nick.? This change looks good to me. >>> >>> Thanks Dean. Could you help me to commit this patch if it >>> doesn't need other reviews? >>> >>> Nick >>> >>> >>> >>> -----Original Message----- >>> From: dean.long at oracle.com >>> >>> Sent: 12 December 2018 06:05 >>> To: Nick Gasson (Arm Technology China) >>> ; hotspot- >>> dev at openjdk.java.net ; Boris >>> Ulasevich >>> >>> Cc: nd ; >>> hotspot-compiler-dev at openjdk.java.net >>> compiler >>> >>> ; aarch32-port- >>> dev at openjdk.java.net >>> Subject: Re: RFR: 8214512: Jtreg test >>> compiler/c2/Test8062950.java fails on >>> ARM32 >>> >>> Hi Nick.? This change looks good to me. >>> >>> dl >>> >>> On 12/2/18 10:06 PM, Nick Gasson wrote: >>> >>> Hi, >>> >>> Could someone please help me review this fix for a >>> test failure on ARM32: >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214512 >>> Webrev: >>> http://cr.openjdk.java.net/~njian/8214512/webrev.0/ >>> >>> This fixes two assertion failures related to biased >>> locking when C2's >>> bias inlining feature is disabled >>> (-XX:-OptoBiasInlining): >>> >>> MacroAssembler::atomic_cas_bool takes three register >>> arguments plus a >>> temporary register for use in the CAS loop. If the >>> caller passes >>> `noreg' as this temporary register (which happens in the >>> !OptoBiasInlining call path from >>> MacroAssembler::fast_lock) then >>> MacroAssembler::atomic_cas_bool will default to using >>> LR as a >>> temporary. But it's possible with C2 that LR is one of >>> the other three >>> arguments which then triggers an >>> assert_different_registers assertion >>> failure. Fix this by supplying an additional scratch >>> register to >>> MacroAssembler::fast_lock if !OptoBiasInlining. >>> >>> In the !OptoBiasInlining case >>> MacroAssembler::fast_lock calls >>> MacroAssembler::biased_locking_enter to handle the >>> case where the mark >>> word contains the biased lock pattern which fast_lock >>> wouldn't >>> otherwise see if OptoBiasInlining was true. However in >>> the case that >>> we fail to acquire the biased lock this falls through >>> to label >>> `failed' and runs the normal fast_lock code that >>> implicitly assumes >>> the mark word does not have the bias pattern. This >>> means we can end up >>> with a BasicLock where _displaced_header contains the >>> biased lock >>> pattern which is an illegal state and subsequently >>> triggers an >>> assertion failure in ObjectSynchronizer::fast_exit. >>> >>> The right thing to do here is branch to `done' >>> whenever the bias lock >>> pattern is present and let the runtime handle the >>> failure case.? Also >>> edited the comment describing >>> MacroAssembler::biased_locking_enter to >>> more accurately describe what it does (the comment >>> about `slow_case' >>> being optional is wrong). >>> >>> Jtreg test compiler/c2/Test8062950.java now passes on >>> ARM32 with this >>> patch. >>> >>> Thanks, >>> Nick >>> From kim.barrett at oracle.com Mon Dec 17 18:55:10 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 17 Dec 2018 13:55:10 -0500 Subject: RFR: 8215492: Rename INTERNAL_EMPTY to something less "internal" In-Reply-To: References: Message-ID: <149C7392-368F-4867-8AF5-E82CED44AE92@oracle.com> > On Dec 17, 2018, at 1:02 PM, Per Liden wrote: > > The INTERNAL_EMPTY macro is sometimes useful outside of access*.[ch]pp. We should change the name of that macro and allow it to be used, for example, in JNIHandle code, which today passes a "0". > > I propose that we change it to DECORATORS_NONE, but I'm open to other suggestions. > > This patch also adjusts the JNI Handle code to make use of this macros instead of using "0". > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215492 > Webrev: http://cr.openjdk.java.net/~pliden/8215492/webrev.0 > > (Note! This patch is based on JDK-8215451, which has not yet propagated from jdk/jdk12 to jdk/jdk) > > /Per Looks good. DECORATORS_NONE seems like a good name to me. Thanks for following up on this. From dean.long at oracle.com Mon Dec 17 19:27:12 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 17 Dec 2018 11:27:12 -0800 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: References: Message-ID: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> On 12/17/18 8:56 AM, Erik ?sterlund wrote: > Hi, > > Today, when converting a not_entrant nmethod to zombie, the sweeper > first clears the IC stubs of that nmethod, and then make_zombie(). > With concurrent class unloading, this ordering is a bit unfortunate. > Between clearing the IC stubs and making it zombie, the concurrent GC > threads may hit a timing window where they will create IC stubs for > concurrently cleaning the ICs of this still is_alive() nmethod. Do we normally have to creates stubs to clean an IC? > The result is that during safepoint cleanup, when we finalize the IC > stubs, we find one associated with a zombie. > > The unregistering of the nmethod from the GC will block during > concurrent nmethod unloading, so clearing the IC stubs after the > nmethod has become zombie is a lot more sane as there can not be any > such races then. > Where is the unregistering happening? Doing the clearing after it's a zombie does sound safer. dl > Bug: > https://bugs.openjdk.java.net/browse/JDK-8215491 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html > > Thanks, > /Erik From erik.osterlund at oracle.com Mon Dec 17 19:57:44 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 17 Dec 2018 20:57:44 +0100 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> References: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> Message-ID: <30496bb1-009c-0259-c039-5511834083e6@oracle.com> Hi Dean, On 2018-12-17 20:27, dean.long at oracle.com wrote: > On 12/17/18 8:56 AM, Erik ?sterlund wrote: >> Hi, >> >> Today, when converting a not_entrant nmethod to zombie, the sweeper >> first clears the IC stubs of that nmethod, and then make_zombie(). >> With concurrent class unloading, this ordering is a bit unfortunate. >> Between clearing the IC stubs and making it zombie, the concurrent GC >> threads may hit a timing window where they will create IC stubs for >> concurrently cleaning the ICs of this still is_alive() nmethod. > > Do we normally have to creates stubs to clean an IC? Yes if you clean an IC in an nmethod that is_alive(), we currently have to do that using IC stubs. I have some ideas going forward for how we can stop doing that, but that is outside of scope for now. > >> The result is that during safepoint cleanup, when we finalize the IC >> stubs, we find one associated with a zombie. >> >> The unregistering of the nmethod from the GC will block during >> concurrent nmethod unloading, so clearing the IC stubs after the >> nmethod has become zombie is a lot more sane as there can not be any >> such races then. >> > > Where is the unregistering happening? Inside of make_zombie() we unregister_nmethod() on CollectedHeap. In there, ZGC makes sure we wait until concurrent nmethod unloading is over before continuing. > > Doing the clearing after it's a zombie does sound safer. Agreed! Thanks for reviewing. /Erik > dl > >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8215491 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html >> >> Thanks, >> /Erik > From ecki at zusammenkunft.net Mon Dec 17 20:42:18 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 17 Dec 2018 21:42:18 +0100 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrongvalue after JDK-8212605 In-Reply-To: <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> Message-ID: <5c180a2a.1c69fb81.c57e.5ba0@mx.google.com> In a related matter, are the existing tests reliable to detect the Situation (at least for the Default runtime/compiler behavior). i.e. are the testcases covering stack Evaluation in a compiled context where EA would elimiiminate it? Gruss Bernd -- http://bernd.eckenfels.net Von: dean.long at oracle.com Gesendet: Montag, 17. Dezember 2018 05:56 An: Claes Redestad; security-dev at openjdk.java.net; hotspot-dev developers Betreff: Re: 12 RFR(M) 8214583: AccessController.getContext may return wrongvalue after JDK-8212605 Unfortunately, I don't think @DontInline on an empty method is sufficient here.? If other code is relying on @DontInline for the same purpose then we might need to reexamine that code.? My understanding from discussing with other compiler engineers is that using a native method is the safest technique that the compilers can't see through.? The problem with @DontInline is that C2 looks at the bytecodes of the target method, even if it isn't inlined (see BCEscapeAnalyzer and the EstimateArgEscape flag). There may be a way to make it work, but that would require more investigation, and I'm not sure the benefit outweighs the risk. dl On 12/15/18 6:48 AM, Claes Redestad wrote: > Hi Dean, > > to avoid escape analysis-eliminated allocations in the past @DontInline > has been sufficient. This means a simpler patch (no changes to native > code needed - added assertions notwithstanding) and passes your tests > with C2 (it'd concern me if Graal's EA sees through this trick, as it > might break some existing places where DontInline is used to this > effect): > > ??? /** > ???? * The value needs to be physically located in the frame, so that it > ???? * can be found by a stack walk. > ???? */ > ??? @Hidden > ??? @DontInline > ??? private static void ensureMaterializedForStackWalk(Object o) {} > > Thanks! > > /Claes > > On 2018-12-15 01:59, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8214583 >> http://cr.openjdk.java.net/~dlong/8214583/webrev >> >> This change includes two new regression test that demonstrate the >> problem, and a fix that allows the tests >> to pass. >> >> The problem happens when the JIT compiler's escape analysis >> eliminates the allocation of the AccessControlContext object passed >> to doPrivileged.? The compiler thinks this is safe because it does >> not see that the object "escapes".? However, getContext needs to be >> able to find the object using a stack walk, so we need a way to tell >> the compiler that it does indeed escape.? To do this we pass the >> value to a native method that does nothing. >> >> Microbenchmark results: >> >> jdk12-b18: >> >> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >> >> >> jdk12-b19: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >> >> this fix: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >> >> >> dl From dean.long at oracle.com Mon Dec 17 21:11:35 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 17 Dec 2018 13:11:35 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrongvalue after JDK-8212605 In-Reply-To: <5c180a2a.1c69fb81.c57e.5ba0@mx.google.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> <5c180a2a.1c69fb81.c57e.5ba0@mx.google.com> Message-ID: <3576b118-d088-89ae-825b-9531eb437bac@oracle.com> I did not find any existing tests that covered this, which is why I added two new tests to specifically test for this. dl On 12/17/18 12:42 PM, Bernd Eckenfels wrote: > In a related matter, are the existing tests reliable to detect the Situation (at least for the Default runtime/compiler behavior). i.e. are the testcases covering stack Evaluation in a compiled context where EA would elimiiminate it? > > Gruss > Bernd From per.liden at oracle.com Mon Dec 17 21:22:56 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 17 Dec 2018 22:22:56 +0100 Subject: RFR: 8215492: Rename INTERNAL_EMPTY to something less "internal" In-Reply-To: <149C7392-368F-4867-8AF5-E82CED44AE92@oracle.com> References: <149C7392-368F-4867-8AF5-E82CED44AE92@oracle.com> Message-ID: <53042abe-a9c0-5ac3-5b08-b8d86bce8e9d@oracle.com> Thanks for reviewing, Kim and Coleen! /Per On 12/17/2018 07:55 PM, Kim Barrett wrote: >> On Dec 17, 2018, at 1:02 PM, Per Liden wrote: >> >> The INTERNAL_EMPTY macro is sometimes useful outside of access*.[ch]pp. We should change the name of that macro and allow it to be used, for example, in JNIHandle code, which today passes a "0". >> >> I propose that we change it to DECORATORS_NONE, but I'm open to other suggestions. >> >> This patch also adjusts the JNI Handle code to make use of this macros instead of using "0". >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8215492 >> Webrev: http://cr.openjdk.java.net/~pliden/8215492/webrev.0 >> >> (Note! This patch is based on JDK-8215451, which has not yet propagated from jdk/jdk12 to jdk/jdk) >> >> /Per > > Looks good. DECORATORS_NONE seems like a good name to me. > > Thanks for following up on this. > From coleen.phillimore at oracle.com Mon Dec 17 21:31:38 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 17 Dec 2018 16:31:38 -0500 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: <30496bb1-009c-0259-c039-5511834083e6@oracle.com> References: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> <30496bb1-009c-0259-c039-5511834083e6@oracle.com> Message-ID: <11826825-4c2e-bc8f-5d4c-1765b7fa2838@oracle.com> On 12/17/18 2:57 PM, Erik ?sterlund wrote: > Hi Dean, > > On 2018-12-17 20:27, dean.long at oracle.com wrote: >> On 12/17/18 8:56 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> Today, when converting a not_entrant nmethod to zombie, the sweeper >>> first clears the IC stubs of that nmethod, and then make_zombie(). >>> With concurrent class unloading, this ordering is a bit unfortunate. >>> Between clearing the IC stubs and making it zombie, the concurrent >>> GC threads may hit a timing window where they will create IC stubs >>> for concurrently cleaning the ICs of this still is_alive() nmethod. >> >> Do we normally have to creates stubs to clean an IC? > > Yes if you clean an IC in an nmethod that is_alive(), we currently > have to do that using IC stubs. I have some ideas going forward for > how we can stop doing that, but that is outside of scope for now. > >> >>> The result is that during safepoint cleanup, when we finalize the IC >>> stubs, we find one associated with a zombie. >>> >>> The unregistering of the nmethod from the GC will block during >>> concurrent nmethod unloading, so clearing the IC stubs after the >>> nmethod has become zombie is a lot more sane as there can not be any >>> such races then. >>> >> >> Where is the unregistering happening? > > Inside of make_zombie() we unregister_nmethod() on CollectedHeap. In > there, ZGC makes sure we wait until concurrent nmethod unloading is > over before continuing. Can you move clean_ic_stubs() inside of make_not_entrant_or_zombie() to maintain the order dependence? thanks, Coleen > >> >> Doing the clearing after it's a zombie does sound safer. > > Agreed! > > Thanks for reviewing. > > /Erik > >> dl >> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8215491 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html >>> >>> Thanks, >>> /Erik >> From david.holmes at oracle.com Mon Dec 17 22:25:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 08:25:01 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: Message-ID: Hi Matthias, On 17/12/2018 6:59 pm, Baesken, Matthias wrote: > Hello, please review the following change. > I noticed that we miss at some places (for example in case of early returns) > where GetByteArrayElements is used, the corresponding ReleaseByteArrayElements call. > > In VirtualMachineImpl.c I also removed a check for isCopy (is the returned byte array a copy ?) > because from what I read at > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html > > the ReleaseByteArrayElements routine should always be called. That's not clear to me. My reading is that you only need to release if you have changes you need to ensure are written back and that is only needed if a copy was made. That said, the change seem semantically correct and harmless in this case. --- src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. Also you change seem wrong to me because it will commit the changes in the buffer even if we "abort" here: 735 if (bytesRead != numBytes) { 736 return 0; 737 } so it seems to me if you really want to release on all paths then the error paths should use a mode of JNI_ABORT and only the success path should use mode 0. --- src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m This change seems okay, though again potentially not necessary - as we never commit any changes. Thanks, David ----- > > bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8215411 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > > > Thanks, Matthias > From dean.long at oracle.com Mon Dec 17 22:40:58 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 17 Dec 2018 14:40:58 -0800 Subject: 12 RFR(S) 8215205: javaVFrame much slower than vframeStream Message-ID: https://bugs.openjdk.java.net/browse/JDK-8215205 http://cr.openjdk.java.net/~dlong/8215205/webrev This change allows a stackwalk to continue using the faster vframeStream iterator, but changing to a javaVFrame lazily as needed.? It introduces a new API javaVFrame* vframeStreamCommon::asJavaVFrame() The first user of this API will be JDK-8214329, which is why this issue is a P3. dl From dean.long at oracle.com Mon Dec 17 22:49:54 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 17 Dec 2018 14:49:54 -0800 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client Message-ID: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8214329 http://cr.openjdk.java.net/~dlong/8214329/webrev/ In "8212605: Pure-Java implementation of AccessController.doPrivileged", the stackwalk in JVM_GetStackAccessControlContext was changed from using a vframeStream to using a javaVFrame, so that it could access locals. However this caused a regression because javaVFrame has more overhead.?? JDK-8215205 fixes the problem by allowing the use of javaVFrame only when needed. This change reverts JVM_GetStackAccessControlContext back to using vframeStream, and uses the new API in JDK-8215205 to get a javaVFrame lazily.? It also removes two unnecessary calls to java_sender(). dl From mandy.chung at oracle.com Mon Dec 17 22:55:30 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 17 Dec 2018 14:55:30 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> Message-ID: <1dd75b6f-35d7-98be-2fde-10a57a6a816b@oracle.com> This looks okay to me. Mandy On 12/14/18 4:59 PM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8214583 > http://cr.openjdk.java.net/~dlong/8214583/webrev > > This change includes two new regression test that demonstrate the > problem, and a fix that allows the tests > to pass. > > The problem happens when the JIT compiler's escape analysis eliminates > the allocation of the AccessControlContext object passed to > doPrivileged.? The compiler thinks this is safe because it does not > see that the object "escapes".? However, getContext needs to be able > to find the object using a stack walk, so we need a way to tell the > compiler that it does indeed escape.? To do this we pass the value to > a native method that does nothing. > > Microbenchmark results: > > jdk12-b18: > > Benchmark??????????????? Mode? Cnt??? Score?? Error? Units > DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op > DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op > > > jdk12-b19: > > Benchmark??????????????? Mode? Cnt? Score??? Error? Units > DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op > DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op > > this fix: > > Benchmark??????????????? Mode? Cnt? Score??? Error? Units > DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op > DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op > > > dl From mandy.chung at oracle.com Mon Dec 17 23:01:58 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 17 Dec 2018 15:01:58 -0800 Subject: 12 RFR(S) 8215205: javaVFrame much slower than vframeStream In-Reply-To: References: Message-ID: The change looks good. It seems useful to add a microbenchmark in getting ACC so that we can proactively compare the performance for relevant stackwalk change.? With JEP 230, the microbenchmarks are in the jdk test/micro directory and it's very handy to run microbenchmarks.? I'm not sure if one already exists; if not, it's good to follow up with a new JBS and add one. Mandy [1] http://openjdk.java.net/jeps/230 On 12/17/18 2:40 PM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8215205 > http://cr.openjdk.java.net/~dlong/8215205/webrev > > This change allows a stackwalk to continue using the faster > vframeStream iterator, but changing to a javaVFrame lazily as needed.? > It introduces a new API > > ? javaVFrame* vframeStreamCommon::asJavaVFrame() > > > The first user of this API will be JDK-8214329, which is why this > issue is a P3. > > dl From dean.long at oracle.com Mon Dec 17 23:30:40 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 17 Dec 2018 15:30:40 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <1dd75b6f-35d7-98be-2fde-10a57a6a816b@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <1dd75b6f-35d7-98be-2fde-10a57a6a816b@oracle.com> Message-ID: <7f090392-e6aa-dfcb-47d3-92e27fe4cb76@oracle.com> Thanks Mandy. dl On 12/17/18 2:55 PM, Mandy Chung wrote: > This looks okay to me. > > Mandy > > On 12/14/18 4:59 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8214583 >> http://cr.openjdk.java.net/~dlong/8214583/webrev >> >> This change includes two new regression test that demonstrate the >> problem, and a fix that allows the tests >> to pass. >> >> The problem happens when the JIT compiler's escape analysis >> eliminates the allocation of the AccessControlContext object passed >> to doPrivileged.? The compiler thinks this is safe because it does >> not see that the object "escapes".? However, getContext needs to be >> able to find the object using a stack walk, so we need a way to tell >> the compiler that it does indeed escape.? To do this we pass the >> value to a native method that does nothing. >> >> Microbenchmark results: >> >> jdk12-b18: >> >> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >> >> >> jdk12-b19: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >> >> this fix: >> >> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >> >> >> dl > From dean.long at oracle.com Mon Dec 17 23:36:57 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 17 Dec 2018 15:36:57 -0800 Subject: 12 RFR(S) 8215205: javaVFrame much slower than vframeStream In-Reply-To: References: Message-ID: <6f2215e3-4d28-a869-31d0-d2090a60c9ca@oracle.com> On 12/17/18 3:01 PM, Mandy Chung wrote: > The change looks good. > Thanks for the review. > It seems useful to add a microbenchmark in getting ACC so that we can > proactively compare the performance for relevant stackwalk change.? > With JEP 230, the microbenchmarks are in the jdk test/micro directory > and it's very handy to run microbenchmarks. I'm not sure if one > already exists; if not, it's good to follow up with a new JBS and add one. > Sure, I'll do that. dl > Mandy > [1] http://openjdk.java.net/jeps/230 > > On 12/17/18 2:40 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8215205 >> http://cr.openjdk.java.net/~dlong/8215205/webrev >> >> This change allows a stackwalk to continue using the faster >> vframeStream iterator, but changing to a javaVFrame lazily as >> needed.? It introduces a new API >> >> ? javaVFrame* vframeStreamCommon::asJavaVFrame() >> >> >> The first user of this API will be JDK-8214329, which is why this >> issue is a P3. >> >> dl > From david.holmes at oracle.com Tue Dec 18 00:19:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 10:19:51 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: Message-ID: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> Correction ... On 18/12/2018 8:25 am, David Holmes wrote: > Hi Matthias, > > On 17/12/2018 6:59 pm, Baesken, Matthias wrote: >> Hello, please review the following change. >> I noticed that we miss at some places (for example in case of early >> returns) >> where GetByteArrayElements is used,? the corresponding >> ReleaseByteArrayElements? call. >> >> In VirtualMachineImpl.c? I also removed a check for isCopy (is the >> returned byte array a copy ?) >> because from what I read at >> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html >> >> >> the ReleaseByteArrayElements? routine should always be called. > > That's not clear to me. My reading is that you only need to release if > you have changes you need to ensure are written back and that is only > needed if a copy was made. Jc pointed out to me that if a copy is made you may need to release to avoid a memory leak. But that is where the mode flags come in - and again only relevant if a copy was made. But as per: https://developer.android.com/training/articles/perf-jni if a copy is not made and pinning is used (as with the "critical" variants) then you also need to release to un-pin. So code should call release, with an appropriate mode, based on whether isCopy was true**, and whether changed data should be written back. ** mode is ignored if not working on a copy so you can just set it assuming a copy was made. Note that the hotspot implementation always makes a copy for GextXXXArrayElements, and the ReleaseXXXArrayElements knows this and so will always free the buffer that is passed in (other than for mode JNI_COMMIT of course). Cheers, David ----- > That said, the change seem semantically correct and harmless in this case. > --- > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and > THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. > > Also you change seem wrong to me because it will commit the changes in > the buffer even if we "abort" here: > > 735?? if (bytesRead != numBytes) { > ?736????? return 0; > ?737?? } > > so it seems to me if you really want to release on all paths then the > error paths should use a mode of JNI_ABORT and only the success path > should use mode 0. > > --- > > ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > > This change seems okay, though again potentially not necessary - as we > never commit any changes. > > Thanks, > David > ----- > >> >> bug/webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8215411 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ >> >> >> Thanks, Matthias >> From matthias.baesken at sap.com Tue Dec 18 08:52:46 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Dec 2018 08:52:46 +0000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> Message-ID: Hi JC / David, thanks for the input . I'm not really sure what you want me to change David, am I right that I can keep the changes to src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c but adjust src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp to also handle theis potential early return 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) env->GetLongField(obj, 724 ptrIDebugDataSpaces_ID); 725 CHECK_EXCEPTION_(0); ? > > Also you change seem wrong to me because it will commit the changes in > > the buffer even if we "abort" here: > > > > 735 if (bytesRead != numBytes) { > > 736 return 0; > > 737 } > > The spec says : https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html >ReleaseArrayElements Routines >void ReleaseArrayElements(JNIEnv *env, ArrayType array, NativeType *elems, jint mode); > >A family of functions that informs the VM that the native code no longer needs access to elems. The elems argument is a pointer derived from array using the corresponding > GetArrayElements() function. If necessary, this function copies back all changes made to elems to the original array. So shouldn't I tell the VM , that the native code no longer needs access to bytePtr before returning here 735 if (bytesRead != numBytes) { 736 return 0; 737 } ? Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Dienstag, 18. Dezember 2018 01:20 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' ; serviceability- > dev at openjdk.java.net; security-dev at openjdk.java.net > Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss > corresponding Release > > Correction ... > > On 18/12/2018 8:25 am, David Holmes wrote: > > Hi Matthias, > > > > On 17/12/2018 6:59 pm, Baesken, Matthias wrote: > >> Hello, please review the following change. > >> I noticed that we miss at some places (for example in case of early > >> returns) > >> where GetByteArrayElements is used,? the corresponding > >> ReleaseByteArrayElements? call. > >> > >> In VirtualMachineImpl.c? I also removed a check for isCopy (is the > >> returned byte array a copy ?) > >> because from what I read at > >> > >> > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function > s.html > >> > >> > >> the ReleaseByteArrayElements? routine should always be called. > > > > That's not clear to me. My reading is that you only need to release if > > you have changes you need to ensure are written back and that is only > > needed if a copy was made. > > Jc pointed out to me that if a copy is made you may need to release to > avoid a memory leak. But that is where the mode flags come in - and > again only relevant if a copy was made. > > But as per: > > https://developer.android.com/training/articles/perf-jni > > if a copy is not made and pinning is used (as with the "critical" > variants) then you also need to release to un-pin. > > So code should call release, with an appropriate mode, based on whether > isCopy was true**, and whether changed data should be written back. > > ** mode is ignored if not working on a copy so you can just set it > assuming a copy was made. > > Note that the hotspot implementation always makes a copy for > GextXXXArrayElements, and the ReleaseXXXArrayElements knows this and > so > will always free the buffer that is passed in (other than for mode > JNI_COMMIT of course). > > Cheers, > David > ----- > > > That said, the change seem semantically correct and harmless in this case. > > --- > > > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > > > AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and > > THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. > > > > Also you change seem wrong to me because it will commit the changes in > > the buffer even if we "abort" here: > > > > 735?? if (bytesRead != numBytes) { > > ?736????? return 0; > > ?737?? } > > > > so it seems to me if you really want to release on all paths then the > > error paths should use a mode of JNI_ABORT and only the success path > > should use mode 0. > > > > --- > > > > ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > > > > This change seems okay, though again potentially not necessary - as we > > never commit any changes. > > > > Thanks, > > David > > ----- > > > >> > >> bug/webrev : > >> > >> https://bugs.openjdk.java.net/browse/JDK-8215411 > >> > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > >> > >> > >> Thanks, Matthias > >> From david.holmes at oracle.com Tue Dec 18 09:03:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 19:03:48 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> Message-ID: <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> Hi Matthias, On 18/12/2018 6:52 pm, Baesken, Matthias wrote: > Hi JC / David, thanks for the input . > > I'm not really sure what you want me to change David, am I right that I can keep the changes to > > src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c Yes. > but adjust > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > to also handle theis potential early return > > 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) env->GetLongField(obj, > 724 ptrIDebugDataSpaces_ID); > 725 CHECK_EXCEPTION_(0); > > ? And I think: 730 THROW_NEW_DEBUGGER_EXCEPTION_("Windbg Error: ReadVirtual failed!", 0); as I assume that won't actually return normally. >>> Also you change seem wrong to me because it will commit the changes in >>> the buffer even if we "abort" here: >>> >>> 735 if (bytesRead != numBytes) { >>> 736 return 0; >>> 737 } >>> > > The spec says : > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html > >> ReleaseArrayElements Routines >> void ReleaseArrayElements(JNIEnv *env, ArrayType array, NativeType *elems, jint mode); >> >> A family of functions that informs the VM that the native code no longer needs access to elems. The elems argument is a pointer derived from array using the corresponding >> GetArrayElements() function. If necessary, this function copies back all changes made to elems to the original array. > > So shouldn't I tell the VM , that the native code no longer needs access to bytePtr before returning here > > 735 if (bytesRead != numBytes) { > 736 return 0; > 737 } > > ? There are two aspects to release: 1. the actual release (or "I don't need this any more") 2. committing any changes made back to the original array This code does: 728 if (ptrIDebugDataSpaces->ReadVirtual((ULONG64) address, (PVOID) bytePtr, 729 (ULONG)numBytes, &bytesRead) != S_OK) { which writes into the array. It then checks if we wrote what we expected: 735 if (bytesRead != numBytes) { 736 return 0; 737 } If we didn't read what was expected what should happen to the original array? Should it be left untouched or updated with the unexpected input? I would say left untouched and that is what the original code did. If everything succeeds you should do the release with mode 0; but if taking an error exit the release should use mode JNI_ABORT so no changes are written back. Cheers, David > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Dienstag, 18. Dezember 2018 01:20 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' ; serviceability- >> dev at openjdk.java.net; security-dev at openjdk.java.net >> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss >> corresponding Release >> >> Correction ... >> >> On 18/12/2018 8:25 am, David Holmes wrote: >>> Hi Matthias, >>> >>> On 17/12/2018 6:59 pm, Baesken, Matthias wrote: >>>> Hello, please review the following change. >>>> I noticed that we miss at some places (for example in case of early >>>> returns) >>>> where GetByteArrayElements is used,? the corresponding >>>> ReleaseByteArrayElements? call. >>>> >>>> In VirtualMachineImpl.c? I also removed a check for isCopy (is the >>>> returned byte array a copy ?) >>>> because from what I read at >>>> >>>> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function >> s.html >>>> >>>> >>>> the ReleaseByteArrayElements? routine should always be called. >>> >>> That's not clear to me. My reading is that you only need to release if >>> you have changes you need to ensure are written back and that is only >>> needed if a copy was made. >> >> Jc pointed out to me that if a copy is made you may need to release to >> avoid a memory leak. But that is where the mode flags come in - and >> again only relevant if a copy was made. >> >> But as per: >> >> https://developer.android.com/training/articles/perf-jni >> >> if a copy is not made and pinning is used (as with the "critical" >> variants) then you also need to release to un-pin. >> >> So code should call release, with an appropriate mode, based on whether >> isCopy was true**, and whether changed data should be written back. >> >> ** mode is ignored if not working on a copy so you can just set it >> assuming a copy was made. >> >> Note that the hotspot implementation always makes a copy for >> GextXXXArrayElements, and the ReleaseXXXArrayElements knows this and >> so >> will always free the buffer that is passed in (other than for mode >> JNI_COMMIT of course). >> >> Cheers, >> David >> ----- >> >>> That said, the change seem semantically correct and harmless in this case. >>> --- >>> >>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>> >>> AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and >>> THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. >>> >>> Also you change seem wrong to me because it will commit the changes in >>> the buffer even if we "abort" here: >>> >>> 735?? if (bytesRead != numBytes) { >>> ?736????? return 0; >>> ?737?? } >>> >>> so it seems to me if you really want to release on all paths then the >>> error paths should use a mode of JNI_ABORT and only the success path >>> should use mode 0. >>> >>> --- >>> >>> ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m >>> >>> This change seems okay, though again potentially not necessary - as we >>> never commit any changes. >>> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> bug/webrev : >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8215411 >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ >>>> >>>> >>>> Thanks, Matthias >>>> From erik.osterlund at oracle.com Tue Dec 18 09:30:51 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 18 Dec 2018 10:30:51 +0100 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: <11826825-4c2e-bc8f-5d4c-1765b7fa2838@oracle.com> References: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> <30496bb1-009c-0259-c039-5511834083e6@oracle.com> <11826825-4c2e-bc8f-5d4c-1765b7fa2838@oracle.com> Message-ID: <5d502248-ca1e-6877-aaed-0fb37a8695cd@oracle.com> Hi Coleen, On 2018-12-17 22:31, coleen.phillimore at oracle.com wrote: > > > On 12/17/18 2:57 PM, Erik ?sterlund wrote: >> Hi Dean, >> >> On 2018-12-17 20:27, dean.long at oracle.com wrote: >>> On 12/17/18 8:56 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> Today, when converting a not_entrant nmethod to zombie, the sweeper >>>> first clears the IC stubs of that nmethod, and then make_zombie(). >>>> With concurrent class unloading, this ordering is a bit unfortunate. >>>> Between clearing the IC stubs and making it zombie, the concurrent >>>> GC threads may hit a timing window where they will create IC stubs >>>> for concurrently cleaning the ICs of this still is_alive() nmethod. >>> >>> Do we normally have to creates stubs to clean an IC? >> >> Yes if you clean an IC in an nmethod that is_alive(), we currently >> have to do that using IC stubs. I have some ideas going forward for >> how we can stop doing that, but that is outside of scope for now. >> >>> >>>> The result is that during safepoint cleanup, when we finalize the IC >>>> stubs, we find one associated with a zombie. >>>> >>>> The unregistering of the nmethod from the GC will block during >>>> concurrent nmethod unloading, so clearing the IC stubs after the >>>> nmethod has become zombie is a lot more sane as there can not be any >>>> such races then. >>>> >>> >>> Where is the unregistering happening? >> >> Inside of make_zombie() we unregister_nmethod() on CollectedHeap. In >> there, ZGC makes sure we wait until concurrent nmethod unloading is >> over before continuing. > > Can you move clean_ic_stubs() inside of make_not_entrant_or_zombie() to > maintain the order dependence? Sure. The implication of that is that the sweeper will redundantly check for IC stubs to clear on unloaded nmethods that can't have any. But I agree that the model is more clear if it is inside of make_not_entrant_or_zombie(), and think that is worth the cost. New webrev: http://cr.openjdk.java.net/~eosterlund/8215491/webrev.01/ Thanks for reviewing. /Erik > thanks, > Coleen >> >>> >>> Doing the clearing after it's a zombie does sound safer. >> >> Agreed! >> >> Thanks for reviewing. >> >> /Erik >> >>> dl >>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8215491 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html >>>> >>>> Thanks, >>>> /Erik >>> > From tobias.hartmann at oracle.com Tue Dec 18 10:59:28 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 18 Dec 2018 11:59:28 +0100 Subject: 12 RFR(S) 8215205: javaVFrame much slower than vframeStream In-Reply-To: References: Message-ID: <73df5c29-6775-7921-f427-e18b35444809@oracle.com> Hi Dean, looks good to me. There is a typo in vframe_hp.hpp:76 "vframd_id" -> "vframe_id". Best regards, Tobias On 17.12.18 23:40, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8215205 > http://cr.openjdk.java.net/~dlong/8215205/webrev > > This change allows a stackwalk to continue using the faster vframeStream iterator, but changing to a > javaVFrame lazily as needed.? It introduces a new API > > ? javaVFrame* vframeStreamCommon::asJavaVFrame() > > > The first user of this API will be JDK-8214329, which is why this issue is a P3. > > dl From robbin.ehn at oracle.com Tue Dec 18 11:17:49 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 18 Dec 2018 12:17:49 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> Message-ID: <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> Hi, here is a new version. Inc: http://cr.openjdk.java.net/~rehn/8214271/3/inc/webrev/ Full: http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ Thanks, Robbin On 11/23/18 5:55 PM, Robbin Ehn wrote: > Forgot RFR in subject. > > /Robbin > > On 2018-11-23 17:51, Robbin Ehn wrote: >> Hi all, please review. >> >> When a safepoint is ended we need a way to get back to 100% utilization as fast >> as possible. 100% utilization means no idle cpu in the system if there is a >> JavaThread that could be executed. The traditional ways to wake many, e.g. >> semaphore, pthread_cond, is not implemented with a single syscall instead they >> typical do one syscall per thread to wake. >> >> This change-set contains that primitive, the WaitBarrier, and a gtest for it. >> No actual users, which is in coming patches. >> >> The WaitBarrier solves by doing a cooperative semaphore posting, threads woken >> will also post. On Linux we can instead directly use a futex and with one >> syscall wake all. Depending on how many threads and cpus the performance vary, >> but a good utilization of the machine, just on the edge of saturated, the time >> to reach 100% utilization is around 3 times faster with the WaitBarrier (where >> futex is faster than semaphore). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8214271 >> >> Passes 100 iterations of gtest on our platforms, both fastdebug and release. >> And have been stable when used in safepoints (t1-8) (coming patches). >> >> Thanks, Robbin From lutz.schmidt at sap.com Tue Dec 18 11:38:34 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 18 Dec 2018 11:38:34 +0000 Subject: RFR (XS): 8214526: Change CodeHeap State Analytics control from UL to Print* In-Reply-To: <61D97750-0580-4A03-8B23-0888EA1A8FB5@sap.com> References: <8C489BC9-FCE2-4D6D-B1FE-BF3FF16264E7@sap.com> <81DBA909-65FE-4464-BE31-F72E39148150@sap.com> <58E62EC2-0A49-4A74-BB08-50501BD72D59@sap.com> <61D97750-0580-4A03-8B23-0888EA1A8FB5@sap.com> Message-ID: Dear all, this is just to let you know that I did not abandon this topic. I'm working hard to create a solution which is acceptable by everyone - not verbose by default, but configurable to extreme verbosity if required. There is quite some work ahead, so please stay tuned. I'll be back... Happy Holidays! Lutz ?On 04.12.18, 10:26, "Schmidt, Lutz" wrote: Hi Coleen, Vladimir, Tobias, Thomas, thanks for your reviews. With that overwhelming number of thumbs up, I'll go ahead and push. Regards, Lutz On 03.12.18, 20:05, "Thomas St?fe" wrote: +1 ..Thomas On Mon, Dec 3, 2018 at 5:49 PM Schmidt, Lutz wrote: > > Hi, > description in codeHeapState.cpp is adapted. In addition, I changed the name of the command line option to PrintCodeHeapAnalytics to better match the jcmd function name. > New webrev is here: http://cr.openjdk.java.net/~lucy/webrevs/8214526.01/ > Thanks, > Lutz > > On 03.12.18, 16:54, "Schmidt, Lutz" wrote: > > Oooohhh yes, good point. > Working on it... > Lutz > > On 03.12.18, 16:44, "Vladimir Kozlov" wrote: > > Please, fix comments which describe usage in codeHeapState.cpp > > Thanks, > Vladimir > > On 12/3/18 3:42 AM, Schmidt, Lutz wrote: > > Hi Tobias, Thomas, > > thanks for the reviews. I decided to convert the switch to type "diagnostic" - that's what it is by nature. I have updated the webrev in-place. > > Thanks, > > Lutz > > > > On 03.12.18, 12:13, "Thomas St?fe" wrote: > > > > This makes much more sense than control via UL and mirrors what we do > > with e.g. NMT. There, statistic printout is triggered by jcmd and with > > -XX:+PrintNMTStatistics. > > > > Change itself looks good. I guess you need either a CSR or make the > > switch diagnostic (again, like PrintNMTStatistics). > > > > ..Thomas > > On Mon, Dec 3, 2018 at 11:41 AM Schmidt, Lutz wrote: > > > > > > Hi all, > > > > > > may I please request reviews for this tiny change. It?s the result of lengthy discussions about which mechanism to use when printing large blocks of diagnostic information. In this case, output control is reverted from (ab)using UL back to a simple command line flag. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214526 > > > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8214526.00/ > > > > > > Thanks, > > > Lutz > > > > > > > > > > > > > > From tobias.hartmann at oracle.com Tue Dec 18 12:08:46 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 18 Dec 2018 13:08:46 +0100 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client In-Reply-To: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> References: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> Message-ID: Hi Dean, this looks good to me. Best regards, Tobias On 17.12.18 23:49, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8214329 > http://cr.openjdk.java.net/~dlong/8214329/webrev/ > > In "8212605: Pure-Java implementation of AccessController.doPrivileged", the stackwalk in > JVM_GetStackAccessControlContext was changed from using a vframeStream to using a javaVFrame, so > that it could access locals. However this caused a regression because javaVFrame has more > overhead.?? JDK-8215205 fixes the problem by allowing the use of javaVFrame only when needed. > > This change reverts JVM_GetStackAccessControlContext back to using vframeStream, and uses the new > API in JDK-8215205 to get a javaVFrame lazily.? It also removes two unnecessary calls to java_sender(). > > dl From sean.mullan at oracle.com Tue Dec 18 15:12:13 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 18 Dec 2018 10:12:13 -0500 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client In-Reply-To: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> References: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> Message-ID: <0ecece2d-4f29-4e3a-2306-eb07e91cc7c5@oracle.com> Looks good, although I think someone from the HotSpot Group should also review it. On the bug, can you add the details below into the Description? Also, since you have no regression test, you will need a noreg label (I guess noreg-perf would be appropriate). Also, the bug says it is BlockedBy JDK-8215205 which should probably be removed. Thanks, Sean On 12/17/18 5:49 PM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8214329 > http://cr.openjdk.java.net/~dlong/8214329/webrev/ > > In "8212605: Pure-Java implementation of AccessController.doPrivileged", > the stackwalk in JVM_GetStackAccessControlContext was changed from using > a vframeStream to using a javaVFrame, so that it could access locals. > However this caused a regression because javaVFrame has more overhead. > JDK-8215205 fixes the problem by allowing the use of javaVFrame only > when needed. > > This change reverts JVM_GetStackAccessControlContext back to using > vframeStream, and uses the new API in JDK-8215205 to get a javaVFrame > lazily.? It also removes two unnecessary calls to java_sender(). > > dl From dean.long at oracle.com Tue Dec 18 16:10:11 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 18 Dec 2018 08:10:11 -0800 Subject: 12 RFR(S) 8215205: javaVFrame much slower than vframeStream In-Reply-To: <73df5c29-6775-7921-f427-e18b35444809@oracle.com> References: <73df5c29-6775-7921-f427-e18b35444809@oracle.com> Message-ID: <5d9f7e9a-a056-4f3a-842e-7474100755a7@oracle.com> Thanks Tobias.? I will fix the typo. dl On 12/18/18 2:59 AM, Tobias Hartmann wrote: > Hi Dean, > > looks good to me. > > There is a typo in vframe_hp.hpp:76 "vframd_id" -> "vframe_id". > > Best regards, > Tobias > > On 17.12.18 23:40, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8215205 >> http://cr.openjdk.java.net/~dlong/8215205/webrev >> >> This change allows a stackwalk to continue using the faster vframeStream iterator, but changing to a >> javaVFrame lazily as needed.? It introduces a new API >> >> ? javaVFrame* vframeStreamCommon::asJavaVFrame() >> >> >> The first user of this API will be JDK-8214329, which is why this issue is a P3. >> >> dl From dean.long at oracle.com Tue Dec 18 16:26:26 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 18 Dec 2018 08:26:26 -0800 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client In-Reply-To: References: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> Message-ID: <0c8463a0-2da0-60ec-e182-91d81f45c337@oracle.com> Thanks Tobias. dl On 12/18/18 4:08 AM, Tobias Hartmann wrote: > Hi Dean, > > this looks good to me. > > Best regards, > Tobias > > On 17.12.18 23:49, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8214329 >> http://cr.openjdk.java.net/~dlong/8214329/webrev/ >> >> In "8212605: Pure-Java implementation of AccessController.doPrivileged", the stackwalk in >> JVM_GetStackAccessControlContext was changed from using a vframeStream to using a javaVFrame, so >> that it could access locals. However this caused a regression because javaVFrame has more >> overhead.?? JDK-8215205 fixes the problem by allowing the use of javaVFrame only when needed. >> >> This change reverts JVM_GetStackAccessControlContext back to using vframeStream, and uses the new >> API in JDK-8215205 to get a javaVFrame lazily.? It also removes two unnecessary calls to java_sender(). >> >> dl From dean.long at oracle.com Tue Dec 18 16:33:07 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 18 Dec 2018 08:33:07 -0800 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client In-Reply-To: <0ecece2d-4f29-4e3a-2306-eb07e91cc7c5@oracle.com> References: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> <0ecece2d-4f29-4e3a-2306-eb07e91cc7c5@oracle.com> Message-ID: <1ae3919b-33a2-c142-950e-03751cbfcfd7@oracle.com> On 12/18/18 7:12 AM, Sean Mullan wrote: > Looks good, although I think someone from the HotSpot Group should > also review it. > Thanks Sean.? Tobias reviewed it. > On the bug, can you add the details below into the Description? Sure. > Also, since you have no regression test, you will need a noreg label > (I guess noreg-perf would be appropriate). OK, thanks. > Also, the bug says it is BlockedBy JDK-8215205 which should probably > be removed. > That's because JDK-8215205 hasn't been pushed yet. dl > Thanks, > Sean > > On 12/17/18 5:49 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8214329 >> http://cr.openjdk.java.net/~dlong/8214329/webrev/ >> >> In "8212605: Pure-Java implementation of >> AccessController.doPrivileged", the stackwalk in >> JVM_GetStackAccessControlContext was changed from using a >> vframeStream to using a javaVFrame, so that it could access locals. >> However this caused a regression because javaVFrame has more >> overhead. JDK-8215205 fixes the problem by allowing the use of >> javaVFrame only when needed. >> >> This change reverts JVM_GetStackAccessControlContext back to using >> vframeStream, and uses the new API in JDK-8215205 to get a javaVFrame >> lazily.? It also removes two unnecessary calls to java_sender(). >> >> dl From matthias.baesken at sap.com Tue Dec 18 17:19:21 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Dec 2018 17:19:21 +0000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> Message-ID: Hello, here is an updated webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.1/ sawindbg.cpp has been adjusted . The exception cases mentioned now also call env->ReleaseByteArrayElements . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Dienstag, 18. Dezember 2018 10:04 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' ; serviceability- > dev at openjdk.java.net; security-dev at openjdk.java.net; JC Beyler > > Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss > corresponding Release > > Hi Matthias, > > On 18/12/2018 6:52 pm, Baesken, Matthias wrote: > > Hi JC / David, thanks for the input . > > > > I'm not really sure what you want me to change David, am I right that I can > keep the changes to > > > > src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > > src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c > > Yes. > > > but adjust > > > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > > > to also handle theis potential early return > > > > 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) > env->GetLongField(obj, > > 724 ptrIDebugDataSpaces_ID); > > 725 CHECK_EXCEPTION_(0); > > > > ? > > And I think: > > 730 THROW_NEW_DEBUGGER_EXCEPTION_("Windbg Error: ReadVirtual > failed!", 0); > > as I assume that won't actually return normally. > > >>> Also you change seem wrong to me because it will commit the changes > in > >>> the buffer even if we "abort" here: > >>> > >>> 735 if (bytesRead != numBytes) { > >>> 736 return 0; > >>> 737 } > >>> > > > > The spec says : > > > > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function > s.html > > > >> ReleaseArrayElements Routines > >> void ReleaseArrayElements(JNIEnv *env, ArrayType > array, NativeType *elems, jint mode); > >> > >> A family of functions that informs the VM that the native code no longer > needs access to elems. The elems argument is a pointer derived from array > using the corresponding > >> GetArrayElements() function. If necessary, this function > copies back all changes made to elems to the original array. > > > > So shouldn't I tell the VM , that the native code no longer needs access to > bytePtr before returning here > > > > 735 if (bytesRead != numBytes) { > > 736 return 0; > > 737 } > > > > ? > > There are two aspects to release: > 1. the actual release (or "I don't need this any more") > 2. committing any changes made back to the original array > > This code does: > > 728 if (ptrIDebugDataSpaces->ReadVirtual((ULONG64) address, (PVOID) > bytePtr, > 729 (ULONG)numBytes, &bytesRead) != > S_OK) { > > which writes into the array. It then checks if we wrote what we expected: > > 735 if (bytesRead != numBytes) { > 736 return 0; > 737 } > > If we didn't read what was expected what should happen to the original > array? Should it be left untouched or updated with the unexpected input? > I would say left untouched and that is what the original code did. > > If everything succeeds you should do the release with mode 0; but if > taking an error exit the release should use mode JNI_ABORT so no changes > are written back. > > Cheers, > David > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Dienstag, 18. Dezember 2018 01:20 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' ; serviceability- > >> dev at openjdk.java.net; security-dev at openjdk.java.net > >> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss > >> corresponding Release > >> > >> Correction ... > >> > >> On 18/12/2018 8:25 am, David Holmes wrote: > >>> Hi Matthias, > >>> > >>> On 17/12/2018 6:59 pm, Baesken, Matthias wrote: > >>>> Hello, please review the following change. > >>>> I noticed that we miss at some places (for example in case of early > >>>> returns) > >>>> where GetByteArrayElements is used,? the corresponding > >>>> ReleaseByteArrayElements? call. > >>>> > >>>> In VirtualMachineImpl.c? I also removed a check for isCopy (is the > >>>> returned byte array a copy ?) > >>>> because from what I read at > >>>> > >>>> > >> > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function > >> s.html > >>>> > >>>> > >>>> the ReleaseByteArrayElements? routine should always be called. > >>> > >>> That's not clear to me. My reading is that you only need to release if > >>> you have changes you need to ensure are written back and that is only > >>> needed if a copy was made. > >> > >> Jc pointed out to me that if a copy is made you may need to release to > >> avoid a memory leak. But that is where the mode flags come in - and > >> again only relevant if a copy was made. > >> > >> But as per: > >> > >> https://developer.android.com/training/articles/perf-jni > >> > >> if a copy is not made and pinning is used (as with the "critical" > >> variants) then you also need to release to un-pin. > >> > >> So code should call release, with an appropriate mode, based on whether > >> isCopy was true**, and whether changed data should be written back. > >> > >> ** mode is ignored if not working on a copy so you can just set it > >> assuming a copy was made. > >> > >> Note that the hotspot implementation always makes a copy for > >> GextXXXArrayElements, and the ReleaseXXXArrayElements knows this > and > >> so > >> will always free the buffer that is passed in (other than for mode > >> JNI_COMMIT of course). > >> > >> Cheers, > >> David > >> ----- > >> > >>> That said, the change seem semantically correct and harmless in this > case. > >>> --- > >>> > >>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > >>> > >>> AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and > >>> THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. > >>> > >>> Also you change seem wrong to me because it will commit the changes > in > >>> the buffer even if we "abort" here: > >>> > >>> 735?? if (bytesRead != numBytes) { > >>> ?736????? return 0; > >>> ?737?? } > >>> > >>> so it seems to me if you really want to release on all paths then the > >>> error paths should use a mode of JNI_ABORT and only the success path > >>> should use mode 0. > >>> > >>> --- > >>> > >>> ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > >>> > >>> This change seems okay, though again potentially not necessary - as we > >>> never commit any changes. > >>> > >>> Thanks, > >>> David > >>> ----- > >>> > >>>> > >>>> bug/webrev : > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8215411 > >>>> > >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > >>>> > >>>> > >>>> Thanks, Matthias > >>>> From sean.mullan at oracle.com Tue Dec 18 19:00:18 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 18 Dec 2018 14:00:18 -0500 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client In-Reply-To: <1ae3919b-33a2-c142-950e-03751cbfcfd7@oracle.com> References: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> <0ecece2d-4f29-4e3a-2306-eb07e91cc7c5@oracle.com> <1ae3919b-33a2-c142-950e-03751cbfcfd7@oracle.com> Message-ID: On 12/18/18 11:33 AM, dean.long at oracle.com wrote: >> Also, the bug says it is BlockedBy JDK-8215205 which should probably >> be removed. >> > > That's because JDK-8215205 hasn't been pushed yet. Ok, but isn't BlockedBy supposed to mean you can't fix this issue until the one that it is blocked by is fixed? That's not the case with this issue though, right? --Sean From sean.mullan at oracle.com Tue Dec 18 19:04:46 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 18 Dec 2018 14:04:46 -0500 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client In-Reply-To: References: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> <0ecece2d-4f29-4e3a-2306-eb07e91cc7c5@oracle.com> <1ae3919b-33a2-c142-950e-03751cbfcfd7@oracle.com> Message-ID: On 12/18/18 2:00 PM, Sean Mullan wrote: > On 12/18/18 11:33 AM, dean.long at oracle.com wrote: >>> Also, the bug says it is BlockedBy JDK-8215205 which should probably >>> be removed. >>> >> >> That's because JDK-8215205 hasn't been pushed yet. > > Ok, but isn't BlockedBy supposed to mean you can't fix this issue until > the one that it is blocked by is fixed? That's not the case with this > issue though, right? Never mind the above, I see now you are using a new method added in JDK-8215205. --Sean From dean.long at oracle.com Tue Dec 18 20:52:49 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 18 Dec 2018 12:52:49 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> <61f4a254-be9d-b0b2-3525-8caa247d9556@oracle.com> <3de758a3-e990-384a-5267-265bda302cbc@oracle.com> Message-ID: David, can I list you as a reviewer? dl On 12/16/18 8:47 PM, dean.long at oracle.com wrote: > On 12/16/18 7:39 PM, dean.long at oracle.com wrote: >> On 12/16/18 7:03 PM, David Holmes wrote: >>> On 17/12/2018 12:49 pm, dean.long at oracle.com wrote: >>>> On 12/16/18 4:06 PM, David Holmes wrote: >>>>> On 15/12/2018 10:59 am, dean.long at oracle.com wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>>>>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>>>>> >>>>>> This change includes two new regression test that demonstrate the >>>>>> problem, and a fix that allows the tests >>>>>> to pass. >>>>>> >>>>>> The problem happens when the JIT compiler's escape analysis >>>>>> eliminates the allocation of the AccessControlContext object >>>>>> passed to doPrivileged.? The compiler thinks this is safe because >>>>>> it does not see that the object "escapes". >>>>> >>>>> Then surely the compiler's notion of "escapes" needs to be updated! >>>>> >>>> >>>> The compiler can inline the callee method and see that the value >>>> doesn't escape.? This is a valid optimization in cases where the >>>> callee method is known. >>> >>> But it's not a valid optimization in this case, so my comment stands. >>> >>> Is this stack walking something this is guaranteed by the spec to be >>> always valid (and hence the JIT is violating the rules), or is the >>> stack walking code making assumptions about whether it will find the >>> context object in the stack? >>> >> >> The stack walking is in the VM and is an internal implementation >> detail, not part of the AccessController API spec.? A different >> thread running normal Java code would never be able to see a >> non-escaping value.? The stack walking code does need to find the >> context object in the stack.? Non-escaping objects won't show up in >> the stack. >> >>> If we have to hack around this with an annotation I'd rather see a >>> specific annotation that addresses the problematic usecase than a >>> generic "don't inline" one. E.g. @StackVisible or something like that. >>> >> >> That sounds like a good idea for 13, but would require changes to >> both C2 and Graal, and it seems a little risky compared to using >> existing mechanisms. >> > > I forgot to address this in my last reply, but I'm not suggesting a > @DontInline annotation.? That was Claes.? My fixes uses a native method. > > dl > >> dl >> >>> Cheers, >>> David >>> >>>> >>>> dl >>>> >>>>> David >>>>> ----- >>>>> >>>>> ? However, getContext needs to be able to find >>>>>> the object using a stack walk, so we need a way to tell the >>>>>> compiler that it does indeed escape. To do this we pass the value >>>>>> to a native method that does nothing. >>>>>> >>>>>> Microbenchmark results: >>>>>> >>>>>> jdk12-b18: >>>>>> >>>>>> Benchmark??????????????? Mode? Cnt??? Score?? Error Units >>>>>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446 ns/op >>>>>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975 ns/op >>>>>> >>>>>> >>>>>> jdk12-b19: >>>>>> >>>>>> Benchmark??????????????? Mode? Cnt? Score??? Error Units >>>>>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001 ns/op >>>>>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001 ns/op >>>>>> >>>>>> this fix: >>>>>> >>>>>> Benchmark??????????????? Mode? Cnt? Score??? Error Units >>>>>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001 ns/op >>>>>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025 ns/op >>>>>> >>>>>> >>>>>> dl >>>> >> > From dean.long at oracle.com Tue Dec 18 20:53:26 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 18 Dec 2018 12:53:26 -0800 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: <790736fc-836c-d116-28c2-ffdc02c1e2ca@oracle.com> References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <07b1b4f0-f051-dd92-d107-8e9ad575211b@oracle.com> <790736fc-836c-d116-28c2-ffdc02c1e2ca@oracle.com> Message-ID: <7eb43d6e-7c16-1565-4ae1-cb16957d9382@oracle.com> Claes, can I list you as a reviewer? dl On 12/16/18 9:52 PM, Claes Redestad wrote: > Fair enough, I wasn't aware EA was looking beyond the inlined code like > this (which means it can't be "dead" or the JIT might see through the > trick at some point). I've skimmed the existing usages in the JDK and > can't find anything that seems to be dependent on DontInline defeating > EA - maybe there was in the past, or I simply misremembered. > > Either way, it might be nice to have a more explicit facility for this > in a future release, as David suggested. Say an @Escaping local > variable/parameter annotation. > > /Claes > > On 2018-12-17 05:45, dean.long at oracle.com wrote: >> Unfortunately, I don't think @DontInline on an empty method is >> sufficient >> here.? If other code is relying on @DontInline for the same purpose then >> we might need to reexamine that code.? My understanding from discussing >> with other compiler engineers is that using a native method is the >> safest >> technique that the compilers can't see through.? The problem with >> @DontInline is that C2 looks at the bytecodes of the target method, even >> if it isn't inlined (see BCEscapeAnalyzer and the EstimateArgEscape >> flag). >> There may be a way to make it work, but that would require more >> investigation, and I'm not sure the benefit outweighs the risk. >> >> dl >> >> On 12/15/18 6:48 AM, Claes Redestad wrote: >>> Hi Dean, >>> >>> to avoid escape analysis-eliminated allocations in the past @DontInline >>> has been sufficient. This means a simpler patch (no changes to native >>> code needed - added assertions notwithstanding) and passes your tests >>> with C2 (it'd concern me if Graal's EA sees through this trick, as it >>> might break some existing places where DontInline is used to this >>> effect): >>> >>> ??? /** >>> ???? * The value needs to be physically located in the frame, so >>> that it >>> ???? * can be found by a stack walk. >>> ???? */ >>> ??? @Hidden >>> ??? @DontInline >>> ??? private static void ensureMaterializedForStackWalk(Object o) {} >>> >>> Thanks! >>> >>> /Claes >>> >>> On 2018-12-15 01:59, dean.long at oracle.com wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>>> >>>> This change includes two new regression test that demonstrate the >>>> problem, and a fix that allows the tests >>>> to pass. >>>> >>>> The problem happens when the JIT compiler's escape analysis >>>> eliminates the allocation of the AccessControlContext object passed >>>> to doPrivileged.? The compiler thinks this is safe because it does >>>> not see that the object "escapes".? However, getContext needs to be >>>> able to find the object using a stack walk, so we need a way to >>>> tell the compiler that it does indeed escape.? To do this we pass >>>> the value to a native method that does nothing. >>>> >>>> Microbenchmark results: >>>> >>>> jdk12-b18: >>>> >>>> Benchmark??????????????? Mode? Cnt??? Score?? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975? ns/op >>>> >>>> >>>> jdk12-b19: >>>> >>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001? ns/op >>>> >>>> this fix: >>>> >>>> Benchmark??????????????? Mode? Cnt? Score??? Error? Units >>>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001? ns/op >>>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025? ns/op >>>> >>>> >>>> dl >> From david.holmes at oracle.com Tue Dec 18 22:19:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Dec 2018 08:19:42 +1000 Subject: 12 RFR(M) 8214583: AccessController.getContext may return wrong value after JDK-8212605 In-Reply-To: References: <83fe9df6-14eb-1d3a-1f70-120e95f08024@oracle.com> <741dfaed-dc81-eb0e-1f9d-d266bc46866e@oracle.com> <61f4a254-be9d-b0b2-3525-8caa247d9556@oracle.com> <3de758a3-e990-384a-5267-265bda302cbc@oracle.com> Message-ID: <879b2fc6-1126-c615-3778-af1e1d6c634c@oracle.com> On 19/12/2018 6:52 am, dean.long at oracle.com wrote: > David, can I list you as a reviewer? No, sorry, I only commented on the general issue. David > dl > > On 12/16/18 8:47 PM, dean.long at oracle.com wrote: >> On 12/16/18 7:39 PM, dean.long at oracle.com wrote: >>> On 12/16/18 7:03 PM, David Holmes wrote: >>>> On 17/12/2018 12:49 pm, dean.long at oracle.com wrote: >>>>> On 12/16/18 4:06 PM, David Holmes wrote: >>>>>> On 15/12/2018 10:59 am, dean.long at oracle.com wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214583 >>>>>>> http://cr.openjdk.java.net/~dlong/8214583/webrev >>>>>>> >>>>>>> This change includes two new regression test that demonstrate the >>>>>>> problem, and a fix that allows the tests >>>>>>> to pass. >>>>>>> >>>>>>> The problem happens when the JIT compiler's escape analysis >>>>>>> eliminates the allocation of the AccessControlContext object >>>>>>> passed to doPrivileged.? The compiler thinks this is safe because >>>>>>> it does not see that the object "escapes". >>>>>> >>>>>> Then surely the compiler's notion of "escapes" needs to be updated! >>>>>> >>>>> >>>>> The compiler can inline the callee method and see that the value >>>>> doesn't escape.? This is a valid optimization in cases where the >>>>> callee method is known. >>>> >>>> But it's not a valid optimization in this case, so my comment stands. >>>> >>>> Is this stack walking something this is guaranteed by the spec to be >>>> always valid (and hence the JIT is violating the rules), or is the >>>> stack walking code making assumptions about whether it will find the >>>> context object in the stack? >>>> >>> >>> The stack walking is in the VM and is an internal implementation >>> detail, not part of the AccessController API spec.? A different >>> thread running normal Java code would never be able to see a >>> non-escaping value.? The stack walking code does need to find the >>> context object in the stack.? Non-escaping objects won't show up in >>> the stack. >>> >>>> If we have to hack around this with an annotation I'd rather see a >>>> specific annotation that addresses the problematic usecase than a >>>> generic "don't inline" one. E.g. @StackVisible or something like that. >>>> >>> >>> That sounds like a good idea for 13, but would require changes to >>> both C2 and Graal, and it seems a little risky compared to using >>> existing mechanisms. >>> >> >> I forgot to address this in my last reply, but I'm not suggesting a >> @DontInline annotation.? That was Claes.? My fixes uses a native method. >> >> dl >> >>> dl >>> >>>> Cheers, >>>> David >>>> >>>>> >>>>> dl >>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>> ? However, getContext needs to be able to find >>>>>>> the object using a stack walk, so we need a way to tell the >>>>>>> compiler that it does indeed escape. To do this we pass the value >>>>>>> to a native method that does nothing. >>>>>>> >>>>>>> Microbenchmark results: >>>>>>> >>>>>>> jdk12-b18: >>>>>>> >>>>>>> Benchmark??????????????? Mode? Cnt??? Score?? Error Units >>>>>>> DoPrivileged.test??????? avgt?? 25? 255.626 ? 6.446 ns/op >>>>>>> DoPrivileged.testInline? avgt?? 25? 250.968 ? 4.975 ns/op >>>>>>> >>>>>>> >>>>>>> jdk12-b19: >>>>>>> >>>>>>> Benchmark??????????????? Mode? Cnt? Score??? Error Units >>>>>>> DoPrivileged.test??????? avgt?? 25? 5.689 ?? 0.001 ns/op >>>>>>> DoPrivileged.testInline? avgt?? 25? 2.765 ?? 0.001 ns/op >>>>>>> >>>>>>> this fix: >>>>>>> >>>>>>> Benchmark??????????????? Mode? Cnt? Score??? Error Units >>>>>>> DoPrivileged.test??????? avgt?? 25? 5.020 ?? 0.001 ns/op >>>>>>> DoPrivileged.testInline? avgt?? 25? 2.774 ?? 0.025 ns/op >>>>>>> >>>>>>> >>>>>>> dl >>>>> >>> >> > From david.holmes at oracle.com Wed Dec 19 04:51:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Dec 2018 14:51:01 +1000 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <15bf0187-fb9e-0724-d1a6-c4b4030fe3b9@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <15bf0187-fb9e-0724-d1a6-c4b4030fe3b9@oracle.com> Message-ID: Hi Robbin, Note this was focusing on the design of the API to ensure the semantics are clear and well-defined so that future users of the API know how to use it correctly and safely. It also then allows a review of the implementation to check back to the spec to see if it behaves as intended and checks all necessary constraints That's why I wanted it to be as complete as possible (spurious wakeups allowed? No - then lets document that) and now allow implementation specific choices unless unavoidable. I'll get to updated webrevs before I finish for the year on Friday. Thanks, David On 17/12/2018 8:39 pm, Robbin Ehn wrote: > Hi David, > > On 11/26/18 7:12 AM, David Holmes wrote: >> Hi Robbin, >> >> On 24/11/2018 2:55 am, Robbin Ehn wrote: >>> Forgot RFR in subject. >> >> Yep and now you have two different review threads happening in >> parallel unfortunately :( >> >> - src/hotspot/share/utilities/waitBarrier.hpp >> >> I'm studying just the WaitBarrierType API. Is this inherently tied to >> safepoint usage or intended as a general synchronization tool? As a >> general tool the API does not have clear semantics on how it should be >> used: > > There is two potential uses-cases in G1, which might require some changes. > >> >> - How do you communicate the current tag between the arming thread and >> the waiting threads? There would seem to be inherent races between >> arm(tag) and wait(tag) unless access to the tag itself is synchronized >> via another mechanism. > > As seen in gtest: > - arm tag > - publish tag > >> >> - What happens if wake() is called before disarm()? Should it be >> disallowed? (For other readers disarm() and wake() are distinct >> operations so that they fit better into the existing safepoint >> protocol which disarms the safepoint polling page at one spot and >> wakes blocked threads at another.) > > Yes, wake() have no effect if you have not disarmed. > There are asserts checking this. > >> >> - Should there be constraints that the same thread must >> arm/disarm/wake? It doesn't really make sense to allow these >> operations to happen in arbitrary order from multiple threads. > > Maybe, *thinking* if there is nice way to get the arming/waking thread > into an > argument. > >> >> The semantics for re-arming with the same tag should be clearly set >> out not "implementation-defined". This should probably be a usage >> error IMHO - but it comes back to how the tag is expected to be used. > > If you have a way to detect that all threads have left the waitbarrier, > there is > no problem with re-using the same tag. So yes, re-arming before that > with the > same tag should be an error. But since the waitbarrier can't tell the > difference > and do not know which tag it was last armed for I have no assert. > >> >> As Andrew mentioned there needs to be documentation regarding spurious >> wakeups or other "interruptions" at the API level. And I assume a >> blocked wait() only ever returns in response to a wake(). > > Yes, if it's disarmed before. Since these are handled internally there > are no > spurious wakeups, not sure why we want to document that, except inside the > implementation. E.g. semaphore.hpp does not document that it do not have > it. > >> >> Nothwithstanding clarification of the above may I suggest the >> following rewrite of the API documentation for further clarity: >> >> /* Platform independent WaitBarrier API. >> ??? An armed WaitBarrier prevents threads from advancing until the >> ??? barrier is disarmed and the waiting threads woken. The barrier is >> ??? armed by setting a non-zero value - the tag. >> >> ??? Expected Usage: >> ????? - Arming thread: >> ????????? tag = ...;? // non-zero value >> ????????? barrier.arm(tag); >> ????????? >> ????????? barrier.disarm(); >> ????????? barrier.wake(); >> >> ??????? - After arm(tag) returns any thread calling wait(tag) will block >> ??????? - After disarm() returns any subsequent calls to wait(tag) >> will not block >> ??????? - After wake() returns all blocked threads are unblocked and >> eligible to execute again >> ??????? - After calling disarm() and wake() the barrier is ready to be >> re-armed with a new tag >> >> ???? - Waiting threads >> ???????? wait(tag); // don't execute following code unless 'safe' >> ???????? >> >> ?????? - A call to wait(tag) will block if the barrier is armed with >> the value 'tag'; else it will return immediately. >> ?????? - A blocked thread is eligible to execute again once the >> barrier is disarmed and wake() has been called. >> >> ??A primary goal of the WaitBarrier implementation is to wake all waiting >> ??threads as fast, and as concurrently, as possible. >> >> */ > > Updated, added after barrier.arm(tag); > >> >> Looking at the "implementation" in this file I'm unclear on the way >> the Linux specialization is being handled here. Why do we need the >> template? Can't this just be done the same way we do Semaphore? > > Because the gtest can test all implementations available on that platform. > Meaning I can test both implementation locally and compare performance > etc.. > > I'll post one update after I sorted out the other review parts also. > > Thanks, Robbin > >> >> More to follow. >> >> Thanks, >> David >> ----- >> >>> /Robbin >>> >>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> When a safepoint is ended we need a way to get back to 100% >>>> utilization as fast >>>> as possible. 100% utilization means no idle cpu in the system if >>>> there is a >>>> JavaThread that could be executed. The traditional ways to wake >>>> many, e.g. >>>> semaphore, pthread_cond, is not implemented with a single syscall >>>> instead they >>>> typical do one syscall per thread to wake. >>>> >>>> This change-set contains that primitive, the WaitBarrier, and a >>>> gtest for it. >>>> No actual users, which is in coming patches. >>>> >>>> The WaitBarrier solves by doing a cooperative semaphore posting, >>>> threads woken >>>> will also post. On Linux we can instead directly use a futex and >>>> with one >>>> syscall wake all. Depending on how many threads and cpus the >>>> performance vary, >>>> but a good utilization of the machine, just on the edge of >>>> saturated, the time to reach 100% utilization is around 3 times >>>> faster with the WaitBarrier (where futex is faster than semaphore). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>> >>>> Passes 100 iterations of gtest on our platforms, both fastdebug and >>>> release. >>>> And have been stable when used in safepoints (t1-8) (coming patches). >>>> >>>> Thanks, Robbin From david.holmes at oracle.com Wed Dec 19 06:20:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Dec 2018 16:20:49 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> Message-ID: <0e0a64cf-0047-417e-f0f6-351e60e3704e@oracle.com> On 19/12/2018 3:19 am, Baesken, Matthias wrote: > Hello, here is an updated webrev : > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.1/ > > sawindbg.cpp has been adjusted . > The exception cases mentioned now also call env->ReleaseByteArrayElements . Looks good - thanks. One minor style nit: 725 if(env->ExceptionOccurred()) { Please add a space after "if" (yes I see the macros have this wrong too). No need for updated webrev. Thanks, David > Best regards, Matthias > >> -----Original Message----- >> From: David Holmes >> Sent: Dienstag, 18. Dezember 2018 10:04 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' ; serviceability- >> dev at openjdk.java.net; security-dev at openjdk.java.net; JC Beyler >> >> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss >> corresponding Release >> >> Hi Matthias, >> >> On 18/12/2018 6:52 pm, Baesken, Matthias wrote: >>> Hi JC / David, thanks for the input . >>> >>> I'm not really sure what you want me to change David, am I right that I can >> keep the changes to >>> >>> src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m >>> src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c >> >> Yes. >> >>> but adjust >>> >>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>> >>> to also handle theis potential early return >>> >>> 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) >> env->GetLongField(obj, >>> 724 ptrIDebugDataSpaces_ID); >>> 725 CHECK_EXCEPTION_(0); >>> >>> ? >> >> And I think: >> >> 730 THROW_NEW_DEBUGGER_EXCEPTION_("Windbg Error: ReadVirtual >> failed!", 0); >> >> as I assume that won't actually return normally. >> >>>>> Also you change seem wrong to me because it will commit the changes >> in >>>>> the buffer even if we "abort" here: >>>>> >>>>> 735 if (bytesRead != numBytes) { >>>>> 736 return 0; >>>>> 737 } >>>>> >>> >>> The spec says : >>> >>> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function >> s.html >>> >>>> ReleaseArrayElements Routines >>>> void ReleaseArrayElements(JNIEnv *env, ArrayType >> array, NativeType *elems, jint mode); >>>> >>>> A family of functions that informs the VM that the native code no longer >> needs access to elems. The elems argument is a pointer derived from array >> using the corresponding >>>> GetArrayElements() function. If necessary, this function >> copies back all changes made to elems to the original array. >>> >>> So shouldn't I tell the VM , that the native code no longer needs access to >> bytePtr before returning here >>> >>> 735 if (bytesRead != numBytes) { >>> 736 return 0; >>> 737 } >>> >>> ? >> >> There are two aspects to release: >> 1. the actual release (or "I don't need this any more") >> 2. committing any changes made back to the original array >> >> This code does: >> >> 728 if (ptrIDebugDataSpaces->ReadVirtual((ULONG64) address, (PVOID) >> bytePtr, >> 729 (ULONG)numBytes, &bytesRead) != >> S_OK) { >> >> which writes into the array. It then checks if we wrote what we expected: >> >> 735 if (bytesRead != numBytes) { >> 736 return 0; >> 737 } >> >> If we didn't read what was expected what should happen to the original >> array? Should it be left untouched or updated with the unexpected input? >> I would say left untouched and that is what the original code did. >> >> If everything succeeds you should do the release with mode 0; but if >> taking an error exit the release should use mode JNI_ABORT so no changes >> are written back. >> >> Cheers, >> David >> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Dienstag, 18. Dezember 2018 01:20 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' ; serviceability- >>>> dev at openjdk.java.net; security-dev at openjdk.java.net >>>> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss >>>> corresponding Release >>>> >>>> Correction ... >>>> >>>> On 18/12/2018 8:25 am, David Holmes wrote: >>>>> Hi Matthias, >>>>> >>>>> On 17/12/2018 6:59 pm, Baesken, Matthias wrote: >>>>>> Hello, please review the following change. >>>>>> I noticed that we miss at some places (for example in case of early >>>>>> returns) >>>>>> where GetByteArrayElements is used,? the corresponding >>>>>> ReleaseByteArrayElements? call. >>>>>> >>>>>> In VirtualMachineImpl.c? I also removed a check for isCopy (is the >>>>>> returned byte array a copy ?) >>>>>> because from what I read at >>>>>> >>>>>> >>>> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function >>>> s.html >>>>>> >>>>>> >>>>>> the ReleaseByteArrayElements? routine should always be called. >>>>> >>>>> That's not clear to me. My reading is that you only need to release if >>>>> you have changes you need to ensure are written back and that is only >>>>> needed if a copy was made. >>>> >>>> Jc pointed out to me that if a copy is made you may need to release to >>>> avoid a memory leak. But that is where the mode flags come in - and >>>> again only relevant if a copy was made. >>>> >>>> But as per: >>>> >>>> https://developer.android.com/training/articles/perf-jni >>>> >>>> if a copy is not made and pinning is used (as with the "critical" >>>> variants) then you also need to release to un-pin. >>>> >>>> So code should call release, with an appropriate mode, based on whether >>>> isCopy was true**, and whether changed data should be written back. >>>> >>>> ** mode is ignored if not working on a copy so you can just set it >>>> assuming a copy was made. >>>> >>>> Note that the hotspot implementation always makes a copy for >>>> GextXXXArrayElements, and the ReleaseXXXArrayElements knows this >> and >>>> so >>>> will always free the buffer that is passed in (other than for mode >>>> JNI_COMMIT of course). >>>> >>>> Cheers, >>>> David >>>> ----- >>>> >>>>> That said, the change seem semantically correct and harmless in this >> case. >>>>> --- >>>>> >>>>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>>>> >>>>> AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and >>>>> THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. >>>>> >>>>> Also you change seem wrong to me because it will commit the changes >> in >>>>> the buffer even if we "abort" here: >>>>> >>>>> 735?? if (bytesRead != numBytes) { >>>>> ?736????? return 0; >>>>> ?737?? } >>>>> >>>>> so it seems to me if you really want to release on all paths then the >>>>> error paths should use a mode of JNI_ABORT and only the success path >>>>> should use mode 0. >>>>> >>>>> --- >>>>> >>>>> ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m >>>>> >>>>> This change seems okay, though again potentially not necessary - as we >>>>> never commit any changes. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> bug/webrev : >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8215411 >>>>>> >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ >>>>>> >>>>>> >>>>>> Thanks, Matthias >>>>>> From Nick.Gasson at arm.com Wed Dec 19 07:31:10 2018 From: Nick.Gasson at arm.com (Nick Gasson (Arm Technology China)) Date: Wed, 19 Dec 2018 07:31:10 +0000 Subject: RFR: 8209414: AArch64: method handle invocation does not respect JVMTI interp_only mode Message-ID: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> Hi, This is a small patch to fix failures of the following jtreg tests on AArch64: vmTestbase/vm/mlvm/indy/stress/jdi/breakpointInCompiledCode/Test.java vmTestbase/vm/mlvm/meth/func/jdi/breakpoint/Test.java Webrev: http://cr.openjdk.java.net/~njian/8209414/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8209414 In MethodHandles::jump_from_method_handle the test on the value of the _interp_only_mode thread field is wrong: if this is non-zero we are tracing or single stepping with JVMTI and should always branch to the interpreted entry, to ensure the JVMTI method entry/exit hooks are called. But in the AArch64 version we branch to the interpreted entry if this value is *zero*, which is backwards. The result of this is mismatched entry/exit callbacks when invoking a method through a MethodHandle while tracing, which triggers an assertion failure in debug builds and strange behaviour in release builds. The ldrb -> ldrw change isn't strictly necessary necessary, but I think it's good for consistency to make the instructions identical to InterpreterMacroAssembler::jump_from_interpreted, and as _interp_only_mode is an int we don't have to worry about endianness, etc. if we load a word. Thanks, Nick From robbin.ehn at oracle.com Wed Dec 19 08:40:27 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 19 Dec 2018 09:40:27 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <15bf0187-fb9e-0724-d1a6-c4b4030fe3b9@oracle.com> Message-ID: Hi David, On 2018-12-19 05:51, David Holmes wrote: > Hi Robbin, > > Note this was focusing on the design of the API to ensure the semantics are > clear and well-defined so that future users of the API know how to use it > correctly and safely. It also then allows a review of the implementation to > check back to the spec to see if it behaves as intended and checks all necessary > constraints That's why I wanted it to be as complete as possible (spurious > wakeups allowed? No - then lets document that) and now allow implementation > specific choices unless unavoidable. Sure! Maybe add that to the wait() method, with something like: + // Guarantees not to return until disarm() is called, + // if called with currently armed tag (otherwise returns immediately). + // Implementation must guarantee no spurious wakeups. 123 // Guarantees to return if disarm() and wake() is called. 124 // Provides a trailing fence. 125 void wait(int barrier_tag) { > > I'll get to updated webrevs before I finish for the year on Friday. Thanks! /Robbin > > Thanks, > David > > On 17/12/2018 8:39 pm, Robbin Ehn wrote: >> Hi David, >> >> On 11/26/18 7:12 AM, David Holmes wrote: >>> Hi Robbin, >>> >>> On 24/11/2018 2:55 am, Robbin Ehn wrote: >>>> Forgot RFR in subject. >>> >>> Yep and now you have two different review threads happening in parallel >>> unfortunately :( >>> >>> - src/hotspot/share/utilities/waitBarrier.hpp >>> >>> I'm studying just the WaitBarrierType API. Is this inherently tied to >>> safepoint usage or intended as a general synchronization tool? As a general >>> tool the API does not have clear semantics on how it should be used: >> >> There is two potential uses-cases in G1, which might require some changes. >> >>> >>> - How do you communicate the current tag between the arming thread and the >>> waiting threads? There would seem to be inherent races between arm(tag) and >>> wait(tag) unless access to the tag itself is synchronized via another mechanism. >> >> As seen in gtest: >> - arm tag >> - publish tag >> >>> >>> - What happens if wake() is called before disarm()? Should it be disallowed? >>> (For other readers disarm() and wake() are distinct operations so that they >>> fit better into the existing safepoint protocol which disarms the safepoint >>> polling page at one spot and wakes blocked threads at another.) >> >> Yes, wake() have no effect if you have not disarmed. >> There are asserts checking this. >> >>> >>> - Should there be constraints that the same thread must arm/disarm/wake? It >>> doesn't really make sense to allow these operations to happen in arbitrary >>> order from multiple threads. >> >> Maybe, *thinking* if there is nice way to get the arming/waking thread into an >> argument. >> >>> >>> The semantics for re-arming with the same tag should be clearly set out not >>> "implementation-defined". This should probably be a usage error IMHO - but it >>> comes back to how the tag is expected to be used. >> >> If you have a way to detect that all threads have left the waitbarrier, there is >> no problem with re-using the same tag. So yes, re-arming before that with the >> same tag should be an error. But since the waitbarrier can't tell the difference >> and do not know which tag it was last armed for I have no assert. >> >>> >>> As Andrew mentioned there needs to be documentation regarding spurious >>> wakeups or other "interruptions" at the API level. And I assume a blocked >>> wait() only ever returns in response to a wake(). >> >> Yes, if it's disarmed before. Since these are handled internally there are no >> spurious wakeups, not sure why we want to document that, except inside the >> implementation. E.g. semaphore.hpp does not document that it do not have it. >> >>> >>> Nothwithstanding clarification of the above may I suggest the following >>> rewrite of the API documentation for further clarity: >>> >>> /* Platform independent WaitBarrier API. >>> ??? An armed WaitBarrier prevents threads from advancing until the >>> ??? barrier is disarmed and the waiting threads woken. The barrier is >>> ??? armed by setting a non-zero value - the tag. >>> >>> ??? Expected Usage: >>> ????? - Arming thread: >>> ????????? tag = ...;? // non-zero value >>> ????????? barrier.arm(tag); >>> ????????? >>> ????????? barrier.disarm(); >>> ????????? barrier.wake(); >>> >>> ??????? - After arm(tag) returns any thread calling wait(tag) will block >>> ??????? - After disarm() returns any subsequent calls to wait(tag) will not >>> block >>> ??????? - After wake() returns all blocked threads are unblocked and eligible >>> to execute again >>> ??????? - After calling disarm() and wake() the barrier is ready to be >>> re-armed with a new tag >>> >>> ???? - Waiting threads >>> ???????? wait(tag); // don't execute following code unless 'safe' >>> ???????? >>> >>> ?????? - A call to wait(tag) will block if the barrier is armed with the >>> value 'tag'; else it will return immediately. >>> ?????? - A blocked thread is eligible to execute again once the barrier is >>> disarmed and wake() has been called. >>> >>> ??A primary goal of the WaitBarrier implementation is to wake all waiting >>> ??threads as fast, and as concurrently, as possible. >>> >>> */ >> >> Updated, added after barrier.arm(tag); >> >>> >>> Looking at the "implementation" in this file I'm unclear on the way the Linux >>> specialization is being handled here. Why do we need the template? Can't this >>> just be done the same way we do Semaphore? >> >> Because the gtest can test all implementations available on that platform. >> Meaning I can test both implementation locally and compare performance etc.. >> >> I'll post one update after I sorted out the other review parts also. >> >> Thanks, Robbin >> >>> >>> More to follow. >>> >>> Thanks, >>> David >>> ----- >>> >>>> /Robbin >>>> >>>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> When a safepoint is ended we need a way to get back to 100% utilization as >>>>> fast >>>>> as possible. 100% utilization means no idle cpu in the system if there is a >>>>> JavaThread that could be executed. The traditional ways to wake many, e.g. >>>>> semaphore, pthread_cond, is not implemented with a single syscall instead they >>>>> typical do one syscall per thread to wake. >>>>> >>>>> This change-set contains that primitive, the WaitBarrier, and a gtest for it. >>>>> No actual users, which is in coming patches. >>>>> >>>>> The WaitBarrier solves by doing a cooperative semaphore posting, threads woken >>>>> will also post. On Linux we can instead directly use a futex and with one >>>>> syscall wake all. Depending on how many threads and cpus the performance vary, >>>>> but a good utilization of the machine, just on the edge of saturated, the >>>>> time to reach 100% utilization is around 3 times faster with the >>>>> WaitBarrier (where futex is faster than semaphore). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>>> >>>>> CR: >>>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>>> >>>>> Passes 100 iterations of gtest on our platforms, both fastdebug and release. >>>>> And have been stable when used in safepoints (t1-8) (coming patches). >>>>> >>>>> Thanks, Robbin From adinn at redhat.com Wed Dec 19 09:25:37 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 19 Dec 2018 09:25:37 +0000 Subject: [aarch64-port-dev ] RFR: 8209414: AArch64: method handle invocation does not respect JVMTI interp_only mode In-Reply-To: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> References: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> Message-ID: <3e1c446f-e949-4769-09b1-99b0f32a5343@redhat.com> Hi Nick, On 19/12/2018 07:31, Nick Gasson (Arm Technology China) wrote: > Hi, > > This is a small patch to fix failures of the following jtreg tests on > AArch64: > vmTestbase/vm/mlvm/indy/stress/jdi/breakpointInCompiledCode/Test.java > vmTestbase/vm/mlvm/meth/func/jdi/breakpoint/Test.java > > Webrev: http://cr.openjdk.java.net/~njian/8209414/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8209414 > > In MethodHandles::jump_from_method_handle the test on the value of the > _interp_only_mode thread field is wrong: if this is non-zero we are > tracing or single stepping with JVMTI and should always branch to the > interpreted entry, to ensure the JVMTI method entry/exit hooks are > called. But in the AArch64 version we branch to the interpreted entry if > this value is *zero*, which is backwards. The result of this is > mismatched entry/exit callbacks when invoking a method through a > MethodHandle while tracing, which triggers an assertion failure in debug > builds and strange behaviour in release builds. > > The ldrb -> ldrw change isn't strictly necessary necessary, but I think > it's good for consistency to make the instructions identical to > InterpreterMacroAssembler::jump_from_interpreted, and as > _interp_only_mode is an int we don't have to worry about endianness, > etc. if we load a word. Yes, nice catch and I agree about the use of ldrw over ldrb. Reviewed. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Nick.Gasson at arm.com Wed Dec 19 09:42:27 2018 From: Nick.Gasson at arm.com (Nick Gasson (Arm Technology China)) Date: Wed, 19 Dec 2018 09:42:27 +0000 Subject: [aarch64-port-dev ] RFR: 8209414: AArch64: method handle invocation does not respect JVMTI interp_only mode In-Reply-To: <3e1c446f-e949-4769-09b1-99b0f32a5343@redhat.com> References: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> <3e1c446f-e949-4769-09b1-99b0f32a5343@redhat.com> Message-ID: On 19/12/2018 17:25, Andrew Dinn wrote: > Yes, nice catch and I agree about the use of ldrw over ldrb. Reviewed. > Thanks Andrew! Could someone sponsor this patch for me? Nick From thomas.schatzl at oracle.com Wed Dec 19 10:07:25 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 19 Dec 2018 11:07:25 +0100 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> Message-ID: On Wed, 2018-12-12 at 18:30 +0800, Man Cao wrote: > Hi, > > Addressed several comments. New webrev: > https://cr.openjdk.java.net/~manc/8212206/webrev.01/ > Diff from webrev.00: > https://cr.openjdk.java.net/~manc/8212206/webrev.diff.00-01/ Looks good, but see the comments below too. > > > > Assuming that all collectors want to implement this, and actually > > > need to I am leaning towards doing so. However the ZGC/Shenandoah > > > people might object to this. > > I haven't moved the GCOverheadChecker instance to CollectedHeap yet. > Should I wait for ZGC/Shenandoah people to give some green light? > I cc'ed zgc-dev and shenandoah-dev. > > Creating a CSR and getting it approved is not a big deal; it may > > even be useful as it clearly communicates the change to the users. > > Additionally I think due to that translation table I mentioned, the > > old name can still be used I think. > > As for the hsperfdata counter sun.gc.policy.gcTimeLimitExceeded, I > found two issues: > (a) The translation table in aliasmap seems to mainly target JDK- > internal usage of the counters. > Only the PerfDataBufferImpl.findByName() method makes use of the > aliasmap. There are use cases that doesn't work with the aliasmap. > E.g.: > $ jstat -J-Djstat.showUnsupported=true -name java.ci.totalTime > // This works > $ jstat -J-Djstat.showUnsupported=true -name hotspot.ci.total.time > // This doesn't work > This is because the "jstat -name" would invoke the > PerfDataBufferImpl.findByPattern() method, > which does not take the aliasmap into account. > > In addition, there are independent implementations that read > /tmp/hsperfdata_/ > file directly, e.g.: > https://github.com/twitter/commons/tree/master/src/python/twitter/common/java/perfdata > And Google internally has a Java implementation that does the job > (but uses Guava library). > These tools do not support aliasmap. Okay, I admit I do not know much about these counters... > > As for this counter, fortunately I found it hardly used anywhere in > OpenJDK or across Google's depot. > And its current implementation is not that useful, as described > below. > > (b) This counter contains a boolean value that is set at every GC. > This makes its usefulness limited, as it is very hard to catch the > moment when it is set to 1. When a full GC sets it to 1 and throws an > OOM exception due to GC overhead exceeded, the JVM could subsequently > trigger another full GC and reset the counter to 0, then terminates > due to the OOM exception. > If -XX:PerfDataSaveFile=foo.hsperf is set, foo.hsperf would contain 0 > for this counter in this case, > which is quite unexpected. > > I'd propose to change this counter to a cumulative counter, i.e, the > total number of GCs that trigger GC-overhead-limit-exceeded event, > and rename this counter as the same time. > I think it is cleaner to do this change in a separate RFE and CSR. > What do you think? I agree to separate this change out. Thomas From adinn at redhat.com Wed Dec 19 11:47:38 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 19 Dec 2018 11:47:38 +0000 Subject: [aarch64-port-dev ] RFR: 8209414: AArch64: method handle invocation does not respect JVMTI interp_only mode In-Reply-To: References: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> <3e1c446f-e949-4769-09b1-99b0f32a5343@redhat.com> Message-ID: On 19/12/2018 09:42, Nick Gasson (Arm Technology China) wrote: > On 19/12/2018 17:25, Andrew Dinn wrote: >> Yes, nice catch and I agree about the use of ldrw over ldrb. Reviewed. >> > > Thanks Andrew! Could someone sponsor this patch for me? The patch has been pushed to the dev repo regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From per.liden at oracle.com Wed Dec 19 14:06:27 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 19 Dec 2018 15:06:27 +0100 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> Message-ID: <7e0c775d-86c1-b80c-b1a6-373ca21206ba@oracle.com> Hi, On 12/19/18 11:07 AM, Thomas Schatzl wrote: > On Wed, 2018-12-12 at 18:30 +0800, Man Cao wrote: >> Hi, >> >> Addressed several comments. New webrev: >> https://cr.openjdk.java.net/~manc/8212206/webrev.01/ >> Diff from webrev.00: >> https://cr.openjdk.java.net/~manc/8212206/webrev.diff.00-01/ > > Looks good, but see the comments below too. > >> >>>> Assuming that all collectors want to implement this, and actually >>>> need to I am leaning towards doing so. However the ZGC/Shenandoah >>>> people might object to this. >> >> I haven't moved the GCOverheadChecker instance to CollectedHeap yet. >> Should I wait for ZGC/Shenandoah people to give some green light? I don't foresee that this will be implemented, or even makes sense, for ZGC. As I see it, this is only a thing STW collectors. For that reason, I don't think it belongs in CollectedHeap. Keeping it as a separate utility class for collectors that want to use it sounds better. I haven't looked very closely at the patch, but couldn't help to notice that the option is called "GCOverheapLimitThreshold" (and "AdaptiveSizePolicyGCTimeLimitThreshold" before that), which is a tautology and a not very good description of what it is. How about we take the opportunity to clean this up and completely ditch the "gc_overhead_limit_count" thing and get rid of this option? It's a "develop" option, so it's not available to normal users anyway. Has anyone of you ever used this option and actually find it valuable? cheers, Per >> > > I cc'ed zgc-dev and shenandoah-dev. > >>> Creating a CSR and getting it approved is not a big deal; it may >>> even be useful as it clearly communicates the change to the users. >>> Additionally I think due to that translation table I mentioned, the >>> old name can still be used I think. >> >> As for the hsperfdata counter sun.gc.policy.gcTimeLimitExceeded, I >> found two issues: >> (a) The translation table in aliasmap seems to mainly target JDK- >> internal usage of the counters. >> Only the PerfDataBufferImpl.findByName() method makes use of the >> aliasmap. There are use cases that doesn't work with the aliasmap. >> E.g.: >> $ jstat -J-Djstat.showUnsupported=true -name java.ci.totalTime >> // This works >> $ jstat -J-Djstat.showUnsupported=true -name hotspot.ci.total.time >> // This doesn't work >> This is because the "jstat -name" would invoke the >> PerfDataBufferImpl.findByPattern() method, >> which does not take the aliasmap into account. >> >> In addition, there are independent implementations that read >> /tmp/hsperfdata_/ >> file directly, e.g.: >> > https://github.com/twitter/commons/tree/master/src/python/twitter/common/java/perfdata >> And Google internally has a Java implementation that does the job >> (but uses Guava library). >> These tools do not support aliasmap. > > Okay, I admit I do not know much about these counters... > >> >> As for this counter, fortunately I found it hardly used anywhere in >> OpenJDK or across Google's depot. >> And its current implementation is not that useful, as described >> below. >> >> (b) This counter contains a boolean value that is set at every GC. >> This makes its usefulness limited, as it is very hard to catch the >> moment when it is set to 1. When a full GC sets it to 1 and throws an >> OOM exception due to GC overhead exceeded, the JVM could subsequently >> trigger another full GC and reset the counter to 0, then terminates >> due to the OOM exception. >> If -XX:PerfDataSaveFile=foo.hsperf is set, foo.hsperf would contain 0 >> for this counter in this case, >> which is quite unexpected. >> >> I'd propose to change this counter to a cumulative counter, i.e, the >> total number of GCs that trigger GC-overhead-limit-exceeded event, >> and rename this counter as the same time. >> I think it is cleaner to do this change in a separate RFE and CSR. >> What do you think? > > I agree to separate this change out. > > Thomas > From matthias.baesken at sap.com Wed Dec 19 17:13:56 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 19 Dec 2018 17:13:56 +0000 Subject: ThreadPriorityPolicy settings for non-root users Message-ID: Hello , currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 (normal, the default) and 1, the so called "aggressive" mode : "1 : Aggressive. "\ " Java thread priorities map over to the entire range of "\ " native thread priorities. Higher Java thread priorities map "\ " to higher native thread priorities. This policy should be "\ " used with care, as sometimes it can cause performance "\ " degradation in the application and/or the entire system. On "\ " Linux this policy requires root privilege.") \ Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the text in globals.hpp mentions only Linux which seems to be not fully correct): if (geteuid() != 0) { ... } in function prio_init(). (looks like the check is not done for other platforms). However the check for root (e.g. on Linux) hinders users to set a ****lower priority**** for a thread (== increase the "niceness" level) when running as a non-root user (there might be strange ways from outside the VM with calling scripts and renice but .... ). In older JDKs (e.g. JDK8) there was a "workaround" to use for example ThreadPriorityPolicy=2 to avoid the root-check, but this is not possible any more in recent JDKs (10/11) after the range check (0,1) has been introduced for the ThreadPriorityPolicy flag (and probably the old workaround was not a good one anyway because it was undocumented). So do you think we could introduce another XX-flag ( AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) that allows using the "aggressive" mode for non-root users ? Another option would be to add another mode 2 for ThreadPriorityPolicy that documents the behavior (like mode 1 but without root-user check). If I get it right, even setting ***higher prios*** (lower niceness) is possible for non-root users on systems configured in an appropriate way (using the CAP_SYS_NICE capability). But setting lower prio / higher niceness is even possible for normal users NOW without special config, it is just disabled by the root-check which is very bad. Best regards, Matthias From aph at redhat.com Wed Dec 19 17:24:39 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 19 Dec 2018 17:24:39 +0000 Subject: RFR: 8209414: AArch64: method handle invocation does not respect JVMTI interp_only mode In-Reply-To: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> References: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> Message-ID: <908bd5e6-327e-54d0-3f7d-0111c7a061c5@redhat.com> On 12/19/18 7:31 AM, Nick Gasson (Arm Technology China) wrote: > In MethodHandles::jump_from_method_handle the test on the value of the > _interp_only_mode thread field is wrong: if this is non-zero we are > tracing or single stepping with JVMTI and should always branch to the > interpreted entry, to ensure the JVMTI method entry/exit hooks are > called. But in the AArch64 version we branch to the interpreted entry if > this value is *zero*, which is backwards. Ouch. That's my fault. This one merits a backport, I think. Is anyone keeping track of which patches must be backported to jdk8? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Wed Dec 19 19:48:41 2018 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 19 Dec 2018 14:48:41 -0500 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: <5d502248-ca1e-6877-aaed-0fb37a8695cd@oracle.com> References: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> <30496bb1-009c-0259-c039-5511834083e6@oracle.com> <11826825-4c2e-bc8f-5d4c-1765b7fa2838@oracle.com> <5d502248-ca1e-6877-aaed-0fb37a8695cd@oracle.com> Message-ID: I think this looks good. Then we don?t have to know who has already cleaned them or if they did it in the right order. Thanks Coleen > On Dec 18, 2018, at 4:30 AM, Erik ?sterlund wrote: > > Hi Coleen, > >> On 2018-12-17 22:31, coleen.phillimore at oracle.com wrote: >>> On 12/17/18 2:57 PM, Erik ?sterlund wrote: >>> Hi Dean, >>> >>>> On 2018-12-17 20:27, dean.long at oracle.com wrote: >>>>> On 12/17/18 8:56 AM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> Today, when converting a not_entrant nmethod to zombie, the sweeper first clears the IC stubs of that nmethod, and then make_zombie(). With concurrent class unloading, this ordering is a bit unfortunate. Between clearing the IC stubs and making it zombie, the concurrent GC threads may hit a timing window where they will create IC stubs for concurrently cleaning the ICs of this still is_alive() nmethod. >>>> >>>> Do we normally have to creates stubs to clean an IC? >>> >>> Yes if you clean an IC in an nmethod that is_alive(), we currently have to do that using IC stubs. I have some ideas going forward for how we can stop doing that, but that is outside of scope for now. >>> >>>> >>>>> The result is that during safepoint cleanup, when we finalize the IC stubs, we find one associated with a zombie. >>>>> >>>>> The unregistering of the nmethod from the GC will block during concurrent nmethod unloading, so clearing the IC stubs after the nmethod has become zombie is a lot more sane as there can not be any such races then. >>>>> >>>> >>>> Where is the unregistering happening? >>> >>> Inside of make_zombie() we unregister_nmethod() on CollectedHeap. In there, ZGC makes sure we wait until concurrent nmethod unloading is over before continuing. >> Can you move clean_ic_stubs() inside of make_not_entrant_or_zombie() to maintain the order dependence? > > Sure. The implication of that is that the sweeper will redundantly check for IC stubs to clear on unloaded nmethods that can't have any. But I agree that the model is more clear if it is inside of make_not_entrant_or_zombie(), and think that is worth the cost. > > New webrev: > http://cr.openjdk.java.net/~eosterlund/8215491/webrev.01/ > > Thanks for reviewing. > > /Erik > >> thanks, >> Coleen >>> >>>> >>>> Doing the clearing after it's a zombie does sound safer. >>> >>> Agreed! >>> >>> Thanks for reviewing. >>> >>> /Erik >>> >>>> dl >>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8215491 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html >>>>> >>>>> Thanks, >>>>> /Erik >>>> From erik.osterlund at oracle.com Wed Dec 19 19:56:45 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 19 Dec 2018 19:56:45 +0000 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: References: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> <30496bb1-009c-0259-c039-5511834083e6@oracle.com> <11826825-4c2e-bc8f-5d4c-1765b7fa2838@oracle.com> <5d502248-ca1e-6877-aaed-0fb37a8695cd@oracle.com> Message-ID: <080BECB8-5A6A-442F-BE1B-F4F1D168BF48@oracle.com> Hi Coleen, Agreed. Thanks for the review! /Erik > On 19 Dec 2018, at 19:48, Coleen Phillimore wrote: > > I think this looks good. Then we don?t have to know who has already cleaned them or if they did it in the right order. > > Thanks > Coleen > >> On Dec 18, 2018, at 4:30 AM, Erik ?sterlund wrote: >> >> Hi Coleen, >> >>>> On 2018-12-17 22:31, coleen.phillimore at oracle.com wrote: >>>> On 12/17/18 2:57 PM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>>>> On 2018-12-17 20:27, dean.long at oracle.com wrote: >>>>>> On 12/17/18 8:56 AM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> Today, when converting a not_entrant nmethod to zombie, the sweeper first clears the IC stubs of that nmethod, and then make_zombie(). With concurrent class unloading, this ordering is a bit unfortunate. Between clearing the IC stubs and making it zombie, the concurrent GC threads may hit a timing window where they will create IC stubs for concurrently cleaning the ICs of this still is_alive() nmethod. >>>>> >>>>> Do we normally have to creates stubs to clean an IC? >>>> >>>> Yes if you clean an IC in an nmethod that is_alive(), we currently have to do that using IC stubs. I have some ideas going forward for how we can stop doing that, but that is outside of scope for now. >>>> >>>>> >>>>>> The result is that during safepoint cleanup, when we finalize the IC stubs, we find one associated with a zombie. >>>>>> >>>>>> The unregistering of the nmethod from the GC will block during concurrent nmethod unloading, so clearing the IC stubs after the nmethod has become zombie is a lot more sane as there can not be any such races then. >>>>>> >>>>> >>>>> Where is the unregistering happening? >>>> >>>> Inside of make_zombie() we unregister_nmethod() on CollectedHeap. In there, ZGC makes sure we wait until concurrent nmethod unloading is over before continuing. >>> Can you move clean_ic_stubs() inside of make_not_entrant_or_zombie() to maintain the order dependence? >> >> Sure. The implication of that is that the sweeper will redundantly check for IC stubs to clear on unloaded nmethods that can't have any. But I agree that the model is more clear if it is inside of make_not_entrant_or_zombie(), and think that is worth the cost. >> >> New webrev: >> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.01/ >> >> Thanks for reviewing. >> >> /Erik >> >>> thanks, >>> Coleen >>>> >>>>> >>>>> Doing the clearing after it's a zombie does sound safer. >>>> >>>> Agreed! >>>> >>>> Thanks for reviewing. >>>> >>>> /Erik >>>> >>>>> dl >>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8215491 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>> > From david.holmes at oracle.com Wed Dec 19 21:21:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 07:21:30 +1000 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic Message-ID: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> Following on from the preliminary RFR: http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html I've merged with Kim's changes from 8215097. Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ Changes since previous discussion: - Replaced Kim's initial_thread_created assertion with a check of Thread::current_or_null, with an explanation - Removed NJT list addition at construction time as it's not needed after 8215097. - Moved the JavaThread call to this->exit(false); from the end of thread_main_inner to the start of post_run - as Markus suggested. This also necessitated a change to the gtest logic compared to the original as it has to override post_run() now to avoid the exit call. - Made get_thread_name_string virtual so that the JavaThread subclasses in the gtest can override it and allow their names to be seen in hs_err files. (Makes debugging so much easier when you know which thread crashed!). Testing: - Mach 5 tiers 1 - 3 - All runtime tests with: - default GC - ZGC - Shenandoah GC - gc/g1 - gc/serial - gc/epsilon - gc/shenandoah Thanks, David From david.holmes at oracle.com Wed Dec 19 22:07:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 08:07:57 +1000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: Message-ID: <5fc815e8-9d81-1589-73ec-60c80900f7aa@oracle.com> Hi Matthias, This is a tricky area and it will need quite a bit of thought. The existing priority related code is very old, intended to address very specific usecases and basically AFAIK generally completely unused for years. We actively discourage people from messing with priority. :) The BSD code was blindly copied from Linux (as was the bulk of the BSD/OS X port). I will need to refresh myself on all the gory details so this is not likely to happen until the New Year as I'll be on Christmas break from Friday until January 7. Cheers, David On 20/12/2018 3:13 am, Baesken, Matthias wrote: > > Hello , > currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 (normal, the default) and 1, > the so called "aggressive" mode : > > "1 : Aggressive. "\ > " Java thread priorities map over to the entire range of "\ > " native thread priorities. Higher Java thread priorities map "\ > " to higher native thread priorities. This policy should be "\ > " used with care, as sometimes it can cause performance "\ > " degradation in the application and/or the entire system. On "\ > " Linux this policy requires root privilege.") \ > > Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the text in globals.hpp mentions only Linux which seems to be not fully correct): > > if (geteuid() != 0) { ... } in function prio_init(). > > (looks like the check is not done for other platforms). > > However the check for root (e.g. on Linux) hinders users to set a ****lower priority**** for a thread (== increase the "niceness" level) > when running as a non-root user (there might be strange ways from outside the VM with calling scripts and renice but .... ). > > > In older JDKs (e.g. JDK8) there was a "workaround" to use for example ThreadPriorityPolicy=2 to avoid the root-check, > but this is not possible any more in recent JDKs (10/11) after the range check (0,1) has been introduced for the ThreadPriorityPolicy flag > (and probably the old workaround was not a good one anyway because it was undocumented). > > So do you think we could introduce another XX-flag ( AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) > that allows using the "aggressive" mode for non-root users ? Another option would be to add another mode 2 for ThreadPriorityPolicy > that documents the behavior (like mode 1 but without root-user check). > > If I get it right, even setting ***higher prios*** (lower niceness) is possible for non-root users on systems configured in an appropriate way > (using the CAP_SYS_NICE capability). > > But setting lower prio / higher niceness is even possible for normal users NOW without special config, it is just disabled by the root-check > which is very bad. > > Best regards, Matthias > From felix.yang at huawei.com Thu Dec 20 01:20:27 2018 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Thu, 20 Dec 2018 01:20:27 +0000 Subject: [aarch64-port-dev ] RFR: 8209414: AArch64: method handle invocation does not respect JVMTI interp_only mode In-Reply-To: <908bd5e6-327e-54d0-3f7d-0111c7a061c5@redhat.com> References: <57547711-b42d-79ff-7cfb-2e3155e7629e@arm.com> <908bd5e6-327e-54d0-3f7d-0111c7a061c5@redhat.com> Message-ID: Hi, Previously, I saw the following ones that may deserve a backport: 8191955, 8202186, 8194256 and 8195588. I will add this one to the list. Please point out if I missed anyone. Thanks, Felix > > On 12/19/18 7:31 AM, Nick Gasson (Arm Technology China) wrote: > > In MethodHandles::jump_from_method_handle the test on the value of the > > _interp_only_mode thread field is wrong: if this is non-zero we are > > tracing or single stepping with JVMTI and should always branch to the > > interpreted entry, to ensure the JVMTI method entry/exit hooks are > > called. But in the AArch64 version we branch to the interpreted entry if > > this value is *zero*, which is backwards. > > Ouch. That's my fault. > > This one merits a backport, I think. Is anyone keeping track of which > patches must be backported to jdk8? > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From dean.long at oracle.com Thu Dec 20 02:18:22 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 19 Dec 2018 18:18:22 -0800 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: References: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> <30496bb1-009c-0259-c039-5511834083e6@oracle.com> <11826825-4c2e-bc8f-5d4c-1765b7fa2838@oracle.com> <5d502248-ca1e-6877-aaed-0fb37a8695cd@oracle.com> Message-ID: <5481b9fe-13fd-70fb-408d-517097275849@oracle.com> +1 dl On 12/19/18 11:48 AM, Coleen Phillimore wrote: > I think this looks good. Then we don?t have to know who has already cleaned them or if they did it in the right order. > > Thanks > Coleen > >> On Dec 18, 2018, at 4:30 AM, Erik ?sterlund wrote: >> >> Hi Coleen, >> >>> On 2018-12-17 22:31, coleen.phillimore at oracle.com wrote: >>>> On 12/17/18 2:57 PM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>>> On 2018-12-17 20:27, dean.long at oracle.com wrote: >>>>>> On 12/17/18 8:56 AM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> Today, when converting a not_entrant nmethod to zombie, the sweeper first clears the IC stubs of that nmethod, and then make_zombie(). With concurrent class unloading, this ordering is a bit unfortunate. Between clearing the IC stubs and making it zombie, the concurrent GC threads may hit a timing window where they will create IC stubs for concurrently cleaning the ICs of this still is_alive() nmethod. >>>>> Do we normally have to creates stubs to clean an IC? >>>> Yes if you clean an IC in an nmethod that is_alive(), we currently have to do that using IC stubs. I have some ideas going forward for how we can stop doing that, but that is outside of scope for now. >>>> >>>>>> The result is that during safepoint cleanup, when we finalize the IC stubs, we find one associated with a zombie. >>>>>> >>>>>> The unregistering of the nmethod from the GC will block during concurrent nmethod unloading, so clearing the IC stubs after the nmethod has become zombie is a lot more sane as there can not be any such races then. >>>>>> >>>>> Where is the unregistering happening? >>>> Inside of make_zombie() we unregister_nmethod() on CollectedHeap. In there, ZGC makes sure we wait until concurrent nmethod unloading is over before continuing. >>> Can you move clean_ic_stubs() inside of make_not_entrant_or_zombie() to maintain the order dependence? >> Sure. The implication of that is that the sweeper will redundantly check for IC stubs to clear on unloaded nmethods that can't have any. But I agree that the model is more clear if it is inside of make_not_entrant_or_zombie(), and think that is worth the cost. >> >> New webrev: >> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.01/ >> >> Thanks for reviewing. >> >> /Erik >> >>> thanks, >>> Coleen >>>>> Doing the clearing after it's a zombie does sound safer. >>>> Agreed! >>>> >>>> Thanks for reviewing. >>>> >>>> /Erik >>>> >>>>> dl >>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8215491 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html >>>>>> >>>>>> Thanks, >>>>>> /Erik From david.holmes at oracle.com Thu Dec 20 05:38:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 15:38:18 +1000 Subject: Truncated stacks Message-ID: <7434c305-becd-fb7a-f830-992a2696650d@oracle.com> We've noticed a problem investigating bug reports involving monitor lock/unlock. It seems that the stacks for threads in hs_err reports, in jstack -mixed output and even in gdb are truncated very high up e.g. Stack: [0x00000041adef0000,0x00000041adff0000] Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [jvm.dll+0xb49bd1] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:369) V [jvm.dll+0xd67860] VMError::report+0xe80 (vmerror.cpp:698) V [jvm.dll+0xd690b4] VMError::report_and_die+0x854 (vmerror.cpp:1463) V [jvm.dll+0xd69774] VMError::report_and_die+0x64 (vmerror.cpp:1287) V [jvm.dll+0x57be7e] report_vm_error+0x7e (debug.cpp:232) V [jvm.dll+0xb21db4] ObjectMonitor::exit+0xf4 (objectmonitor.cpp:1025) V [jvm.dll+0xc0f08a] SharedRuntime::complete_monitor_unlocking_C+0x17a (sharedruntime.cpp:2062) C 0x0000004187bfcf5e or: Thread 1005 (Thread 0x7f30bb1ff700 (LWP 7275)): #0 0x00007f30f950d705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f30f7b2d033 in os::PlatformEvent::park() () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #2 0x00007f30f7af7d2d in ObjectMonitor::EnterI(Thread*) () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #3 0x00007f30f7af9a62 in ObjectMonitor::enter(Thread*) [clone .part.90] () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #4 0x00007f30f7c9bcd2 in ObjectSynchronizer::fast_enter(Handle, BasicLock*, bool, Thread*) () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #5 0x00007f30f7c02584 in SharedRuntime::complete_monitor_locking_C(oopDesc*, BasicLock*, JavaThread*) () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #6 0x00007f30ddde2228 in ?? () #7 0x0000000638200fc8 in ?? () #8 0x00007f30e57e1330 in ?? () #9 0x00000001000003e8 in ?? () #10 0x0000000638200920 in ?? () #11 0x000000080113fa80 in ?? () #12 0x0000000638200fe8 in ?? () #13 0xc70405d300000000 in ?? () #14 0x0000000638200fd8 in ?? () #15 0x0000000801146328 in ?? () #16 0x00000006382014d8 in ?? () #17 0x0000000638202e98 in ?? () #18 0x0000000638202eb0 in ?? () #19 0xc704029b382317f8 in ?? () #20 0x0000000000000000 in ?? () Anyone have any ideas about what have broken things? I haven't had time to try and determine how long this has been occurring. Thanks, David From david.holmes at oracle.com Thu Dec 20 06:08:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 16:08:53 +1000 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> Message-ID: <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> Hi Robbin, Looks good, small doc follow up below ... On 18/12/2018 9:17 pm, Robbin Ehn wrote: > Hi, here is a new version. > > Inc: > http://cr.openjdk.java.net/~rehn/8214271/3/inc/webrev/ src/hotspot/share/utilities/waitBarrier.hpp I like the idea of there being an "owner" thread but it needs to be documented. Suggest: // An armed WaitBarrier prevents threads from advancing until the // barrier is disarmed and the waiting threads woken. The barrier is // armed by setting a non-zero value - the tag. + // When the WaitBarrier is created, a thread is designated the owner + // and is the thread that should arm/disarm/wake the WaitBarrier. In + // debug builds this is enforced. // // Expected Usage: ! // - Arming/owning thread: I would also like to see a small section on usage errors: // It is a usage error to: // - call wake on a barrier that is still armed // - call arm on a barrier that is already armed // - call disarm on a barrier that is not armed // - arm with the same tag as last used // Usage errors are checked in debug builds but may be ignored otherwise. Sound reasonable? Otherwise this all looks good! Thanks, David ----- > Full: > http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ > > Thanks, Robbin > > On 11/23/18 5:55 PM, Robbin Ehn wrote: >> Forgot RFR in subject. >> >> /Robbin >> >> On 2018-11-23 17:51, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> When a safepoint is ended we need a way to get back to 100% >>> utilization as fast >>> as possible. 100% utilization means no idle cpu in the system if >>> there is a >>> JavaThread that could be executed. The traditional ways to wake many, >>> e.g. >>> semaphore, pthread_cond, is not implemented with a single syscall >>> instead they >>> typical do one syscall per thread to wake. >>> >>> This change-set contains that primitive, the WaitBarrier, and a gtest >>> for it. >>> No actual users, which is in coming patches. >>> >>> The WaitBarrier solves by doing a cooperative semaphore posting, >>> threads woken >>> will also post. On Linux we can instead directly use a futex and with >>> one >>> syscall wake all. Depending on how many threads and cpus the >>> performance vary, >>> but a good utilization of the machine, just on the edge of saturated, >>> the time to reach 100% utilization is around 3 times faster with the >>> WaitBarrier (where futex is faster than semaphore). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>> >>> Passes 100 iterations of gtest on our platforms, both fastdebug and >>> release. >>> And have been stable when used in safepoints (t1-8) (coming patches). >>> >>> Thanks, Robbin From erik.osterlund at oracle.com Thu Dec 20 08:45:19 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 20 Dec 2018 08:45:19 +0000 Subject: RFR: 8215491: ICStubInterface::finalize finds zombie nmethod with ZGC concurrent class unloading In-Reply-To: <5481b9fe-13fd-70fb-408d-517097275849@oracle.com> References: <92c66358-f119-6130-c22f-fda8ae4a922a@oracle.com> <30496bb1-009c-0259-c039-5511834083e6@oracle.com> <11826825-4c2e-bc8f-5d4c-1765b7fa2838@oracle.com> <5d502248-ca1e-6877-aaed-0fb37a8695cd@oracle.com> <5481b9fe-13fd-70fb-408d-517097275849@oracle.com> Message-ID: Hi Dean, Thanks for the review. /Erik > On 20 Dec 2018, at 02:18, dean.long at oracle.com wrote: > > +1 > > dl > >> On 12/19/18 11:48 AM, Coleen Phillimore wrote: >> I think this looks good. Then we don?t have to know who has already cleaned them or if they did it in the right order. >> Thanks >> Coleen >> >>> On Dec 18, 2018, at 4:30 AM, Erik ?sterlund wrote: >>> >>> Hi Coleen, >>> >>>>> On 2018-12-17 22:31, coleen.phillimore at oracle.com wrote: >>>>> On 12/17/18 2:57 PM, Erik ?sterlund wrote: >>>>> Hi Dean, >>>>> >>>>>>> On 2018-12-17 20:27, dean.long at oracle.com wrote: >>>>>>> On 12/17/18 8:56 AM, Erik ?sterlund wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Today, when converting a not_entrant nmethod to zombie, the sweeper first clears the IC stubs of that nmethod, and then make_zombie(). With concurrent class unloading, this ordering is a bit unfortunate. Between clearing the IC stubs and making it zombie, the concurrent GC threads may hit a timing window where they will create IC stubs for concurrently cleaning the ICs of this still is_alive() nmethod. >>>>>> Do we normally have to creates stubs to clean an IC? >>>>> Yes if you clean an IC in an nmethod that is_alive(), we currently have to do that using IC stubs. I have some ideas going forward for how we can stop doing that, but that is outside of scope for now. >>>>> >>>>>>> The result is that during safepoint cleanup, when we finalize the IC stubs, we find one associated with a zombie. >>>>>>> >>>>>>> The unregistering of the nmethod from the GC will block during concurrent nmethod unloading, so clearing the IC stubs after the nmethod has become zombie is a lot more sane as there can not be any such races then. >>>>>>> >>>>>> Where is the unregistering happening? >>>>> Inside of make_zombie() we unregister_nmethod() on CollectedHeap. In there, ZGC makes sure we wait until concurrent nmethod unloading is over before continuing. >>>> Can you move clean_ic_stubs() inside of make_not_entrant_or_zombie() to maintain the order dependence? >>> Sure. The implication of that is that the sweeper will redundantly check for IC stubs to clear on unloaded nmethods that can't have any. But I agree that the model is more clear if it is inside of make_not_entrant_or_zombie(), and think that is worth the cost. >>> >>> New webrev: >>> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.01/ >>> >>> Thanks for reviewing. >>> >>> /Erik >>> >>>> thanks, >>>> Coleen >>>>>> Doing the clearing after it's a zombie does sound safer. >>>>> Agreed! >>>>> >>>>> Thanks for reviewing. >>>>> >>>>> /Erik >>>>> >>>>>> dl >>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8215491 >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8215491/webrev.00/index.html >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik > From matthias.baesken at sap.com Thu Dec 20 09:21:53 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 20 Dec 2018 09:21:53 +0000 Subject: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam Message-ID: Hello, while looking at the thread prio + scheduling handling in hotspot, I noticed that at one place we seem to handle pthread_getschedparam in a wrong way. According to the API documentation, pthread_getschedparam returns an int which is 0 when successful, otherwise it is a value != 0. See https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/pthread_setschedparam.3.html "If successful, these functions return 0. Otherwise, an error number is returned to indicate the error." At some places the return code of pthread_getschedparam is checked in the OpenJDK hotspot sources, however at os::get_native_priority (os_bsd.cpp) it is not done, but should be added. The current coding handles the function in a getpriority-like way (which might or might not work), but at least it is not what is documented in the API doc and done at other places in the coding. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8215707 http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.0/ Thanks, Matthias From david.holmes at oracle.com Thu Dec 20 10:45:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 20:45:09 +1000 Subject: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam In-Reply-To: References: Message-ID: Hi Matthias, On 20/12/2018 7:21 pm, Baesken, Matthias wrote: > Hello, while looking at the thread prio + scheduling handling in hotspot, I noticed that at one place we seem to handle pthread_getschedparam in a wrong way. Note this code is likely never exercised due to the default ThreadPriorityPolicy of 0. It is entirely possible no one has ever really used it. That seems more likely given that the code is incorrect: pthread_getschedparam(pthread_self(), &policy, &sp); this always gets the priority of the current thread, but the method need not be called on the current thread! > According to the API documentation, pthread_getschedparam returns an int which is 0 when successful, otherwise it is a value != 0. > See > > https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/pthread_setschedparam.3.html > > > "If successful, these functions return 0. Otherwise, an error number is returned to indicate the error." The only possible error for pthread_getschedparam is ESRCH if the thread id is not valid - though that is impossible with the current code using pthread_self()! So the current code - as written - did not need to check the return value. If you fix the real bug then you should check the return value. I've also highly suspicious of using this API because the man page claims that the scheduling policy can only be SCHED_FIFO or SCHED_RR - that surprised me because they are both fixed-priority preemptive scheduling algorithms normally reserved for "real-time". There is no mention of SCHED_OTHER which would be the normal non-realtime scheduling mode. So how this function can return only SCHED_FIFO or SCHED_RR if you've never actually set that scheduling mode seems a problem! maybe the manpage is just wrong. > > At some places the return code of pthread_getschedparam is checked in the OpenJDK hotspot sources, however at os::get_native_priority (os_bsd.cpp) it is not done, but should be added. > The current coding handles the function in a getpriority-like way (which might or might not work), but at least it is not what is documented in the API doc and done at other places in the coding. > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8215707 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.0/ ! int res = pthread_getschedparam(pthread_self(), &policy, &sp); *priority_ptr = sp.sched_priority; + if (res != 0) { + return OS_ERR; + } else { + return OS_OK; + } You should only set *priority_ptr if pthread_getschedParam() was successful! Seriously you are walking through a minefield with this scheduling stuff, especially on OS X. Cheers, David ----- > > Thanks, Matthias > From david.holmes at oracle.com Thu Dec 20 11:05:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 21:05:39 +1000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: Message-ID: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> Hi Matthias, The more I think about this the more I see it as a huge can of worms. There are very, very, limited usecases for managing the priority of individual threads within a running Java application. Adjusting the process priority/nice-ness is effective and much simpler. Cheers, David On 20/12/2018 3:13 am, Baesken, Matthias wrote: > > Hello , > currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 (normal, the default) and 1, > the so called "aggressive" mode : > > "1 : Aggressive. "\ > " Java thread priorities map over to the entire range of "\ > " native thread priorities. Higher Java thread priorities map "\ > " to higher native thread priorities. This policy should be "\ > " used with care, as sometimes it can cause performance "\ > " degradation in the application and/or the entire system. On "\ > " Linux this policy requires root privilege.") \ > > Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the text in globals.hpp mentions only Linux which seems to be not fully correct): > > if (geteuid() != 0) { ... } in function prio_init(). > > (looks like the check is not done for other platforms). > > However the check for root (e.g. on Linux) hinders users to set a ****lower priority**** for a thread (== increase the "niceness" level) > when running as a non-root user (there might be strange ways from outside the VM with calling scripts and renice but .... ). > > > In older JDKs (e.g. JDK8) there was a "workaround" to use for example ThreadPriorityPolicy=2 to avoid the root-check, > but this is not possible any more in recent JDKs (10/11) after the range check (0,1) has been introduced for the ThreadPriorityPolicy flag > (and probably the old workaround was not a good one anyway because it was undocumented). > > So do you think we could introduce another XX-flag ( AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) > that allows using the "aggressive" mode for non-root users ? Another option would be to add another mode 2 for ThreadPriorityPolicy > that documents the behavior (like mode 1 but without root-user check). > > If I get it right, even setting ***higher prios*** (lower niceness) is possible for non-root users on systems configured in an appropriate way > (using the CAP_SYS_NICE capability). > > But setting lower prio / higher niceness is even possible for normal users NOW without special config, it is just disabled by the root-check > which is very bad. > > Best regards, Matthias > From robbin.ehn at oracle.com Thu Dec 20 11:53:27 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 20 Dec 2018 12:53:27 +0100 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> Message-ID: Hi David, Looks good, thanks! /Robbin On 12/19/18 10:21 PM, David Holmes wrote: > Following on from the preliminary RFR: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html > > I've merged with Kim's changes from 8215097. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 > Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ > > Changes since previous discussion: > > - Replaced Kim's initial_thread_created assertion with a check of > Thread::current_or_null, with an explanation > > - Removed NJT list addition at construction time as it's not needed after 8215097. > > - Moved the JavaThread call to this->exit(false); from the end of > thread_main_inner to the start of post_run - as Markus suggested. This also > necessitated a change to the gtest logic compared to the original as it has to > override post_run() now to avoid the exit call. > > - Made get_thread_name_string virtual so that the JavaThread subclasses in the > gtest can override it and allow their names to be seen in hs_err files. (Makes > debugging so much easier when you know which thread crashed!). > > Testing: > ?- Mach 5 tiers 1 - 3 > ?- All runtime tests with: > ?? - default GC > ?? - ZGC > ?? - Shenandoah GC > - gc/g1 > - gc/serial > - gc/epsilon > - gc/shenandoah > > Thanks, > David From matthias.baesken at sap.com Thu Dec 20 11:56:35 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 20 Dec 2018 11:56:35 +0000 Subject: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam In-Reply-To: References: Message-ID: > Note this code is likely never exercised due to the default > ThreadPriorityPolicy of 0. Hi David, ThreadPriorityPolicy=1 is a valid setting / product flag. You find quite a few people setting ThreadPriorityPolicy when you search for it, so the coding should better be fixed I guess . > > That seems more likely given that the code is incorrect: > > pthread_getschedparam(pthread_self(), &policy, &sp); > > this always gets the priority of the current thread, but the method need > not be called on the current thread! Yes , true ! Same for os::set_native_priority in os_bsd.cpp , there pthread_self() is also used instead of the real Thread . 2338OSReturn os::set_native_priority(Thread* thread, int newpri) { 2339 if (!UseThreadPriorities || ThreadPriorityPolicy == 0) return OS_OK; 2340 .... 2346#elif defined(__APPLE__) || defined(__NetBSD__) 2347 struct sched_param sp; 2348 int policy; 2349 pthread_t self = pthread_self(); 2350 2351 if (pthread_getschedparam(self, &policy, &sp) != 0) { 2352 return OS_ERR; 2353 } Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 20. Dezember 2018 11:45 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: RFR [XS] : 8215707: [macosx] check return code of > pthread_getschedparam > > Hi Matthias, > > On 20/12/2018 7:21 pm, Baesken, Matthias wrote: > > Hello, while looking at the thread prio + scheduling handling in hotspot, I > noticed that at one place we seem to handle pthread_getschedparam in a > wrong way. > > Note this code is likely never exercised due to the default > ThreadPriorityPolicy of 0. It is entirely possible no one has ever > really used it. That seems more likely given that the code is incorrect: > > pthread_getschedparam(pthread_self(), &policy, &sp); > > this always gets the priority of the current thread, but the method need > not be called on the current thread! > > > According to the API documentation, pthread_getschedparam returns an > int which is 0 when successful, otherwise it is a value != 0. > > See > > > > > https://developer.apple.com/library/archive/documentation/System/Conce > ptual/ManPages_iPhoneOS/man3/pthread_setschedparam.3.html > > > > > > "If successful, these functions return 0. Otherwise, an error number is > returned to indicate the error." > > The only possible error for pthread_getschedparam is ESRCH if the thread > id is not valid - though that is impossible with the current code using > pthread_self()! So the current code - as written - did not need to check > the return value. If you fix the real bug then you should check the > return value. > > I've also highly suspicious of using this API because the man page > claims that the scheduling policy can only be SCHED_FIFO or SCHED_RR - > that surprised me because they are both fixed-priority preemptive > scheduling algorithms normally reserved for "real-time". There is no > mention of SCHED_OTHER which would be the normal non-realtime > scheduling > mode. So how this function can return only SCHED_FIFO or SCHED_RR if > you've never actually set that scheduling mode seems a problem! maybe > the manpage is just wrong. > > > > > At some places the return code of pthread_getschedparam is checked in > the OpenJDK hotspot sources, however at os::get_native_priority > (os_bsd.cpp) it is not done, but should be added. > > The current coding handles the function in a getpriority-like way (which > might or might not work), but at least it is not what is documented in the API > doc and done at other places in the coding. > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8215707 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.0/ > > ! int res = pthread_getschedparam(pthread_self(), &policy, &sp); > *priority_ptr = sp.sched_priority; > + if (res != 0) { > + return OS_ERR; > + } else { > + return OS_OK; > + } > > You should only set *priority_ptr if pthread_getschedParam() was > successful! > > Seriously you are walking through a minefield with this scheduling > stuff, especially on OS X. > > Cheers, > David > ----- > > > > > Thanks, Matthias > > From martin.doerr at sap.com Thu Dec 20 11:57:44 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 20 Dec 2018 11:57:44 +0000 Subject: Truncated stacks In-Reply-To: <7434c305-becd-fb7a-f830-992a2696650d@oracle.com> References: <7434c305-becd-fb7a-f830-992a2696650d@oracle.com> Message-ID: Hi David, is only this unlock call affected? I think regular Java stack walking doesn't work by design. C2 calls SharedRuntime::complete_monitor_unlocking_C without setting last Java frame (PhaseMacroExpand::expand_unlock_node). So the last Java frame based walking won't work. Native stack walking may or may not work depending on platform and build options. On x86, you will probably need backlinks to find a frame which belongs to generated code below a C++ frame. Is only x86 affected? Best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of David Holmes Sent: Donnerstag, 20. Dezember 2018 06:38 To: hotspot-dev at openjdk.java.net Subject: Truncated stacks We've noticed a problem investigating bug reports involving monitor lock/unlock. It seems that the stacks for threads in hs_err reports, in jstack -mixed output and even in gdb are truncated very high up e.g. Stack: [0x00000041adef0000,0x00000041adff0000] Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [jvm.dll+0xb49bd1] os::platform_print_native_stack+0xf1 (os_windows_x86.cpp:369) V [jvm.dll+0xd67860] VMError::report+0xe80 (vmerror.cpp:698) V [jvm.dll+0xd690b4] VMError::report_and_die+0x854 (vmerror.cpp:1463) V [jvm.dll+0xd69774] VMError::report_and_die+0x64 (vmerror.cpp:1287) V [jvm.dll+0x57be7e] report_vm_error+0x7e (debug.cpp:232) V [jvm.dll+0xb21db4] ObjectMonitor::exit+0xf4 (objectmonitor.cpp:1025) V [jvm.dll+0xc0f08a] SharedRuntime::complete_monitor_unlocking_C+0x17a (sharedruntime.cpp:2062) C 0x0000004187bfcf5e or: Thread 1005 (Thread 0x7f30bb1ff700 (LWP 7275)): #0 0x00007f30f950d705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f30f7b2d033 in os::PlatformEvent::park() () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #2 0x00007f30f7af7d2d in ObjectMonitor::EnterI(Thread*) () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #3 0x00007f30f7af9a62 in ObjectMonitor::enter(Thread*) [clone .part.90] () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #4 0x00007f30f7c9bcd2 in ObjectSynchronizer::fast_enter(Handle, BasicLock*, bool, Thread*) () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #5 0x00007f30f7c02584 in SharedRuntime::complete_monitor_locking_C(oopDesc*, BasicLock*, JavaThread*) () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so #6 0x00007f30ddde2228 in ?? () #7 0x0000000638200fc8 in ?? () #8 0x00007f30e57e1330 in ?? () #9 0x00000001000003e8 in ?? () #10 0x0000000638200920 in ?? () #11 0x000000080113fa80 in ?? () #12 0x0000000638200fe8 in ?? () #13 0xc70405d300000000 in ?? () #14 0x0000000638200fd8 in ?? () #15 0x0000000801146328 in ?? () #16 0x00000006382014d8 in ?? () #17 0x0000000638202e98 in ?? () #18 0x0000000638202eb0 in ?? () #19 0xc704029b382317f8 in ?? () #20 0x0000000000000000 in ?? () Anyone have any ideas about what have broken things? I haven't had time to try and determine how long this has been occurring. Thanks, David From david.holmes at oracle.com Thu Dec 20 12:07:55 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 22:07:55 +1000 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> Message-ID: <01738b62-eb54-38a7-1129-86ad4da9640f@oracle.com> Thanks for looking at this Robbin! David On 20/12/2018 9:53 pm, Robbin Ehn wrote: > Hi David, > > Looks good, thanks! > > /Robbin > > On 12/19/18 10:21 PM, David Holmes wrote: >> Following on from the preliminary RFR: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html >> >> >> I've merged with Kim's changes from 8215097. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ >> >> Changes since previous discussion: >> >> - Replaced Kim's initial_thread_created assertion with a check of >> Thread::current_or_null, with an explanation >> >> - Removed NJT list addition at construction time as it's not needed >> after 8215097. >> >> - Moved the JavaThread call to this->exit(false); from the end of >> thread_main_inner to the start of post_run - as Markus suggested. This >> also necessitated a change to the gtest logic compared to the original >> as it has to override post_run() now to avoid the exit call. >> >> - Made get_thread_name_string virtual so that the JavaThread >> subclasses in the gtest can override it and allow their names to be >> seen in hs_err files. (Makes debugging so much easier when you know >> which thread crashed!). >> >> Testing: >> ??- Mach 5 tiers 1 - 3 >> ??- All runtime tests with: >> ??? - default GC >> ??? - ZGC >> ??? - Shenandoah GC >> - gc/g1 >> - gc/serial >> - gc/epsilon >> - gc/shenandoah >> >> Thanks, >> David From robbin.ehn at oracle.com Thu Dec 20 12:10:40 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 20 Dec 2018 13:10:40 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> Message-ID: Hi David, On 12/20/18 7:08 AM, David Holmes wrote: > Hi Robbin, > > Looks good, small doc follow up below ... Thanks! > Sound reasonable? Yes, I also added the comment from my other mail, let me know what you think. Inc: http://cr.openjdk.java.net/~rehn/8214271/4/inc/webrev/ Full: http://cr.openjdk.java.net/~rehn/8214271/4/full/webrev/ /Robbin > > Otherwise this all looks good! > > Thanks, > David > ----- > > >> Full: >> http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ >> >> Thanks, Robbin >> >> On 11/23/18 5:55 PM, Robbin Ehn wrote: >>> Forgot RFR in subject. >>> >>> /Robbin >>> >>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> When a safepoint is ended we need a way to get back to 100% utilization as fast >>>> as possible. 100% utilization means no idle cpu in the system if there is a >>>> JavaThread that could be executed. The traditional ways to wake many, e.g. >>>> semaphore, pthread_cond, is not implemented with a single syscall instead they >>>> typical do one syscall per thread to wake. >>>> >>>> This change-set contains that primitive, the WaitBarrier, and a gtest for it. >>>> No actual users, which is in coming patches. >>>> >>>> The WaitBarrier solves by doing a cooperative semaphore posting, threads woken >>>> will also post. On Linux we can instead directly use a futex and with one >>>> syscall wake all. Depending on how many threads and cpus the performance vary, >>>> but a good utilization of the machine, just on the edge of saturated, the >>>> time to reach 100% utilization is around 3 times faster with the WaitBarrier >>>> (where futex is faster than semaphore). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>> >>>> Passes 100 iterations of gtest on our platforms, both fastdebug and release. >>>> And have been stable when used in safepoints (t1-8) (coming patches). >>>> >>>> Thanks, Robbin From david.holmes at oracle.com Thu Dec 20 12:18:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 22:18:37 +1000 Subject: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam In-Reply-To: References: Message-ID: <58f42910-8a3f-f403-51d0-5022a8210ee9@oracle.com> On 20/12/2018 9:56 pm, Baesken, Matthias wrote: >> Note this code is likely never exercised due to the default >> ThreadPriorityPolicy of 0. > > Hi David, ThreadPriorityPolicy=1 is a valid setting / product flag. Yes it is. That doesn't mean it is actually used to any great extent. > You find quite a few people setting ThreadPriorityPolicy when you search for it, so the coding should better be fixed I guess . Quite a few? I could only find a couple (Cassandra, Gatling) and they were on Linux. I was surprised to find how many people knew about the bug that allowed the root check to be by-passed. The fact this code is incorrect on OS X indicates to me it is not actually being used - or by chance priority is only ever changed in the current thread. >> >> That seems more likely given that the code is incorrect: >> >> pthread_getschedparam(pthread_self(), &policy, &sp); >> >> this always gets the priority of the current thread, but the method need >> not be called on the current thread! > > Yes , true ! > Same for os::set_native_priority in os_bsd.cpp , there pthread_self() is also used instead of the real Thread . > > 2338OSReturn os::set_native_priority(Thread* thread, int newpri) { > 2339 if (!UseThreadPriorities || ThreadPriorityPolicy == 0) return OS_OK; > 2340 > .... > 2346#elif defined(__APPLE__) || defined(__NetBSD__) > 2347 struct sched_param sp; > 2348 int policy; > 2349 pthread_t self = pthread_self(); > 2350 > 2351 if (pthread_getschedparam(self, &policy, &sp) != 0) { > 2352 return OS_ERR; > 2353 } If you intend fixing this probably best to modify the bug synopsis and description. Cheers, David > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 20. Dezember 2018 11:45 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Subject: Re: RFR [XS] : 8215707: [macosx] check return code of >> pthread_getschedparam >> >> Hi Matthias, >> >> On 20/12/2018 7:21 pm, Baesken, Matthias wrote: >>> Hello, while looking at the thread prio + scheduling handling in hotspot, I >> noticed that at one place we seem to handle pthread_getschedparam in a >> wrong way. >> >> Note this code is likely never exercised due to the default >> ThreadPriorityPolicy of 0. It is entirely possible no one has ever >> really used it. That seems more likely given that the code is incorrect: >> >> pthread_getschedparam(pthread_self(), &policy, &sp); >> >> this always gets the priority of the current thread, but the method need >> not be called on the current thread! >> >>> According to the API documentation, pthread_getschedparam returns an >> int which is 0 when successful, otherwise it is a value != 0. >>> See >>> >>> >> https://developer.apple.com/library/archive/documentation/System/Conce >> ptual/ManPages_iPhoneOS/man3/pthread_setschedparam.3.html >>> >>> >>> "If successful, these functions return 0. Otherwise, an error number is >> returned to indicate the error." >> >> The only possible error for pthread_getschedparam is ESRCH if the thread >> id is not valid - though that is impossible with the current code using >> pthread_self()! So the current code - as written - did not need to check >> the return value. If you fix the real bug then you should check the >> return value. >> >> I've also highly suspicious of using this API because the man page >> claims that the scheduling policy can only be SCHED_FIFO or SCHED_RR - >> that surprised me because they are both fixed-priority preemptive >> scheduling algorithms normally reserved for "real-time". There is no >> mention of SCHED_OTHER which would be the normal non-realtime >> scheduling >> mode. So how this function can return only SCHED_FIFO or SCHED_RR if >> you've never actually set that scheduling mode seems a problem! maybe >> the manpage is just wrong. >> >>> >>> At some places the return code of pthread_getschedparam is checked in >> the OpenJDK hotspot sources, however at os::get_native_priority >> (os_bsd.cpp) it is not done, but should be added. >>> The current coding handles the function in a getpriority-like way (which >> might or might not work), but at least it is not what is documented in the API >> doc and done at other places in the coding. >>> >>> >>> Bug/webrev : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8215707 >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.0/ >> >> ! int res = pthread_getschedparam(pthread_self(), &policy, &sp); >> *priority_ptr = sp.sched_priority; >> + if (res != 0) { >> + return OS_ERR; >> + } else { >> + return OS_OK; >> + } >> >> You should only set *priority_ptr if pthread_getschedParam() was >> successful! >> >> Seriously you are walking through a minefield with this scheduling >> stuff, especially on OS X. >> >> Cheers, >> David >> ----- >> >>> >>> Thanks, Matthias >>> From david.holmes at oracle.com Thu Dec 20 12:21:47 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 22:21:47 +1000 Subject: Truncated stacks In-Reply-To: References: <7434c305-becd-fb7a-f830-992a2696650d@oracle.com> Message-ID: <82a351f6-3cb5-a96c-44fe-0c60ff672d23@oracle.com> Hi Martin, On 20/12/2018 9:57 pm, Doerr, Martin wrote: > Hi David, > > is only this unlock call affected? No - the first example is unlock, the second is lock. > I think regular Java stack walking doesn't work by design. C2 calls SharedRuntime::complete_monitor_unlocking_C without setting last Java frame (PhaseMacroExpand::expand_unlock_node). So the last Java frame based walking won't work. That's interesting, I wonder why? And can we fix that? > Native stack walking may or may not work depending on platform and build options. On x86, you will probably need backlinks to find a frame which belongs to generated code below a C++ frame. Is only x86 affected? Both bugs were reported on x86 and that's all I've looked at so far. I may have to whip up a simple test to check the behaviour more broadly. Thanks, David > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of David Holmes > Sent: Donnerstag, 20. Dezember 2018 06:38 > To: hotspot-dev at openjdk.java.net > Subject: Truncated stacks > > We've noticed a problem investigating bug reports involving monitor > lock/unlock. It seems that the stacks for threads in hs_err reports, in > jstack -mixed output and even in gdb are truncated very high up e.g. > > Stack: [0x00000041adef0000,0x00000041adff0000] > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, > C=native code) > V [jvm.dll+0xb49bd1] os::platform_print_native_stack+0xf1 > (os_windows_x86.cpp:369) > V [jvm.dll+0xd67860] VMError::report+0xe80 (vmerror.cpp:698) > V [jvm.dll+0xd690b4] VMError::report_and_die+0x854 (vmerror.cpp:1463) > V [jvm.dll+0xd69774] VMError::report_and_die+0x64 (vmerror.cpp:1287) > V [jvm.dll+0x57be7e] report_vm_error+0x7e (debug.cpp:232) > V [jvm.dll+0xb21db4] ObjectMonitor::exit+0xf4 (objectmonitor.cpp:1025) > V [jvm.dll+0xc0f08a] SharedRuntime::complete_monitor_unlocking_C+0x17a > (sharedruntime.cpp:2062) > C 0x0000004187bfcf5e > > > or: > > Thread 1005 (Thread 0x7f30bb1ff700 (LWP 7275)): > #0 0x00007f30f950d705 in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/libpthread.so.0 > #1 0x00007f30f7b2d033 in os::PlatformEvent::park() () from > /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #2 0x00007f30f7af7d2d in ObjectMonitor::EnterI(Thread*) () from > /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #3 0x00007f30f7af9a62 in ObjectMonitor::enter(Thread*) [clone .part.90] > () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #4 0x00007f30f7c9bcd2 in ObjectSynchronizer::fast_enter(Handle, > BasicLock*, bool, Thread*) () from > /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #5 0x00007f30f7c02584 in > SharedRuntime::complete_monitor_locking_C(oopDesc*, BasicLock*, > JavaThread*) () > from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #6 0x00007f30ddde2228 in ?? () > #7 0x0000000638200fc8 in ?? () > #8 0x00007f30e57e1330 in ?? () > #9 0x00000001000003e8 in ?? () > #10 0x0000000638200920 in ?? () > #11 0x000000080113fa80 in ?? () > #12 0x0000000638200fe8 in ?? () > #13 0xc70405d300000000 in ?? () > #14 0x0000000638200fd8 in ?? () > #15 0x0000000801146328 in ?? () > #16 0x00000006382014d8 in ?? () > #17 0x0000000638202e98 in ?? () > #18 0x0000000638202eb0 in ?? () > #19 0xc704029b382317f8 in ?? () > #20 0x0000000000000000 in ?? () > > Anyone have any ideas about what have broken things? I haven't had time > to try and determine how long this has been occurring. > > Thanks, > David > From david.holmes at oracle.com Thu Dec 20 12:26:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 22:26:43 +1000 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> Message-ID: <2a2679cc-b0e0-f8d0-7336-8666e1a42950@oracle.com> On 20/12/2018 10:10 pm, Robbin Ehn wrote: > Hi David, > > On 12/20/18 7:08 AM, David Holmes wrote: >> Hi Robbin, >> >> Looks good, small doc follow up below ... > > Thanks! > >> Sound reasonable? > > Yes, I also added the comment from my other mail, let me know what you > think. + // Guarantees not to return until disarm() is called, + // if called with currently armed tag (otherwise returns immediately). + // Implementation must guarantee no spurious wakeups. // Guarantees to return if disarm() and wake() is called. Should the first line say "disarm() and wake()" or just "wake()"? s/Implementation/Implementations/ The fourth line is no longer needed. Thanks, David > Inc: > http://cr.openjdk.java.net/~rehn/8214271/4/inc/webrev/ > > Full: > http://cr.openjdk.java.net/~rehn/8214271/4/full/webrev/ > > /Robbin > >> >> Otherwise this all looks good! >> >> Thanks, >> David >> ----- >> >> >>> Full: >>> http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ >>> >>> Thanks, Robbin >>> >>> On 11/23/18 5:55 PM, Robbin Ehn wrote: >>>> Forgot RFR in subject. >>>> >>>> /Robbin >>>> >>>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> When a safepoint is ended we need a way to get back to 100% >>>>> utilization as fast >>>>> as possible. 100% utilization means no idle cpu in the system if >>>>> there is a >>>>> JavaThread that could be executed. The traditional ways to wake >>>>> many, e.g. >>>>> semaphore, pthread_cond, is not implemented with a single syscall >>>>> instead they >>>>> typical do one syscall per thread to wake. >>>>> >>>>> This change-set contains that primitive, the WaitBarrier, and a >>>>> gtest for it. >>>>> No actual users, which is in coming patches. >>>>> >>>>> The WaitBarrier solves by doing a cooperative semaphore posting, >>>>> threads woken >>>>> will also post. On Linux we can instead directly use a futex and >>>>> with one >>>>> syscall wake all. Depending on how many threads and cpus the >>>>> performance vary, >>>>> but a good utilization of the machine, just on the edge of >>>>> saturated, the time to reach 100% utilization is around 3 times >>>>> faster with the WaitBarrier (where futex is faster than semaphore). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>>> >>>>> CR: >>>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>>> >>>>> Passes 100 iterations of gtest on our platforms, both fastdebug and >>>>> release. >>>>> And have been stable when used in safepoints (t1-8) (coming patches). >>>>> >>>>> Thanks, Robbin From glaubitz at physik.fu-berlin.de Thu Dec 20 13:00:44 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Thu, 20 Dec 2018 14:00:44 +0100 Subject: Zero JVM segfaulting on linux-sparc Message-ID: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> Hi! There has been a regression in the Zero JVM on linux-sparc for a while now which I haven't had the time yet to debug. The backtrace looks as follows: (gdb) r --version Starting program: /home/glaubitz/jdk/build/linux-sparcv9-zero-release/jdk/bin/java --version [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/sparc64-linux-gnu/libthread_db.so.1". [New Thread 0xfff800010134b900 (LWP 57786)] Thread 2 "java" received signal SIGBUS, Bus error. [Switching to Thread 0xfff800010134b900 (LWP 57786)] 0xfff8000100c3152c in SharedRuntime::generate_stubs () at /home/glaubitz/jdk/src/hotspot/share/runtime/sharedRuntime.cpp:107 107 _resolve_static_call_blob = generate_resolve_blob(CAST_FROM_FN_PTR(address, SharedRuntime::resolve_static_call_C), "resolve_static_call"); (gdb) bt #0 0xfff8000100c3152c in SharedRuntime::generate_stubs () at /home/glaubitz/jdk/src/hotspot/share/runtime/sharedRuntime.cpp:107 #1 0xfff8000100a32e1c in init_globals () at /home/glaubitz/jdk/src/hotspot/share/runtime/init.cpp:124 #2 0xfff8000100c80248 in Threads::create_vm (args=, canTryAgain=0xfff800010134ac4f) at /home/glaubitz/jdk/src/hotspot/share/runtime/thread.cpp:3731 #3 0xfff8000100a6ef0c in JNI_CreateJavaVM_inner (args=0xfff800010134ad68, penv=0xfff800010134ad60, vm=0xfff800010134ad58) at /home/glaubitz/jdk/src/hotspot/share/prims/jni.cpp:3935 #4 JNI_CreateJavaVM (vm=0xfff800010134ad58, penv=0xfff800010134ad60, args=0xfff800010134ad68) at /home/glaubitz/jdk/src/hotspot/share/prims/jni.cpp:4021 #5 0xfff800010024b97c in InitializeJVM (ifn=, penv=0xfff800010134ad60, pvm=0xfff800010134ad58) at /home/glaubitz/jdk/src/java.base/share/native/libjli/java.c:1529 #6 JavaMain (_args=) at /home/glaubitz/jdk/src/java.base/share/native/libjli/java.c:414 #7 0xfff800010035fe68 in start_thread (arg=0xfff800010134b900) at pthread_create.c:486 #8 0xfff800010066c274 in __thread_start () at ../sysdeps/unix/sysv/linux/sparc/sparc64/clone.S:78 Backtrace stopped: previous frame identical to this frame (corrupt stack?) (gdb) I haven't started bisecting yet, so I was wondering whether this crash looks familiar to anyone? The normal server build has no issues. -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From martin.doerr at sap.com Thu Dec 20 13:30:36 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 20 Dec 2018 13:30:36 +0000 Subject: Truncated stacks In-Reply-To: <82a351f6-3cb5-a96c-44fe-0c60ff672d23@oracle.com> References: <7434c305-becd-fb7a-f830-992a2696650d@oracle.com> <82a351f6-3cb5-a96c-44fe-0c60ff672d23@oracle.com> Message-ID: <2d12ba8b7f3d4a0995289cc6a4b0e1ca@sap.com> Hi David, > That's interesting, I wonder why? And can we fix that? The "unlock" is implemented as a leaf call (no safepoint). So the last Java frame is not strictly required. In contrast to this, "lock" uses an OptoRuntime stub which does set last Java frame. I general, it would be possible to do this for the "unlock", too, but that would need C2 adaptations. Anyway, this may not be necessary to get a native stack trace. I rather guess that something confuses the stack walk on x86 (or no backlink available). Best regards, Martin -----Original Message----- From: David Holmes Sent: Donnerstag, 20. Dezember 2018 13:22 To: Doerr, Martin ; hotspot-dev at openjdk.java.net Subject: Re: Truncated stacks Hi Martin, On 20/12/2018 9:57 pm, Doerr, Martin wrote: > Hi David, > > is only this unlock call affected? No - the first example is unlock, the second is lock. > I think regular Java stack walking doesn't work by design. C2 calls SharedRuntime::complete_monitor_unlocking_C without setting last Java frame (PhaseMacroExpand::expand_unlock_node). So the last Java frame based walking won't work. That's interesting, I wonder why? And can we fix that? > Native stack walking may or may not work depending on platform and build options. On x86, you will probably need backlinks to find a frame which belongs to generated code below a C++ frame. Is only x86 affected? Both bugs were reported on x86 and that's all I've looked at so far. I may have to whip up a simple test to check the behaviour more broadly. Thanks, David > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of David Holmes > Sent: Donnerstag, 20. Dezember 2018 06:38 > To: hotspot-dev at openjdk.java.net > Subject: Truncated stacks > > We've noticed a problem investigating bug reports involving monitor > lock/unlock. It seems that the stacks for threads in hs_err reports, in > jstack -mixed output and even in gdb are truncated very high up e.g. > > Stack: [0x00000041adef0000,0x00000041adff0000] > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, > C=native code) > V [jvm.dll+0xb49bd1] os::platform_print_native_stack+0xf1 > (os_windows_x86.cpp:369) > V [jvm.dll+0xd67860] VMError::report+0xe80 (vmerror.cpp:698) > V [jvm.dll+0xd690b4] VMError::report_and_die+0x854 (vmerror.cpp:1463) > V [jvm.dll+0xd69774] VMError::report_and_die+0x64 (vmerror.cpp:1287) > V [jvm.dll+0x57be7e] report_vm_error+0x7e (debug.cpp:232) > V [jvm.dll+0xb21db4] ObjectMonitor::exit+0xf4 (objectmonitor.cpp:1025) > V [jvm.dll+0xc0f08a] SharedRuntime::complete_monitor_unlocking_C+0x17a > (sharedruntime.cpp:2062) > C 0x0000004187bfcf5e > > > or: > > Thread 1005 (Thread 0x7f30bb1ff700 (LWP 7275)): > #0 0x00007f30f950d705 in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/libpthread.so.0 > #1 0x00007f30f7b2d033 in os::PlatformEvent::park() () from > /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #2 0x00007f30f7af7d2d in ObjectMonitor::EnterI(Thread*) () from > /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #3 0x00007f30f7af9a62 in ObjectMonitor::enter(Thread*) [clone .part.90] > () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #4 0x00007f30f7c9bcd2 in ObjectSynchronizer::fast_enter(Handle, > BasicLock*, bool, Thread*) () from > /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #5 0x00007f30f7c02584 in > SharedRuntime::complete_monitor_locking_C(oopDesc*, BasicLock*, > JavaThread*) () > from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so > #6 0x00007f30ddde2228 in ?? () > #7 0x0000000638200fc8 in ?? () > #8 0x00007f30e57e1330 in ?? () > #9 0x00000001000003e8 in ?? () > #10 0x0000000638200920 in ?? () > #11 0x000000080113fa80 in ?? () > #12 0x0000000638200fe8 in ?? () > #13 0xc70405d300000000 in ?? () > #14 0x0000000638200fd8 in ?? () > #15 0x0000000801146328 in ?? () > #16 0x00000006382014d8 in ?? () > #17 0x0000000638202e98 in ?? () > #18 0x0000000638202eb0 in ?? () > #19 0xc704029b382317f8 in ?? () > #20 0x0000000000000000 in ?? () > > Anyone have any ideas about what have broken things? I haven't had time > to try and determine how long this has been occurring. > > Thanks, > David > From robbin.ehn at oracle.com Thu Dec 20 14:52:13 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 20 Dec 2018 15:52:13 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <2a2679cc-b0e0-f8d0-7336-8666e1a42950@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> <2a2679cc-b0e0-f8d0-7336-8666e1a42950@oracle.com> Message-ID: Hi David, On 12/20/18 1:26 PM, David Holmes wrote: > On 20/12/2018 10:10 pm, Robbin Ehn wrote: >> Hi David, >> >> On 12/20/18 7:08 AM, David Holmes wrote: >>> Hi Robbin, >>> >>> Looks good, small doc follow up below ... >> >> Thanks! >> >>> Sound reasonable? >> >> Yes, I also added the comment from my other mail, let me know what you think. > > +?? // Guarantees not to return until disarm() is called, > +?? // if called with currently armed tag (otherwise returns immediately). > +?? // Implementation must guarantee no spurious wakeups. > ??? // Guarantees to return if disarm() and wake() is called. > > Should the first line say "disarm() and wake()" or just "wake()"? A waiter thread calling wait() that gets context switched immediately after entering wait() and stay off proc until after disarm() is called but comes back before wake() is called, may see the waitbarrier as disarmed. So disarm() can wake a waiter thread (returning from wait()). That was what I tried to say there. > > s/Implementation/Implementations/ Fixed > > The fourth line is no longer needed. Above is the reason I would like to keep the fourth line, since only if you call both disarm() and wake() you have that guarantee that waiter threads will return. Thanks, Robbin > > Thanks, > David > > >> Inc: >> http://cr.openjdk.java.net/~rehn/8214271/4/inc/webrev/ >> >> Full: >> http://cr.openjdk.java.net/~rehn/8214271/4/full/webrev/ >> >> /Robbin >> >>> >>> Otherwise this all looks good! >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> Full: >>>> http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ >>>> >>>> Thanks, Robbin >>>> >>>> On 11/23/18 5:55 PM, Robbin Ehn wrote: >>>>> Forgot RFR in subject. >>>>> >>>>> /Robbin >>>>> >>>>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>>>> Hi all, please review. >>>>>> >>>>>> When a safepoint is ended we need a way to get back to 100% utilization as >>>>>> fast >>>>>> as possible. 100% utilization means no idle cpu in the system if there is a >>>>>> JavaThread that could be executed. The traditional ways to wake many, e.g. >>>>>> semaphore, pthread_cond, is not implemented with a single syscall instead >>>>>> they >>>>>> typical do one syscall per thread to wake. >>>>>> >>>>>> This change-set contains that primitive, the WaitBarrier, and a gtest for it. >>>>>> No actual users, which is in coming patches. >>>>>> >>>>>> The WaitBarrier solves by doing a cooperative semaphore posting, threads >>>>>> woken >>>>>> will also post. On Linux we can instead directly use a futex and with one >>>>>> syscall wake all. Depending on how many threads and cpus the performance >>>>>> vary, >>>>>> but a good utilization of the machine, just on the edge of saturated, the >>>>>> time to reach 100% utilization is around 3 times faster with the >>>>>> WaitBarrier (where futex is faster than semaphore). >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>>>> >>>>>> CR: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>>>> >>>>>> Passes 100 iterations of gtest on our platforms, both fastdebug and release. >>>>>> And have been stable when used in safepoints (t1-8) (coming patches). >>>>>> >>>>>> Thanks, Robbin From markus.gronlund at oracle.com Thu Dec 20 15:07:40 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 20 Dec 2018 07:07:40 -0800 (PST) Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> Message-ID: <9eb09e67-b1f3-4a7b-8146-d8973d67a127@default> Thanks David for doing this work. Looks good. " - Moved the JavaThread call to this->exit(false); from the end of thread_main_inner to the start of post_run - as Markus suggested. This also necessitated a change to the gtest logic compared to the original as it has to override post_run() now to avoid the exit call." Just so that I understand the reason for the updates to the gtest part: The reason you need to add a post_run() specialization in threadHelper.inline.hpp, is because the JavaThread::post_run() now invokes JavaThread::exit() (thanks for accommodating that btw), and JavaThread::exit() expects the thread to have a valid thread oop, something that the gtests do not setup/create? JavaThread::exit() contains assertions like: ... assert(threadObj.not_null(), "Java thread object should be created"); Nit: Maybe the comment in threadHelper.inline.hpp: // override as JavaThread::post_run() calls Thread::exit Could become // override as JavaThread::post_run() calls JavaThread::exit() which expect a valid thread object oop Thanks Markus -----Original Message----- From: David Holmes Sent: den 19 december 2018 22:22 To: hotspot-dev developers ; KIM.BARRETT ; Markus Gr?nlund Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic Following on from the preliminary RFR: http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html I've merged with Kim's changes from 8215097. Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ Changes since previous discussion: - Replaced Kim's initial_thread_created assertion with a check of Thread::current_or_null, with an explanation - Removed NJT list addition at construction time as it's not needed after 8215097. - Moved the JavaThread call to this->exit(false); from the end of thread_main_inner to the start of post_run - as Markus suggested. This also necessitated a change to the gtest logic compared to the original as it has to override post_run() now to avoid the exit call. - Made get_thread_name_string virtual so that the JavaThread subclasses in the gtest can override it and allow their names to be seen in hs_err files. (Makes debugging so much easier when you know which thread crashed!). Testing: - Mach 5 tiers 1 - 3 - All runtime tests with: - default GC - ZGC - Shenandoah GC - gc/g1 - gc/serial - gc/epsilon - gc/shenandoah Thanks, David From aph at redhat.com Thu Dec 20 17:31:51 2018 From: aph at redhat.com (Andrew Haley) Date: Thu, 20 Dec 2018 17:31:51 +0000 Subject: Sorry, I messed up [Was: 8215100: AArch64: fix compareTo intrinsic with four-character Latin/Unicode] Message-ID: <8956e452-65d7-df33-8d39-629686ae1c20@redhat.com> I mixed up my Submit repo results and committed the patch for 8215100 to jdk12 after there was a test failure. I'm investigating. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Thu Dec 20 17:54:30 2018 From: aph at redhat.com (Andrew Haley) Date: Thu, 20 Dec 2018 17:54:30 +0000 Subject: Sorry, I messed up [Was: 8215100: AArch64: fix compareTo intrinsic with four-character Latin/Unicode] In-Reply-To: <8956e452-65d7-df33-8d39-629686ae1c20@redhat.com> References: <8956e452-65d7-df33-8d39-629686ae1c20@redhat.com> Message-ID: <3aef3e06-afe0-2361-8a12-b0780e3d4997@redhat.com> On 12/20/18 5:31 PM, Andrew Haley wrote: > I mixed up my Submit repo results and committed the patch for 8215100 to jdk12 after > there was a test failure. I'm investigating. I'm completely baffled. If anyone can let me know what has gone wrong here I'd be grateful. Thanks, and apologies. [Mach5] mach5-one-aph-JDK-8215100-20181219-1718-15405: FAILED, Failed tests: 8 Build Details: 2018-12-19-1711567.aph.source 8 Failed Tests Test Tier Platform Keywords Description Task gtest/GTestWrapper.java tier1 windows-x64 native Exception: RUN_ALL_TESTS() failed. Error ... task gtest/GTestWrapper.java tier1 windows-x64-debug native Exception: RUN_ALL_TESTS() failed. Error ... task gtest/GTestWrapper.java tier1 solaris-sparcv9-debug native Exception: RUN_ALL_TESTS() failed. Error ... task gtest/GTestWrapper.java tier1 macosx-x64-debug native Exception: RUN_ALL_TESTS() failed. Error ... task gtest/GTestWrapper.java tier1 macosx-x64 native Exception: RUN_ALL_TESTS() failed. Error ... task gtest/GTestWrapper.java tier1 solaris-sparcv9 native Exception: RUN_ALL_TESTS() failed. Error ... task gtest/GTestWrapper.java tier1 linux-x64 native Exception: RUN_ALL_TESTS() failed. Error ... task gtest/GTestWrapper.java tier1 linux-x64-debug native Exception: RUN_ALL_TESTS() failed. Error ... task Mach5 Tasks Results Summary * FAILED: 0 * EXECUTED_WITH_FAILURE: 8 * NA: 0 * KILLED: 0 * PASSED: 68 * UNABLE_TO_RUN: 0 Test 8 Executed with failure o tier1-product-open_test_hotspot_jtreg_tier1_common-linux-x64-19 Results: total: 2, passed: 1, failed: 1, skipped: 0 o tier1-debug-open_test_hotspot_jtreg_tier1_common-linux-x64-debug-25 Results: total: 2, passed: 1, failed: 1, skipped: 0 o tier1-product-open_test_hotspot_jtreg_tier1_common-macosx-x64-20 Results: total: 2, passed: 1, failed: 1, skipped: 0 o tier1-debug-open_test_hotspot_jtreg_tier1_common-macosx-x64-debug-26 Results: total: 2, passed: 1, failed: 1, skipped: 0 o tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-58 Results: total: 2, passed: 1, failed: 1, skipped: 0 o tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-59 Results: total: 2, passed: 1, failed: 1, skipped: 0 o tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-21 Results: total: 2, passed: 1, failed: 1, skipped: 0 o tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-27 Results: total: 2, passed: 1, failed: 1, skipped: 0 -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From jesper.wilhelmsson at oracle.com Thu Dec 20 17:59:16 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Thu, 20 Dec 2018 18:59:16 +0100 Subject: Sorry, I messed up [Was: 8215100: AArch64: fix compareTo intrinsic with four-character Latin/Unicode] In-Reply-To: <3aef3e06-afe0-2361-8a12-b0780e3d4997@redhat.com> References: <8956e452-65d7-df33-8d39-629686ae1c20@redhat.com> <3aef3e06-afe0-2361-8a12-b0780e3d4997@redhat.com> Message-ID: <3E4012A5-02B1-4347-BBE5-D8354B3E4B0F@oracle.com> test\hotspot\jtreg\gtest\GTestWrapper.java [ FAILED ] 4 tests, listed below: [ FAILED ] JfrTestNetworkUtilization.RequestFunctionBasic_test_vm [ FAILED ] JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm [ FAILED ] JfrTestNetworkUtilization.InterfaceRemoved_test_vm [ FAILED ] JfrTestNetworkUtilization.InterfaceReset_test_vm You were hit by JDK-8215362. Did you pull the latest changes before running the tests? /Jesper > On 20 Dec 2018, at 18:54, Andrew Haley wrote: > > On 12/20/18 5:31 PM, Andrew Haley wrote: >> I mixed up my Submit repo results and committed the patch for 8215100 to jdk12 after >> there was a test failure. I'm investigating. > > I'm completely baffled. If anyone can let me know what has gone wrong here > I'd be grateful. Thanks, and apologies. > > > [Mach5] mach5-one-aph-JDK-8215100-20181219-1718-15405: FAILED, Failed tests: 8 > > Build Details: 2018-12-19-1711567.aph.source > > > 8 Failed Tests > > Test Tier Platform Keywords Description Task > gtest/GTestWrapper.java tier1 windows-x64 native Exception: RUN_ALL_TESTS() > failed. Error ... task > gtest/GTestWrapper.java tier1 windows-x64-debug native Exception: > RUN_ALL_TESTS() failed. Error ... task > gtest/GTestWrapper.java tier1 solaris-sparcv9-debug native Exception: > RUN_ALL_TESTS() failed. Error ... task > gtest/GTestWrapper.java tier1 macosx-x64-debug native Exception: > RUN_ALL_TESTS() failed. Error ... task > gtest/GTestWrapper.java tier1 macosx-x64 native Exception: RUN_ALL_TESTS() > failed. Error ... task > gtest/GTestWrapper.java tier1 solaris-sparcv9 native Exception: > RUN_ALL_TESTS() failed. Error ... task > gtest/GTestWrapper.java tier1 linux-x64 native Exception: RUN_ALL_TESTS() > failed. Error ... task > gtest/GTestWrapper.java tier1 linux-x64-debug native Exception: > RUN_ALL_TESTS() failed. Error ... task > > > Mach5 Tasks Results Summary > > * FAILED: 0 > * EXECUTED_WITH_FAILURE: 8 > * NA: 0 > * KILLED: 0 > * PASSED: 68 > * UNABLE_TO_RUN: 0 > > > Test > > > 8 Executed with failure > > o tier1-product-open_test_hotspot_jtreg_tier1_common-linux-x64-19 > Results: total: 2, passed: 1, failed: 1, skipped: 0 > o tier1-debug-open_test_hotspot_jtreg_tier1_common-linux-x64-debug-25 > Results: total: 2, passed: 1, failed: 1, skipped: 0 > o tier1-product-open_test_hotspot_jtreg_tier1_common-macosx-x64-20 > Results: total: 2, passed: 1, failed: 1, skipped: 0 > o tier1-debug-open_test_hotspot_jtreg_tier1_common-macosx-x64-debug-26 > Results: total: 2, passed: 1, failed: 1, skipped: 0 > o tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-58 > Results: total: 2, passed: 1, failed: 1, skipped: 0 > o tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-59 > Results: total: 2, passed: 1, failed: 1, skipped: 0 > o tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-21 > Results: total: 2, passed: 1, failed: 1, skipped: 0 > o tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-27 Results: > total: 2, passed: 1, failed: 1, skipped: 0 > > > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Thu Dec 20 18:04:02 2018 From: aph at redhat.com (Andrew Haley) Date: Thu, 20 Dec 2018 18:04:02 +0000 Subject: Sorry, I messed up [Was: 8215100: AArch64: fix compareTo intrinsic with four-character Latin/Unicode] In-Reply-To: <3E4012A5-02B1-4347-BBE5-D8354B3E4B0F@oracle.com> References: <8956e452-65d7-df33-8d39-629686ae1c20@redhat.com> <3aef3e06-afe0-2361-8a12-b0780e3d4997@redhat.com> <3E4012A5-02B1-4347-BBE5-D8354B3E4B0F@oracle.com> Message-ID: On 12/20/18 5:59 PM, jesper.wilhelmsson at oracle.com wrote: > You were hit by JDK-8215362. Did you pull the latest changes before running the tests? > /Jesper Ah, I guess not. I'll leave the patch in then. Thanks! -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Thu Dec 20 18:41:27 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 20 Dec 2018 13:41:27 -0500 Subject: 12 RFR(XS) 8214329: SwingMark SubMenus 9% regression in 12-b19 on Linux client In-Reply-To: <0ecece2d-4f29-4e3a-2306-eb07e91cc7c5@oracle.com> References: <55c235ee-1733-42e2-e0f7-e75016dacd05@oracle.com> <0ecece2d-4f29-4e3a-2306-eb07e91cc7c5@oracle.com> Message-ID: <34f540e3-7eab-d865-21ab-58757f769697@oracle.com> This makes sense to me.? Looks good. Coleen On 12/18/18 10:12 AM, Sean Mullan wrote: > Looks good, although I think someone from the HotSpot Group should > also review it. > > On the bug, can you add the details below into the Description? Also, > since you have no regression test, you will need a noreg label (I > guess noreg-perf would be appropriate). Also, the bug says it is > BlockedBy JDK-8215205 which should probably be removed. > > Thanks, > Sean > > On 12/17/18 5:49 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8214329 >> http://cr.openjdk.java.net/~dlong/8214329/webrev/ >> >> In "8212605: Pure-Java implementation of >> AccessController.doPrivileged", the stackwalk in >> JVM_GetStackAccessControlContext was changed from using a >> vframeStream to using a javaVFrame, so that it could access locals. >> However this caused a regression because javaVFrame has more >> overhead. JDK-8215205 fixes the problem by allowing the use of >> javaVFrame only when needed. >> >> This change reverts JVM_GetStackAccessControlContext back to using >> vframeStream, and uses the new API in JDK-8215205 to get a javaVFrame >> lazily.? It also removes two unnecessary calls to java_sender(). >> >> dl From glaubitz at physik.fu-berlin.de Fri Dec 21 00:39:38 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 21 Dec 2018 01:39:38 +0100 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> Message-ID: Hello! On 12/20/18 2:00 PM, John Paul Adrian Glaubitz wrote: > I haven't started bisecting yet, so I was wondering whether this crash looks familiar > to anyone? The normal server build has no issues. This regression is caused by [1]: 8210425: [x86] sharedRuntimeTrig/sharedRuntimeTrans compiled without optimization Reverting that change unbreaks Zero on linux-sparc. Adrian > [1] http://hg.openjdk.java.net/jdk/jdk/rev/a50956e2368c?revcount=120 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From david.holmes at oracle.com Fri Dec 21 02:48:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 12:48:09 +1000 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <9eb09e67-b1f3-4a7b-8146-d8973d67a127@default> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> <9eb09e67-b1f3-4a7b-8146-d8973d67a127@default> Message-ID: Hi Markus, On 21/12/2018 1:07 am, Markus Gronlund wrote: > Thanks David for doing this work. > > Looks good. Thanks for looking at this. > " > - Moved the JavaThread call to this->exit(false); from the end of thread_main_inner to the start of post_run - as Markus suggested. This also necessitated a change to the gtest logic compared to the original as it has to override post_run() now to avoid the exit call." > > Just so that I understand the reason for the updates to the gtest part: > > The reason you need to add a post_run() specialization in threadHelper.inline.hpp, is because the JavaThread::post_run() now invokes JavaThread::exit() (thanks for accommodating that btw), and JavaThread::exit() expects the thread to have a valid thread oop, something that the gtests do not setup/create? > > JavaThread::exit() contains assertions like: > ... > assert(threadObj.not_null(), "Java thread object should be created"); That's correct. And without a Thread oop, and being a real Java thread, little else in JavaThread::exit makes any sense. > Nit: > > Maybe the comment in threadHelper.inline.hpp: > > // override as JavaThread::post_run() calls Thread::exit > > Could become > > // override as JavaThread::post_run() calls JavaThread::exit() which expect a valid thread object oop Sure. The Thread::exit rather than JavaThread::exit was a typo. Thanks, David > Thanks > Markus > > > -----Original Message----- > From: David Holmes > Sent: den 19 december 2018 22:22 > To: hotspot-dev developers ; KIM.BARRETT ; Markus Gr?nlund > Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic > > Following on from the preliminary RFR: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html > > I've merged with Kim's changes from 8215097. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 > Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ > > Changes since previous discussion: > > - Replaced Kim's initial_thread_created assertion with a check of Thread::current_or_null, with an explanation > > - Removed NJT list addition at construction time as it's not needed after 8215097. > > - Moved the JavaThread call to this->exit(false); from the end of thread_main_inner to the start of post_run - as Markus suggested. This also necessitated a change to the gtest logic compared to the original as it has to override post_run() now to avoid the exit call. > > - Made get_thread_name_string virtual so that the JavaThread subclasses in the gtest can override it and allow their names to be seen in hs_err files. (Makes debugging so much easier when you know which thread crashed!). > > Testing: > - Mach 5 tiers 1 - 3 > - All runtime tests with: > - default GC > - ZGC > - Shenandoah GC > - gc/g1 > - gc/serial > - gc/epsilon > - gc/shenandoah > > Thanks, > David > From david.holmes at oracle.com Fri Dec 21 03:17:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 13:17:17 +1000 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> <2a2679cc-b0e0-f8d0-7336-8666e1a42950@oracle.com> Message-ID: On 21/12/2018 12:52 am, Robbin Ehn wrote: > Hi David, > > On 12/20/18 1:26 PM, David Holmes wrote: >> On 20/12/2018 10:10 pm, Robbin Ehn wrote: >>> Hi David, >>> >>> On 12/20/18 7:08 AM, David Holmes wrote: >>>> Hi Robbin, >>>> >>>> Looks good, small doc follow up below ... >>> >>> Thanks! >>> >>>> Sound reasonable? >>> >>> Yes, I also added the comment from my other mail, let me know what >>> you think. >> >> +?? // Guarantees not to return until disarm() is called, >> +?? // if called with currently armed tag (otherwise returns >> immediately). >> +?? // Implementation must guarantee no spurious wakeups. >> ???? // Guarantees to return if disarm() and wake() is called. >> >> Should the first line say "disarm() and wake()" or just "wake()"? > > A waiter thread calling wait() that gets context switched immediately after > entering wait() and stay off proc until after disarm() is called but comes > back before wake() is called, may see the waitbarrier as disarmed. > So disarm() can wake a waiter thread (returning from wait()). > That was what I tried to say there. Got it - subtle. Further this sounds like a race that could lead to bugs if not used very carefully ie. you can't assume between disarm() and wake() that all threads are blocked. I think perhaps this needs to be expanded to make this more obvious: 68 // - A call to wait(tag) will block if the barrier is armed with the value 69 // 'tag'; else it will return immediately. 70 // - A blocked thread is eligible to execute again once the barrier is 71 // disarmed and wake() has been called. + - A call to wait(tag) that would block if it continued, but instead + is descheduled, may return immediately if scheduled after a + call to disarm(), but before the call to wake(). It also made me realize that in the general case (not when used with safepoints I think due to other state checks) a wake() may stall due to threads with a previous tag entering the wait() late. Thanks, David >> >> s/Implementation/Implementations/ > > Fixed > >> >> The fourth line is no longer needed. > > Above is the reason I would like to keep the fourth line, since only if > you call > both disarm() and wake() you have that guarantee that waiter threads will > return. > > Thanks, Robbin > >> >> Thanks, >> David >> >> >>> Inc: >>> http://cr.openjdk.java.net/~rehn/8214271/4/inc/webrev/ >>> >>> Full: >>> http://cr.openjdk.java.net/~rehn/8214271/4/full/webrev/ >>> >>> /Robbin >>> >>>> >>>> Otherwise this all looks good! >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>>> Full: >>>>> http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ >>>>> >>>>> Thanks, Robbin >>>>> >>>>> On 11/23/18 5:55 PM, Robbin Ehn wrote: >>>>>> Forgot RFR in subject. >>>>>> >>>>>> /Robbin >>>>>> >>>>>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>>>>> Hi all, please review. >>>>>>> >>>>>>> When a safepoint is ended we need a way to get back to 100% >>>>>>> utilization as fast >>>>>>> as possible. 100% utilization means no idle cpu in the system if >>>>>>> there is a >>>>>>> JavaThread that could be executed. The traditional ways to wake >>>>>>> many, e.g. >>>>>>> semaphore, pthread_cond, is not implemented with a single syscall >>>>>>> instead they >>>>>>> typical do one syscall per thread to wake. >>>>>>> >>>>>>> This change-set contains that primitive, the WaitBarrier, and a >>>>>>> gtest for it. >>>>>>> No actual users, which is in coming patches. >>>>>>> >>>>>>> The WaitBarrier solves by doing a cooperative semaphore posting, >>>>>>> threads woken >>>>>>> will also post. On Linux we can instead directly use a futex and >>>>>>> with one >>>>>>> syscall wake all. Depending on how many threads and cpus the >>>>>>> performance vary, >>>>>>> but a good utilization of the machine, just on the edge of >>>>>>> saturated, the time to reach 100% utilization is around 3 times >>>>>>> faster with the WaitBarrier (where futex is faster than semaphore). >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>>>>> >>>>>>> CR: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>>>>> >>>>>>> Passes 100 iterations of gtest on our platforms, both fastdebug >>>>>>> and release. >>>>>>> And have been stable when used in safepoints (t1-8) (coming >>>>>>> patches). >>>>>>> >>>>>>> Thanks, Robbin From david.holmes at oracle.com Fri Dec 21 05:11:59 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 15:11:59 +1000 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> Message-ID: <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> Hi Adrian, On 21/12/2018 10:39 am, John Paul Adrian Glaubitz wrote: > Hello! > > On 12/20/18 2:00 PM, John Paul Adrian Glaubitz wrote: >> I haven't started bisecting yet, so I was wondering whether this crash looks familiar >> to anyone? The normal server build has no issues. > > This regression is caused by [1]: > > 8210425: [x86] sharedRuntimeTrig/sharedRuntimeTrans compiled without optimization > > Reverting that change unbreaks Zero on linux-sparc. Okay, so either you need to figure out why the optimization flags cause the failure or else restore the: - ifeq ($(OPENJDK_TARGET_CPU_ARCH), x86) that was removed. I know what is the simpler option :) Cheers, David > Adrian > >> [1] http://hg.openjdk.java.net/jdk/jdk/rev/a50956e2368c?revcount=120 > From david.holmes at oracle.com Fri Dec 21 05:38:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 15:38:10 +1000 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> Message-ID: Correction ... On 21/12/2018 3:11 pm, David Holmes wrote: > Hi Adrian, > > On 21/12/2018 10:39 am, John Paul Adrian Glaubitz wrote: >> Hello! >> >> On 12/20/18 2:00 PM, John Paul Adrian Glaubitz wrote: >>> I haven't started bisecting yet, so I was wondering whether this >>> crash looks familiar >>> to anyone? The normal server build has no issues. >> >> This regression is caused by [1]: >> >> 8210425: [x86] sharedRuntimeTrig/sharedRuntimeTrans compiled without >> optimization >> >> Reverting that change unbreaks Zero on linux-sparc. > > Okay, so either you need to figure out why the optimization flags cause > the failure or else restore the: > > -? ifeq ($(OPENJDK_TARGET_CPU_ARCH), x86) > > that was removed. > > I know what is the simpler option :) Maybe not that simple as other archs are involved. AFAICS before the patch the compilation should have the default optimization level; and after it will have "none" unless FDLIBM_CFLAGS is set. Seems odd that "none" would cause a regression like this. Can you generate before/after build logs to check the actual flags? David > Cheers, > David > >> Adrian >> >>> [1] http://hg.openjdk.java.net/jdk/jdk/rev/a50956e2368c?revcount=120 >> From matthias.baesken at sap.com Fri Dec 21 07:01:28 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Dec 2018 07:01:28 +0000 Subject: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam Message-ID: Hello David, I adjusted the bug synopsis and description, the second webrev uses the pthread id of the passed Thread* instead of pthread_self() : https://bugs.openjdk.java.net/browse/JDK-8215707 http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.1/ Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 20. Dezember 2018 13:19 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: RFR [XS] : 8215707: [macosx] check return code of > pthread_getschedparam > > On 20/12/2018 9:56 pm, Baesken, Matthias wrote: > >> Note this code is likely never exercised due to the default > >> ThreadPriorityPolicy of 0. > > > > Hi David, ThreadPriorityPolicy=1 is a valid setting / product flag. > > Yes it is. That doesn't mean it is actually used to any great extent. > > > You find quite a few people setting ThreadPriorityPolicy when you search > for it, so the coding should better be fixed I guess . > > Quite a few? I could only find a couple (Cassandra, Gatling) and they > were on Linux. I was surprised to find how many people knew about the > bug that allowed the root check to be by-passed. > > The fact this code is incorrect on OS X indicates to me it is not > actually being used - or by chance priority is only ever changed in the > current thread. > > >> > >> That seems more likely given that the code is incorrect: > >> > >> pthread_getschedparam(pthread_self(), &policy, &sp); > >> > >> this always gets the priority of the current thread, but the method need > >> not be called on the current thread! > > > > Yes , true ! > > Same for os::set_native_priority in os_bsd.cpp , there pthread_self() is > also used instead of the real Thread . > > > > 2338OSReturn os::set_native_priority(Thread* thread, int newpri) { > > 2339 if (!UseThreadPriorities || ThreadPriorityPolicy == 0) return OS_OK; > > 2340 > > .... > > 2346#elif defined(__APPLE__) || defined(__NetBSD__) > > 2347 struct sched_param sp; > > 2348 int policy; > > 2349 pthread_t self = pthread_self(); > > 2350 > > 2351 if (pthread_getschedparam(self, &policy, &sp) != 0) { > > 2352 return OS_ERR; > > 2353 } > > If you intend fixing this probably best to modify the bug synopsis and > description. > > Cheers, > David > From david.holmes at oracle.com Fri Dec 21 07:19:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 17:19:42 +1000 Subject: Truncated stacks In-Reply-To: <2d12ba8b7f3d4a0995289cc6a4b0e1ca@sap.com> References: <7434c305-becd-fb7a-f830-992a2696650d@oracle.com> <82a351f6-3cb5-a96c-44fe-0c60ff672d23@oracle.com> <2d12ba8b7f3d4a0995289cc6a4b0e1ca@sap.com> Message-ID: I've been unable to reproduce this in a simple test case unfortunately. David On 20/12/2018 11:30 pm, Doerr, Martin wrote: > Hi David, > >> That's interesting, I wonder why? And can we fix that? > The "unlock" is implemented as a leaf call (no safepoint). So the last Java frame is not strictly required. > In contrast to this, "lock" uses an OptoRuntime stub which does set last Java frame. > I general, it would be possible to do this for the "unlock", too, but that would need C2 adaptations. > > Anyway, this may not be necessary to get a native stack trace. I rather guess that something confuses the stack walk on x86 (or no backlink available). > > Best regards, > Martin > > > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 20. Dezember 2018 13:22 > To: Doerr, Martin ; hotspot-dev at openjdk.java.net > Subject: Re: Truncated stacks > > Hi Martin, > > On 20/12/2018 9:57 pm, Doerr, Martin wrote: >> Hi David, >> >> is only this unlock call affected? > > No - the first example is unlock, the second is lock. > >> I think regular Java stack walking doesn't work by design. C2 calls SharedRuntime::complete_monitor_unlocking_C without setting last Java frame (PhaseMacroExpand::expand_unlock_node). So the last Java frame based walking won't work. > > That's interesting, I wonder why? And can we fix that? > >> Native stack walking may or may not work depending on platform and build options. On x86, you will probably need backlinks to find a frame which belongs to generated code below a C++ frame. Is only x86 affected? > > Both bugs were reported on x86 and that's all I've looked at so far. > > I may have to whip up a simple test to check the behaviour more broadly. > > Thanks, > David > >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: hotspot-dev On Behalf Of David Holmes >> Sent: Donnerstag, 20. Dezember 2018 06:38 >> To: hotspot-dev at openjdk.java.net >> Subject: Truncated stacks >> >> We've noticed a problem investigating bug reports involving monitor >> lock/unlock. It seems that the stacks for threads in hs_err reports, in >> jstack -mixed output and even in gdb are truncated very high up e.g. >> >> Stack: [0x00000041adef0000,0x00000041adff0000] >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >> C=native code) >> V [jvm.dll+0xb49bd1] os::platform_print_native_stack+0xf1 >> (os_windows_x86.cpp:369) >> V [jvm.dll+0xd67860] VMError::report+0xe80 (vmerror.cpp:698) >> V [jvm.dll+0xd690b4] VMError::report_and_die+0x854 (vmerror.cpp:1463) >> V [jvm.dll+0xd69774] VMError::report_and_die+0x64 (vmerror.cpp:1287) >> V [jvm.dll+0x57be7e] report_vm_error+0x7e (debug.cpp:232) >> V [jvm.dll+0xb21db4] ObjectMonitor::exit+0xf4 (objectmonitor.cpp:1025) >> V [jvm.dll+0xc0f08a] SharedRuntime::complete_monitor_unlocking_C+0x17a >> (sharedruntime.cpp:2062) >> C 0x0000004187bfcf5e >> >> >> or: >> >> Thread 1005 (Thread 0x7f30bb1ff700 (LWP 7275)): >> #0 0x00007f30f950d705 in pthread_cond_wait@@GLIBC_2.3.2 () from >> /lib64/libpthread.so.0 >> #1 0x00007f30f7b2d033 in os::PlatformEvent::park() () from >> /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so >> #2 0x00007f30f7af7d2d in ObjectMonitor::EnterI(Thread*) () from >> /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so >> #3 0x00007f30f7af9a62 in ObjectMonitor::enter(Thread*) [clone .part.90] >> () from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so >> #4 0x00007f30f7c9bcd2 in ObjectSynchronizer::fast_enter(Handle, >> BasicLock*, bool, Thread*) () from >> /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so >> #5 0x00007f30f7c02584 in >> SharedRuntime::complete_monitor_locking_C(oopDesc*, BasicLock*, >> JavaThread*) () >> from /scratch/fairoz/JAVA/jdk12/jdk-12-ea+24/lib/server/libjvm.so >> #6 0x00007f30ddde2228 in ?? () >> #7 0x0000000638200fc8 in ?? () >> #8 0x00007f30e57e1330 in ?? () >> #9 0x00000001000003e8 in ?? () >> #10 0x0000000638200920 in ?? () >> #11 0x000000080113fa80 in ?? () >> #12 0x0000000638200fe8 in ?? () >> #13 0xc70405d300000000 in ?? () >> #14 0x0000000638200fd8 in ?? () >> #15 0x0000000801146328 in ?? () >> #16 0x00000006382014d8 in ?? () >> #17 0x0000000638202e98 in ?? () >> #18 0x0000000638202eb0 in ?? () >> #19 0xc704029b382317f8 in ?? () >> #20 0x0000000000000000 in ?? () >> >> Anyone have any ideas about what have broken things? I haven't had time >> to try and determine how long this has been occurring. >> >> Thanks, >> David >> From david.holmes at oracle.com Fri Dec 21 07:23:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 17:23:02 +1000 Subject: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam In-Reply-To: References: Message-ID: Hi Matthias, On 21/12/2018 5:01 pm, Baesken, Matthias wrote: > Hello David, I adjusted the bug synopsis and description, the second webrev uses the pthread id of the passed Thread* instead of pthread_self() : > > > https://bugs.openjdk.java.net/browse/JDK-8215707 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.1/ Looks good. Thanks, David > > > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 20. Dezember 2018 13:19 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Subject: Re: RFR [XS] : 8215707: [macosx] check return code of >> pthread_getschedparam >> >> On 20/12/2018 9:56 pm, Baesken, Matthias wrote: >>>> Note this code is likely never exercised due to the default >>>> ThreadPriorityPolicy of 0. >>> >>> Hi David, ThreadPriorityPolicy=1 is a valid setting / product flag. >> >> Yes it is. That doesn't mean it is actually used to any great extent. >> >>> You find quite a few people setting ThreadPriorityPolicy when you search >> for it, so the coding should better be fixed I guess . >> >> Quite a few? I could only find a couple (Cassandra, Gatling) and they >> were on Linux. I was surprised to find how many people knew about the >> bug that allowed the root check to be by-passed. >> >> The fact this code is incorrect on OS X indicates to me it is not >> actually being used - or by chance priority is only ever changed in the >> current thread. >> >>>> >>>> That seems more likely given that the code is incorrect: >>>> >>>> pthread_getschedparam(pthread_self(), &policy, &sp); >>>> >>>> this always gets the priority of the current thread, but the method need >>>> not be called on the current thread! >>> >>> Yes , true ! >>> Same for os::set_native_priority in os_bsd.cpp , there pthread_self() is >> also used instead of the real Thread . >>> >>> 2338OSReturn os::set_native_priority(Thread* thread, int newpri) { >>> 2339 if (!UseThreadPriorities || ThreadPriorityPolicy == 0) return OS_OK; >>> 2340 >>> .... >>> 2346#elif defined(__APPLE__) || defined(__NetBSD__) >>> 2347 struct sched_param sp; >>> 2348 int policy; >>> 2349 pthread_t self = pthread_self(); >>> 2350 >>> 2351 if (pthread_getschedparam(self, &policy, &sp) != 0) { >>> 2352 return OS_ERR; >>> 2353 } >> >> If you intend fixing this probably best to modify the bug synopsis and >> description. >> >> Cheers, >> David >> > From matthias.baesken at sap.com Fri Dec 21 07:46:03 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Dec 2018 07:46:03 +0000 Subject: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam In-Reply-To: References: Message-ID: Thanks ! Is there a (jtreg) test available to check that the prio/niceness level changes with os::set_native_priority used ? As far as I know , at least on Linux it would be possible to observe the changed thread niceness values from outside with e.g. top -H (shows the NI (niceness values) of all thread ), or ps -T -l (long format, -l adds the niceness values too at least on my distro). Ps -l works also on Mac and disaplays the NI value . However currently the root -check would block any testing under "normal" test users . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Freitag, 21. Dezember 2018 08:23 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and > pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check > return code of pthread_getschedparam > > Hi Matthias, > > On 21/12/2018 5:01 pm, Baesken, Matthias wrote: > > Hello David, I adjusted the bug synopsis and description, the second > webrev uses the pthread id of the passed Thread* instead of > pthread_self() : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8215707 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.1/ > > Looks good. > > Thanks, > David > > > > > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Donnerstag, 20. Dezember 2018 13:19 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Subject: Re: RFR [XS] : 8215707: [macosx] check return code of > >> pthread_getschedparam > >> > >> On 20/12/2018 9:56 pm, Baesken, Matthias wrote: > >>>> Note this code is likely never exercised due to the default > >>>> ThreadPriorityPolicy of 0. > >>> > >>> Hi David, ThreadPriorityPolicy=1 is a valid setting / product flag. > >> > >> Yes it is. That doesn't mean it is actually used to any great extent. > >> > >>> You find quite a few people setting ThreadPriorityPolicy when you > search > >> for it, so the coding should better be fixed I guess . > >> > >> Quite a few? I could only find a couple (Cassandra, Gatling) and they > >> were on Linux. I was surprised to find how many people knew about the > >> bug that allowed the root check to be by-passed. > >> > >> The fact this code is incorrect on OS X indicates to me it is not > >> actually being used - or by chance priority is only ever changed in the > >> current thread. > >> > >>>> > >>>> That seems more likely given that the code is incorrect: > >>>> > >>>> pthread_getschedparam(pthread_self(), &policy, &sp); > >>>> > >>>> this always gets the priority of the current thread, but the method > need > >>>> not be called on the current thread! > >>> > >>> Yes , true ! > >>> Same for os::set_native_priority in os_bsd.cpp , there pthread_self() > is > >> also used instead of the real Thread . > >>> > >>> 2338OSReturn os::set_native_priority(Thread* thread, int newpri) { > >>> 2339 if (!UseThreadPriorities || ThreadPriorityPolicy == 0) return > OS_OK; > >>> 2340 > >>> .... > >>> 2346#elif defined(__APPLE__) || defined(__NetBSD__) > >>> 2347 struct sched_param sp; > >>> 2348 int policy; > >>> 2349 pthread_t self = pthread_self(); > >>> 2350 > >>> 2351 if (pthread_getschedparam(self, &policy, &sp) != 0) { > >>> 2352 return OS_ERR; > >>> 2353 } > >> > >> If you intend fixing this probably best to modify the bug synopsis and > >> description. > >> > >> Cheers, > >> David > >> > > From matthias.baesken at sap.com Fri Dec 21 08:39:57 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Dec 2018 08:39:57 +0000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> Message-ID: Hi David , it might be that the functionality is seen as not very helpful by some and removed or deprecated in some future Java release. However it is present in the current JDKs and should work there nicely . Currently I have some points I do not like about the current state : - the root-check is not consistent , it is present on Linux / BSD (Mac) but I don't see it on Solaris - It ignores currently the CAP_SYS_NICE capability - it ignores that setting a higher niceness works nicely on most OS (checked Linux/Solaris/BSD) without being root (or having special capabilities) - the root check makes testing hard (maybe that's why the Mac version was a bit broken?) Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 20. Dezember 2018 12:06 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: ThreadPriorityPolicy settings for non-root users > > Hi Matthias, > > The more I think about this the more I see it as a huge can of worms. > There are very, very, limited usecases for managing the priority of > individual threads within a running Java application. Adjusting the > process priority/nice-ness is effective and much simpler. > > Cheers, > David > > On 20/12/2018 3:13 am, Baesken, Matthias wrote: > > > > Hello , > > currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 (normal, > the default) and 1, > > the so called "aggressive" mode : > > > > "1 : Aggressive. "\ > > " Java thread priorities map over to the entire range of "\ > > " native thread priorities. Higher Java thread priorities map "\ > > " to higher native thread priorities. This policy should be "\ > > " used with care, as sometimes it can cause performance "\ > > " degradation in the application and/or the entire system. On "\ > > " Linux this policy requires root privilege.") \ > > > > Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the > text in globals.hpp mentions only Linux which seems to be not fully correct): > > > > if (geteuid() != 0) { ... } in function prio_init(). > > > > (looks like the check is not done for other platforms). > > > > However the check for root (e.g. on Linux) hinders users to set a > ****lower priority**** for a thread (== increase the "niceness" level) > > when running as a non-root user (there might be strange ways from > outside the VM with calling scripts and renice but .... ). > > > > > > In older JDKs (e.g. JDK8) there was a "workaround" to use for example > ThreadPriorityPolicy=2 to avoid the root-check, > > but this is not possible any more in recent JDKs (10/11) after the range > check (0,1) has been introduced for the ThreadPriorityPolicy flag > > (and probably the old workaround was not a good one anyway because it > was undocumented). > > > > So do you think we could introduce another XX-flag ( > AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) > > that allows using the "aggressive" mode for non-root users ? Another > option would be to add another mode 2 for ThreadPriorityPolicy > > that documents the behavior (like mode 1 but without root-user check). > > > > If I get it right, even setting ***higher prios*** (lower niceness) is > possible for non-root users on systems configured in an appropriate way > > (using the CAP_SYS_NICE capability). > > > > But setting lower prio / higher niceness is even possible for normal users > NOW without special config, it is just disabled by the root-check > > which is very bad. > > > > Best regards, Matthias > > From sgehwolf at redhat.com Fri Dec 21 09:12:46 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 21 Dec 2018 10:12:46 +0100 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> Message-ID: <48cbf65f09472c2e233a8ee64105467f14679a2d.camel@redhat.com> On Fri, 2018-12-21 at 15:38 +1000, David Holmes wrote: > Correction ... > > On 21/12/2018 3:11 pm, David Holmes wrote: > > Hi Adrian, > > > > On 21/12/2018 10:39 am, John Paul Adrian Glaubitz wrote: > > > Hello! > > > > > > On 12/20/18 2:00 PM, John Paul Adrian Glaubitz wrote: > > > > I haven't started bisecting yet, so I was wondering whether this > > > > crash looks familiar > > > > to anyone? The normal server build has no issues. > > > > > > This regression is caused by [1]: > > > > > > 8210425: [x86] sharedRuntimeTrig/sharedRuntimeTrans compiled without > > > optimization > > > > > > Reverting that change unbreaks Zero on linux-sparc. > > > > Okay, so either you need to figure out why the optimization flags cause > > the failure or else restore the: > > > > - ifeq ($(OPENJDK_TARGET_CPU_ARCH), x86) > > > > that was removed. > > > > I know what is the simpler option :) > > Maybe not that simple as other archs are involved. AFAICS before the > patch the compilation should have the default optimization level; and > after it will have "none" unless FDLIBM_CFLAGS is set. Seems odd that > "none" would cause a regression like this. Can you generate before/after > build logs to check the actual flags? Yes, that. Here is what it looks like for me: $ cat /disk/openjdk/upstream-sources/openjdk-head/build/linux-x86_64-zero-release/hotspot/variant-zero/libjvm/objs/sharedRuntimeTrig.o.cmdline /usr/bin/g++ -MMD -MF /disk/openjdk/upstream-sources/openjdk-head/build/linux-x86_64-zero-release/hotspot/variant-zero/libjvm/objs/sharedRuntimeTrig.d -I/disk/openjdk/upstream-sources/openjdk-head/build/linux-x86_64-zero-release/hotspot/variant-zero/libjvm/objs/precompiled -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D_GNU_SOURCE -D_REENTRANT -pipe -fno-rtti -fno-exceptions -fvisibility=hidden -fno-strict-aliasing -fno-omit-frame-pointer -fcheck-new -std=gnu++98 -DSUPPORTS_CLOCK_MONOTONIC -DLINUX -Wall -Wextra -Wformat=2 -Wpointer-arith -Wsign-compare -Wunused-function -Wundef -Wunused-value -Wreturn-type -Woverloaded-virtual -Wreorder -fPIC -DVM_LITTLE_ENDIAN -D_LP64=1 -fno-delete-null-pointer-checks -fno-lifetime-dse -Wno-format-zero-length -Wtype-limits -Wuninitialized -m64 -DPRODUCT -DTARGET_ARCH_zero -DINCLUDE_SUFFIX_OS=_linux -DINCLUDE_SUFFIX_CPU=_zero -DINCLUDE_SUFFIX_COMPILER=_gcc -DTARGET_COMPILER_gcc -DAMD64 -DHOTSPOT_LIB_ARCH='"amd64"' -DZERO -DCC_INTERP -DZERO_LIBARCH='"amd64"' -DDTRACE_ENABLED -DINCLUDE_JVMCI=0 -DINCLUDE_CDS=0 -DINCLUDE_AOT=0 -DINCLUDE_G1GC=0 -DINCLUDE_EPSILONGC=0 -DINCLUDE_ZGC=0 -DINCLUDE_SHENANDOAHGC=0 -DINCLUDE_JFR=0 -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/share -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/os/linux -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/os/posix -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/cpu/zero -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/os_cpu/linux_zero -I/disk/openjdk/upstream-sources/openjdk-head/build/linux-x86_64-zero-release/hotspot/variant-zero/gensrc -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/share/precompiled -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/share/include -I/disk/openjdk/upstream-sources/openjdk-head/src/hotspot/os/posix/include -I/disk/openjdk/upstream-sources/openjdk-head/build/linux-x86_64-zero-release/support/modules_include/java.base -I/disk/openjdk/upstream-sources/openjdk-head/build/linux-x86_64-zero-release/support/modules_include/java.base/linux -I/disk/openjdk/upstream-sources/openjdk-head/src/java.base/share/native/libjimage -m64 -g -Wno-unused-parameter -Wno-unused -Wno-extra -Wno-parentheses -Wno-comment -Wno-unknown-pragmas -Wno-address -Wno-delete-non-virtual-dtor -Wno-char-subscripts -Wno-array-bounds -Wno-int-in-bool-context -Wno-ignored-qualifiers -Wno-missing-field-initializers -Wno-implicit-fallthrough -Wno-empty-body -Wno-strict-overflow -Wno-sequence-point -Wno-maybe-uninitialized -Wno-misleading-indentation -Wno-return-type -Wno-switch -O3 -DNO_PCH -ffp-contract=off -O2 -DTHIS_FILE='"sharedRuntimeTrig.cpp"' -c -o /disk/openjdk/upstream-sources/openjdk-head/build/linux-x86_64-zero-release/hotspot/variant-zero/libjvm/objs/sharedRuntimeTrig.o /disk/openjdk/upstream-sources/openjdk-head/src/hotspot/share/runtime/sharedRuntimeTrig.cpp In addition, I'd be interested in seeing what FDLIBM_CFLAGS are after configure. This is what I have on x86_64: $ grep FDLIBM_CFLAGS build/linux-x86_64-zero-release/spec.gmk FDLIBM_CFLAGS := -ffp-contract=off Also, you should see something like this in config.log at build/linux- x86_64-zero-release/configure-support/config.log: configure:55336: checking if the C compiler supports "-ffp-contract=off" configure:55352: /usr/bin/gcc -c -m64 -ffp-contract=off conftest.c >&5 configure:55352: $? = 0 configure:55366: result: yes configure:55436: checking if the C++ compiler supports "-ffp-contract=off" configure:55452: /usr/bin/g++ -c -ffp-contract=off conftest.cpp >&5 configure:55452: $? = 0 configure:55466: result: yes configure:55488: checking if both compilers support "-ffp-contract=off" configure:55493: result: yes It could be that you are hitting a GCC bug specific to linux sparc. Then again, why wouldn't you hit it for a server JVM. As a temporary work-around you could try this (compile without opt as prior JDK- 8210425, not tested): diff --git a/make/hotspot/lib/JvmOverrideFiles.gmk b/make/hotspot/lib/JvmOverrideFiles.gmk --- a/make/hotspot/lib/JvmOverrideFiles.gmk +++ b/make/hotspot/lib/JvmOverrideFiles.gmk @@ -49,6 +49,15 @@ # by using -ffp-contract=off on GCC/Clang platforms. ifneq ($(FDLIBM_CFLAGS), ) LIBJVM_FDLIBM_COPY_OPT_FLAG := $(CXX_O_FLAG_NORM) + # Don't optimize fdlibm-fork for Zero on Linux spark + # See JDK-XXXX + ifeq ($(call check-jvm-feature, zero), true) + ifeq ($(OPENJDK_TARGET_OS), linux) + ifeq ($(OPENJDK_TARGET_CPU_ARCH), spark) + LIBJVM_FDLIBM_COPY_OPT_FLAG := $(CXX_O_FLAG_NONE) + endif + endif + endif endif ifeq ($(OPENJDK_TARGET_OS), linux) Thanks, Severin > David > > > Cheers, > > David > > > > > Adrian > > > > > > > [1] http://hg.openjdk.java.net/jdk/jdk/rev/a50956e2368c?revcount=120 From robbin.ehn at oracle.com Fri Dec 21 09:45:54 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 21 Dec 2018 10:45:54 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> <2a2679cc-b0e0-f8d0-7336-8666e1a42950@oracle.com> Message-ID: <01873a0f-a0fb-18b9-f7d4-98bb638e9b57@oracle.com> Hi David, On 2018-12-21 04:17, David Holmes wrote: > > Got it - subtle. > > Further this sounds like a race that could lead to bugs if not used very > carefully ie. you can't assume between disarm() and wake() that all threads are > blocked. I didn't realize how subtle this is. I think your original comment that disarm/wake should be one operation was spot on. Investigating... thinking... testing... yes I think this will work, fixed! Sorry for not looking more into this before. > > I think perhaps this needs to be expanded to make this more obvious: > > ? 68 //??? - A call to wait(tag) will block if the barrier is armed with the value > ? 69 //????? 'tag'; else it will return immediately. > ? 70 //??? - A blocked thread is eligible to execute again once the barrier is > ? 71 //????? disarmed and wake() has been called. > +????????? - A call to wait(tag) that would block if it continued, but instead > +??????????? is descheduled, may return immediately if scheduled after a > +?????????? call to disarm(), but before the call to wake(). > > It also made me realize that in the general case (not when used with safepoints > I think due to other state checks) a wake() may stall due to threads with a > previous tag entering the wait() late. I added a double checking in the semaphore version, this means both implementation should have progress guarantee. Making this v5 a bit large due to a lot of comments being changed. Inc: http://cr.openjdk.java.net/~rehn/8214271/5/inc/webrev/ Full: http://cr.openjdk.java.net/~rehn/8214271/5/full/webrev/ gtest passes thousands of loops locally and hundreds in mach5. Thanks, Robbin > > Thanks, > David > >>> >>> s/Implementation/Implementations/ >> >> Fixed >> >>> >>> The fourth line is no longer needed. >> >> Above is the reason I would like to keep the fourth line, since only if you call >> both disarm() and wake() you have that guarantee that waiter threads will >> return. >> >> Thanks, Robbin >> >>> >>> Thanks, >>> David >>> >>> >>>> Inc: >>>> http://cr.openjdk.java.net/~rehn/8214271/4/inc/webrev/ >>>> >>>> Full: >>>> http://cr.openjdk.java.net/~rehn/8214271/4/full/webrev/ >>>> >>>> /Robbin >>>> >>>>> >>>>> Otherwise this all looks good! >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> >>>>>> Full: >>>>>> http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ >>>>>> >>>>>> Thanks, Robbin >>>>>> >>>>>> On 11/23/18 5:55 PM, Robbin Ehn wrote: >>>>>>> Forgot RFR in subject. >>>>>>> >>>>>>> /Robbin >>>>>>> >>>>>>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>>>>>> Hi all, please review. >>>>>>>> >>>>>>>> When a safepoint is ended we need a way to get back to 100% utilization >>>>>>>> as fast >>>>>>>> as possible. 100% utilization means no idle cpu in the system if there is a >>>>>>>> JavaThread that could be executed. The traditional ways to wake many, e.g. >>>>>>>> semaphore, pthread_cond, is not implemented with a single syscall >>>>>>>> instead they >>>>>>>> typical do one syscall per thread to wake. >>>>>>>> >>>>>>>> This change-set contains that primitive, the WaitBarrier, and a gtest >>>>>>>> for it. >>>>>>>> No actual users, which is in coming patches. >>>>>>>> >>>>>>>> The WaitBarrier solves by doing a cooperative semaphore posting, threads >>>>>>>> woken >>>>>>>> will also post. On Linux we can instead directly use a futex and with one >>>>>>>> syscall wake all. Depending on how many threads and cpus the performance >>>>>>>> vary, >>>>>>>> but a good utilization of the machine, just on the edge of saturated, >>>>>>>> the time to reach 100% utilization is around 3 times faster with the >>>>>>>> WaitBarrier (where futex is faster than semaphore). >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>>>>>> >>>>>>>> CR: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>>>>>> >>>>>>>> Passes 100 iterations of gtest on our platforms, both fastdebug and >>>>>>>> release. >>>>>>>> And have been stable when used in safepoints (t1-8) (coming patches). >>>>>>>> >>>>>>>> Thanks, Robbin From gunter.haug at sap.com Fri Dec 21 10:37:18 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Fri, 21 Dec 2018 10:37:18 +0000 Subject: RFR [XXS]: Tiny bug in VM monitoring/management Message-ID: Hi all, can I please have reviews and a sponsor fort the following change: https://bugs.openjdk.java.net/browse/JDK-8215791 http://cr.openjdk.java.net/~ghaug/webrevs/8215791/ This was obviously just a copy and paste mishap. Thanks and happy holidays, Gunter From david.holmes at oracle.com Fri Dec 21 10:44:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 20:44:09 +1000 Subject: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam In-Reply-To: References: Message-ID: <1a4ef1ac-b0ad-b3cf-b865-fbbaa893808c@oracle.com> On 21/12/2018 5:46 pm, Baesken, Matthias wrote: > Thanks ! > > Is there a (jtreg) test available to check that the prio/niceness level changes with os::set_native_priority used ? Nope. As I've said this code is basically unused. The only time I've heard of it being used was on Solaris and that was years back. David ----- > > As far as I know , at least on Linux it would be possible to observe the changed thread niceness values from outside with e.g. > > top -H (shows the NI (niceness values) of all thread ), or ps -T -l (long format, -l adds the niceness values too at least on my distro). > Ps -l works also on Mac and disaplays the NI value . > > However currently the root -check would block any testing under "normal" test users . > > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Freitag, 21. Dezember 2018 08:23 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Subject: Re: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and >> pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check >> return code of pthread_getschedparam >> >> Hi Matthias, >> >> On 21/12/2018 5:01 pm, Baesken, Matthias wrote: >>> Hello David, I adjusted the bug synopsis and description, the second >> webrev uses the pthread id of the passed Thread* instead of >> pthread_self() : >>> >>> >>> https://bugs.openjdk.java.net/browse/JDK-8215707 >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.1/ >> >> Looks good. >> >> Thanks, >> David >> >>> >>> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Donnerstag, 20. Dezember 2018 13:19 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Subject: Re: RFR [XS] : 8215707: [macosx] check return code of >>>> pthread_getschedparam >>>> >>>> On 20/12/2018 9:56 pm, Baesken, Matthias wrote: >>>>>> Note this code is likely never exercised due to the default >>>>>> ThreadPriorityPolicy of 0. >>>>> >>>>> Hi David, ThreadPriorityPolicy=1 is a valid setting / product flag. >>>> >>>> Yes it is. That doesn't mean it is actually used to any great extent. >>>> >>>>> You find quite a few people setting ThreadPriorityPolicy when you >> search >>>> for it, so the coding should better be fixed I guess . >>>> >>>> Quite a few? I could only find a couple (Cassandra, Gatling) and they >>>> were on Linux. I was surprised to find how many people knew about the >>>> bug that allowed the root check to be by-passed. >>>> >>>> The fact this code is incorrect on OS X indicates to me it is not >>>> actually being used - or by chance priority is only ever changed in the >>>> current thread. >>>> >>>>>> >>>>>> That seems more likely given that the code is incorrect: >>>>>> >>>>>> pthread_getschedparam(pthread_self(), &policy, &sp); >>>>>> >>>>>> this always gets the priority of the current thread, but the method >> need >>>>>> not be called on the current thread! >>>>> >>>>> Yes , true ! >>>>> Same for os::set_native_priority in os_bsd.cpp , there pthread_self() >> is >>>> also used instead of the real Thread . >>>>> >>>>> 2338OSReturn os::set_native_priority(Thread* thread, int newpri) { >>>>> 2339 if (!UseThreadPriorities || ThreadPriorityPolicy == 0) return >> OS_OK; >>>>> 2340 >>>>> .... >>>>> 2346#elif defined(__APPLE__) || defined(__NetBSD__) >>>>> 2347 struct sched_param sp; >>>>> 2348 int policy; >>>>> 2349 pthread_t self = pthread_self(); >>>>> 2350 >>>>> 2351 if (pthread_getschedparam(self, &policy, &sp) != 0) { >>>>> 2352 return OS_ERR; >>>>> 2353 } >>>> >>>> If you intend fixing this probably best to modify the bug synopsis and >>>> description. >>>> >>>> Cheers, >>>> David >>>> >>> From david.holmes at oracle.com Fri Dec 21 10:56:29 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 20:56:29 +1000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> Message-ID: <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> On 21/12/2018 6:39 pm, Baesken, Matthias wrote: > Hi David , it might be that the functionality is seen as not very helpful by some and removed or deprecated in some future Java release. > > However it is present in the current JDKs and should work there nicely . Sorry but that's a bit naive. The code is old and bit-rotted and in some cases (Mac port) likely never used, so the idea that "it's there so it should work" is just not realistic - sorry. > Currently I have some points I do not like about the current state : > > - the root-check is not consistent , it is present on Linux / BSD (Mac) but I don't see it on Solaris Wasn't needed on Solaris. User-level capabilities sufficed. > - It ignores currently the CAP_SYS_NICE capability It never supported it. AFAIK the linux code doesn't really support any capability based permissions. > - it ignores that setting a higher niceness works nicely on most OS (checked Linux/Solaris/BSD) without being root (or having special capabilities) The priority control was never really about tweaking niceness levels. > - the root check makes testing hard (maybe that's why the Mac version was a bit broken?) Running under sudo isn't that hard. Sorry I'm not very supportive here - this isn't something that needs some minor tweaking to bring back online, it's something that may never have worked well in the first place. Have you got real use cases for this? Cheers, David > > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 20. Dezember 2018 12:06 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Subject: Re: ThreadPriorityPolicy settings for non-root users >> >> Hi Matthias, >> >> The more I think about this the more I see it as a huge can of worms. >> There are very, very, limited usecases for managing the priority of >> individual threads within a running Java application. Adjusting the >> process priority/nice-ness is effective and much simpler. >> >> Cheers, >> David >> >> On 20/12/2018 3:13 am, Baesken, Matthias wrote: >>> >>> Hello , >>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 (normal, >> the default) and 1, >>> the so called "aggressive" mode : >>> >>> "1 : Aggressive. "\ >>> " Java thread priorities map over to the entire range of "\ >>> " native thread priorities. Higher Java thread priorities map "\ >>> " to higher native thread priorities. This policy should be "\ >>> " used with care, as sometimes it can cause performance "\ >>> " degradation in the application and/or the entire system. On "\ >>> " Linux this policy requires root privilege.") \ >>> >>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the >> text in globals.hpp mentions only Linux which seems to be not fully correct): >>> >>> if (geteuid() != 0) { ... } in function prio_init(). >>> >>> (looks like the check is not done for other platforms). >>> >>> However the check for root (e.g. on Linux) hinders users to set a >> ****lower priority**** for a thread (== increase the "niceness" level) >>> when running as a non-root user (there might be strange ways from >> outside the VM with calling scripts and renice but .... ). >>> >>> >>> In older JDKs (e.g. JDK8) there was a "workaround" to use for example >> ThreadPriorityPolicy=2 to avoid the root-check, >>> but this is not possible any more in recent JDKs (10/11) after the range >> check (0,1) has been introduced for the ThreadPriorityPolicy flag >>> (and probably the old workaround was not a good one anyway because it >> was undocumented). >>> >>> So do you think we could introduce another XX-flag ( >> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) >>> that allows using the "aggressive" mode for non-root users ? Another >> option would be to add another mode 2 for ThreadPriorityPolicy >>> that documents the behavior (like mode 1 but without root-user check). >>> >>> If I get it right, even setting ***higher prios*** (lower niceness) is >> possible for non-root users on systems configured in an appropriate way >>> (using the CAP_SYS_NICE capability). >>> >>> But setting lower prio / higher niceness is even possible for normal users >> NOW without special config, it is just disabled by the root-check >>> which is very bad. >>> >>> Best regards, Matthias >>> From david.holmes at oracle.com Fri Dec 21 11:00:24 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 21:00:24 +1000 Subject: RFR [XXS]: Tiny bug in VM monitoring/management In-Reply-To: References: Message-ID: <3185e4bb-be42-c899-cfe5-34b2edfb5e44@oracle.com> Hi Gunter, On 21/12/2018 8:37 pm, Haug, Gunter wrote: > Hi all, > > can I please have reviews and a sponsor fort the following change: > > https://bugs.openjdk.java.net/browse/JDK-8215791 > http://cr.openjdk.java.net/~ghaug/webrevs/8215791/ > > This was obviously just a copy and paste mishap. Obviously! But I have to wonder if this will suddenly turn this on or off for some platform where it was off or on?? Ideally we will find both flags are normally set the same. > Thanks and happy holidays, Likewise! Cheers, David > Gunter > From matthias.baesken at sap.com Fri Dec 21 11:09:37 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Dec 2018 11:09:37 +0000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> Message-ID: > > Have you got real use cases for this? > Hi David, yes we have a user who was setting the "infamous" ThreadPriorityPolicy=2 (+ then different prios/niceness levels for different threads) , but that is now no longer working in current JDK11 . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Freitag, 21. Dezember 2018 11:56 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: ThreadPriorityPolicy settings for non-root users > > On 21/12/2018 6:39 pm, Baesken, Matthias wrote: > > Hi David , it might be that the functionality is seen as not very helpful by > some and removed or deprecated in some future Java release. > > > > However it is present in the current JDKs and should work there nicely . > > Sorry but that's a bit naive. The code is old and bit-rotted and in some > cases (Mac port) likely never used, so the idea that "it's there so it > should work" is just not realistic - sorry. > > > Currently I have some points I do not like about the current state : > > > > - the root-check is not consistent , it is present on Linux / BSD (Mac) but > I don't see it on Solaris > > Wasn't needed on Solaris. User-level capabilities sufficed. > > > - It ignores currently the CAP_SYS_NICE capability > > It never supported it. AFAIK the linux code doesn't really support any > capability based permissions. > > > - it ignores that setting a higher niceness works nicely on most OS > (checked Linux/Solaris/BSD) without being root (or having special > capabilities) > > The priority control was never really about tweaking niceness levels. > > > - the root check makes testing hard (maybe that's why the Mac version > was a bit broken?) > > Running under sudo isn't that hard. > > Sorry I'm not very supportive here - this isn't something that needs > some minor tweaking to bring back online, it's something that may never > have worked well in the first place. > > Have you got real use cases for this? > > Cheers, > David > > > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Donnerstag, 20. Dezember 2018 12:06 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Subject: Re: ThreadPriorityPolicy settings for non-root users > >> > >> Hi Matthias, > >> > >> The more I think about this the more I see it as a huge can of worms. > >> There are very, very, limited usecases for managing the priority of > >> individual threads within a running Java application. Adjusting the > >> process priority/nice-ness is effective and much simpler. > >> > >> Cheers, > >> David > >> > >> On 20/12/2018 3:13 am, Baesken, Matthias wrote: > >>> > >>> Hello , > >>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 > (normal, > >> the default) and 1, > >>> the so called "aggressive" mode : > >>> > >>> "1 : Aggressive. "\ > >>> " Java thread priorities map over to the entire range of "\ > >>> " native thread priorities. Higher Java thread priorities map "\ > >>> " to higher native thread priorities. This policy should be "\ > >>> " used with care, as sometimes it can cause performance "\ > >>> " degradation in the application and/or the entire system. On "\ > >>> " Linux this policy requires root privilege.") \ > >>> > >>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the > >> text in globals.hpp mentions only Linux which seems to be not fully > correct): > >>> > >>> if (geteuid() != 0) { ... } in function prio_init(). > >>> > >>> (looks like the check is not done for other platforms). > >>> > >>> However the check for root (e.g. on Linux) hinders users to set a > >> ****lower priority**** for a thread (== increase the "niceness" level) > >>> when running as a non-root user (there might be strange ways from > >> outside the VM with calling scripts and renice but .... ). > >>> > >>> > >>> In older JDKs (e.g. JDK8) there was a "workaround" to use for example > >> ThreadPriorityPolicy=2 to avoid the root-check, > >>> but this is not possible any more in recent JDKs (10/11) after the range > >> check (0,1) has been introduced for the ThreadPriorityPolicy flag > >>> (and probably the old workaround was not a good one anyway because > it > >> was undocumented). > >>> > >>> So do you think we could introduce another XX-flag ( > >> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) > >>> that allows using the "aggressive" mode for non-root users ? Another > >> option would be to add another mode 2 for ThreadPriorityPolicy > >>> that documents the behavior (like mode 1 but without root-user check). > >>> > >>> If I get it right, even setting ***higher prios*** (lower niceness) is > >> possible for non-root users on systems configured in an appropriate way > >>> (using the CAP_SYS_NICE capability). > >>> > >>> But setting lower prio / higher niceness is even possible for normal users > >> NOW without special config, it is just disabled by the root-check > >>> which is very bad. > >>> > >>> Best regards, Matthias > >>> From goetz.lindenmaier at sap.com Fri Dec 21 11:24:43 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 21 Dec 2018 11:24:43 +0000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> Message-ID: Hi David, Yes, as Matthias stated we have at least one application that relies on thread priorities. Their going to Java 11 is blocked by this. Also are you aware that this is the implementation of Thread.setPriority()? So isn't it great we finally fix the old and bit-rotten code that implements this official method? Best regards, Goetz. > -----Original Message----- > From: hotspot-dev On Behalf Of > David Holmes > Sent: Freitag, 21. Dezember 2018 11:56 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: ThreadPriorityPolicy settings for non-root users > > On 21/12/2018 6:39 pm, Baesken, Matthias wrote: > > Hi David , it might be that the functionality is seen as not very helpful > by some and removed or deprecated in some future Java release. > > > > However it is present in the current JDKs and should work there nicely . > > Sorry but that's a bit naive. The code is old and bit-rotted and in some > cases (Mac port) likely never used, so the idea that "it's there so it > should work" is just not realistic - sorry. > > > Currently I have some points I do not like about the current state : > > > > - the root-check is not consistent , it is present on Linux / BSD (Mac) but > I don't see it on Solaris > > Wasn't needed on Solaris. User-level capabilities sufficed. > > > - It ignores currently the CAP_SYS_NICE capability > > It never supported it. AFAIK the linux code doesn't really support any > capability based permissions. > > > - it ignores that setting a higher niceness works nicely on most OS > (checked Linux/Solaris/BSD) without being root (or having special > capabilities) > > The priority control was never really about tweaking niceness levels. > > > - the root check makes testing hard (maybe that's why the Mac version > was a bit broken?) > > Running under sudo isn't that hard. > > Sorry I'm not very supportive here - this isn't something that needs > some minor tweaking to bring back online, it's something that may never > have worked well in the first place. > > Have you got real use cases for this? > > Cheers, > David > > > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Donnerstag, 20. Dezember 2018 12:06 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Subject: Re: ThreadPriorityPolicy settings for non-root users > >> > >> Hi Matthias, > >> > >> The more I think about this the more I see it as a huge can of worms. > >> There are very, very, limited usecases for managing the priority of > >> individual threads within a running Java application. Adjusting the > >> process priority/nice-ness is effective and much simpler. > >> > >> Cheers, > >> David > >> > >> On 20/12/2018 3:13 am, Baesken, Matthias wrote: > >>> > >>> Hello , > >>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 > (normal, > >> the default) and 1, > >>> the so called "aggressive" mode : > >>> > >>> "1 : Aggressive. "\ > >>> " Java thread priorities map over to the entire range of "\ > >>> " native thread priorities. Higher Java thread priorities map "\ > >>> " to higher native thread priorities. This policy should be "\ > >>> " used with care, as sometimes it can cause performance "\ > >>> " degradation in the application and/or the entire system. On "\ > >>> " Linux this policy requires root privilege.") \ > >>> > >>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the > >> text in globals.hpp mentions only Linux which seems to be not fully > correct): > >>> > >>> if (geteuid() != 0) { ... } in function prio_init(). > >>> > >>> (looks like the check is not done for other platforms). > >>> > >>> However the check for root (e.g. on Linux) hinders users to set a > >> ****lower priority**** for a thread (== increase the "niceness" level) > >>> when running as a non-root user (there might be strange ways from > >> outside the VM with calling scripts and renice but .... ). > >>> > >>> > >>> In older JDKs (e.g. JDK8) there was a "workaround" to use for example > >> ThreadPriorityPolicy=2 to avoid the root-check, > >>> but this is not possible any more in recent JDKs (10/11) after the range > >> check (0,1) has been introduced for the ThreadPriorityPolicy flag > >>> (and probably the old workaround was not a good one anyway because > it > >> was undocumented). > >>> > >>> So do you think we could introduce another XX-flag ( > >> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) > >>> that allows using the "aggressive" mode for non-root users ? Another > >> option would be to add another mode 2 for ThreadPriorityPolicy > >>> that documents the behavior (like mode 1 but without root-user check). > >>> > >>> If I get it right, even setting ***higher prios*** (lower niceness) is > >> possible for non-root users on systems configured in an appropriate way > >>> (using the CAP_SYS_NICE capability). > >>> > >>> But setting lower prio / higher niceness is even possible for normal users > >> NOW without special config, it is just disabled by the root-check > >>> which is very bad. > >>> > >>> Best regards, Matthias > >>> From glaubitz at physik.fu-berlin.de Fri Dec 21 12:27:21 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 21 Dec 2018 13:27:21 +0100 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: <48cbf65f09472c2e233a8ee64105467f14679a2d.camel@redhat.com> References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> <48cbf65f09472c2e233a8ee64105467f14679a2d.camel@redhat.com> Message-ID: Hi Severin! On 12/21/18 10:12 AM, Severin Gehwolf wrote: > It could be that you are hitting a GCC bug specific to linux sparc. > Then again, why wouldn't you hit it for a server JVM. As a temporary > work-around you could try this (compile without opt as prior JDK- > 8210425, not tested): > > diff --git a/make/hotspot/lib/JvmOverrideFiles.gmk b/make/hotspot/lib/JvmOverrideFiles.gmk > --- a/make/hotspot/lib/JvmOverrideFiles.gmk > +++ b/make/hotspot/lib/JvmOverrideFiles.gmk > @@ -49,6 +49,15 @@ > # by using -ffp-contract=off on GCC/Clang platforms. > ifneq ($(FDLIBM_CFLAGS), ) > LIBJVM_FDLIBM_COPY_OPT_FLAG := $(CXX_O_FLAG_NORM) > + # Don't optimize fdlibm-fork for Zero on Linux spark > + # See JDK-XXXX > + ifeq ($(call check-jvm-feature, zero), true) > + ifeq ($(OPENJDK_TARGET_OS), linux) > + ifeq ($(OPENJDK_TARGET_CPU_ARCH), spark) > + LIBJVM_FDLIBM_COPY_OPT_FLAG := $(CXX_O_FLAG_NONE) > + endif > + endif > + endif > endif This indeed fixes the problem for me, with the spelling fix for "sparc" ;-). Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From david.holmes at oracle.com Fri Dec 21 12:33:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 22:33:08 +1000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> Message-ID: On 21/12/2018 9:09 pm, Baesken, Matthias wrote: >> >> Have you got real use cases for this? >> > > Hi David, yes we have a user who was setting the "infamous" ThreadPriorityPolicy=2 (+ then different prios/niceness levels for different threads) , but that is now no longer working in current JDK11 . I'd be interested to know what the actual usecase is. But I think a native method would be the best way to handle it. David > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Freitag, 21. Dezember 2018 11:56 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Subject: Re: ThreadPriorityPolicy settings for non-root users >> >> On 21/12/2018 6:39 pm, Baesken, Matthias wrote: >>> Hi David , it might be that the functionality is seen as not very helpful by >> some and removed or deprecated in some future Java release. >>> >>> However it is present in the current JDKs and should work there nicely . >> >> Sorry but that's a bit naive. The code is old and bit-rotted and in some >> cases (Mac port) likely never used, so the idea that "it's there so it >> should work" is just not realistic - sorry. >> >>> Currently I have some points I do not like about the current state : >>> >>> - the root-check is not consistent , it is present on Linux / BSD (Mac) but >> I don't see it on Solaris >> >> Wasn't needed on Solaris. User-level capabilities sufficed. >> >>> - It ignores currently the CAP_SYS_NICE capability >> >> It never supported it. AFAIK the linux code doesn't really support any >> capability based permissions. >> >>> - it ignores that setting a higher niceness works nicely on most OS >> (checked Linux/Solaris/BSD) without being root (or having special >> capabilities) >> >> The priority control was never really about tweaking niceness levels. >> >>> - the root check makes testing hard (maybe that's why the Mac version >> was a bit broken?) >> >> Running under sudo isn't that hard. >> >> Sorry I'm not very supportive here - this isn't something that needs >> some minor tweaking to bring back online, it's something that may never >> have worked well in the first place. >> >> Have you got real use cases for this? >> >> Cheers, >> David >> >>> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Donnerstag, 20. Dezember 2018 12:06 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Subject: Re: ThreadPriorityPolicy settings for non-root users >>>> >>>> Hi Matthias, >>>> >>>> The more I think about this the more I see it as a huge can of worms. >>>> There are very, very, limited usecases for managing the priority of >>>> individual threads within a running Java application. Adjusting the >>>> process priority/nice-ness is effective and much simpler. >>>> >>>> Cheers, >>>> David >>>> >>>> On 20/12/2018 3:13 am, Baesken, Matthias wrote: >>>>> >>>>> Hello , >>>>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 >> (normal, >>>> the default) and 1, >>>>> the so called "aggressive" mode : >>>>> >>>>> "1 : Aggressive. "\ >>>>> " Java thread priorities map over to the entire range of "\ >>>>> " native thread priorities. Higher Java thread priorities map "\ >>>>> " to higher native thread priorities. This policy should be "\ >>>>> " used with care, as sometimes it can cause performance "\ >>>>> " degradation in the application and/or the entire system. On "\ >>>>> " Linux this policy requires root privilege.") \ >>>>> >>>>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the >>>> text in globals.hpp mentions only Linux which seems to be not fully >> correct): >>>>> >>>>> if (geteuid() != 0) { ... } in function prio_init(). >>>>> >>>>> (looks like the check is not done for other platforms). >>>>> >>>>> However the check for root (e.g. on Linux) hinders users to set a >>>> ****lower priority**** for a thread (== increase the "niceness" level) >>>>> when running as a non-root user (there might be strange ways from >>>> outside the VM with calling scripts and renice but .... ). >>>>> >>>>> >>>>> In older JDKs (e.g. JDK8) there was a "workaround" to use for example >>>> ThreadPriorityPolicy=2 to avoid the root-check, >>>>> but this is not possible any more in recent JDKs (10/11) after the range >>>> check (0,1) has been introduced for the ThreadPriorityPolicy flag >>>>> (and probably the old workaround was not a good one anyway because >> it >>>> was undocumented). >>>>> >>>>> So do you think we could introduce another XX-flag ( >>>> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) >>>>> that allows using the "aggressive" mode for non-root users ? Another >>>> option would be to add another mode 2 for ThreadPriorityPolicy >>>>> that documents the behavior (like mode 1 but without root-user check). >>>>> >>>>> If I get it right, even setting ***higher prios*** (lower niceness) is >>>> possible for non-root users on systems configured in an appropriate way >>>>> (using the CAP_SYS_NICE capability). >>>>> >>>>> But setting lower prio / higher niceness is even possible for normal users >>>> NOW without special config, it is just disabled by the root-check >>>>> which is very bad. >>>>> >>>>> Best regards, Matthias >>>>> From matthias.baesken at sap.com Fri Dec 21 12:44:05 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Dec 2018 12:44:05 +0000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> Message-ID: > I'd be interested to know what the actual usecase is. But I think a > native method would be the best way to handle it. Hi David, sorry I am not sure what native method are you talking about ? Do you mean the application - coding should be rewritten from Java into some C/C++-JNI + Java mixture where the setpriority - calls to adjust the threads are done from the C/C++ coding part ? Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Freitag, 21. Dezember 2018 13:33 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Cc: Haug, Gunter > Subject: Re: ThreadPriorityPolicy settings for non-root users > > On 21/12/2018 9:09 pm, Baesken, Matthias wrote: > >> > >> Have you got real use cases for this? > >> > > > > Hi David, yes we have a user who was setting the "infamous" > ThreadPriorityPolicy=2 (+ then different prios/niceness levels for different > threads) , but that is now no longer working in current JDK11 . > > I'd be interested to know what the actual usecase is. But I think a > native method would be the best way to handle it. > > David > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Freitag, 21. Dezember 2018 11:56 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Subject: Re: ThreadPriorityPolicy settings for non-root users > >> > >> On 21/12/2018 6:39 pm, Baesken, Matthias wrote: > >>> Hi David , it might be that the functionality is seen as not very helpful > by > >> some and removed or deprecated in some future Java release. > >>> > >>> However it is present in the current JDKs and should work there nicely . > >> > >> Sorry but that's a bit naive. The code is old and bit-rotted and in some > >> cases (Mac port) likely never used, so the idea that "it's there so it > >> should work" is just not realistic - sorry. > >> > >>> Currently I have some points I do not like about the current state : > >>> > >>> - the root-check is not consistent , it is present on Linux / BSD (Mac) > but > >> I don't see it on Solaris > >> > >> Wasn't needed on Solaris. User-level capabilities sufficed. > >> > >>> - It ignores currently the CAP_SYS_NICE capability > >> > >> It never supported it. AFAIK the linux code doesn't really support any > >> capability based permissions. > >> > >>> - it ignores that setting a higher niceness works nicely on most OS > >> (checked Linux/Solaris/BSD) without being root (or having special > >> capabilities) > >> > >> The priority control was never really about tweaking niceness levels. > >> > >>> - the root check makes testing hard (maybe that's why the Mac version > >> was a bit broken?) > >> > >> Running under sudo isn't that hard. > >> > >> Sorry I'm not very supportive here - this isn't something that needs > >> some minor tweaking to bring back online, it's something that may never > >> have worked well in the first place. > >> > >> Have you got real use cases for this? > >> > >> Cheers, > >> David > >> > >>> > >>> Best regards, Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Donnerstag, 20. Dezember 2018 12:06 > >>>> To: Baesken, Matthias ; 'hotspot- > >>>> dev at openjdk.java.net' > >>>> Subject: Re: ThreadPriorityPolicy settings for non-root users > >>>> > >>>> Hi Matthias, > >>>> > >>>> The more I think about this the more I see it as a huge can of worms. > >>>> There are very, very, limited usecases for managing the priority of > >>>> individual threads within a running Java application. Adjusting the > >>>> process priority/nice-ness is effective and much simpler. > >>>> > >>>> Cheers, > >>>> David > >>>> > >>>> On 20/12/2018 3:13 am, Baesken, Matthias wrote: > >>>>> > >>>>> Hello , > >>>>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 > >> (normal, > >>>> the default) and 1, > >>>>> the so called "aggressive" mode : > >>>>> > >>>>> "1 : Aggressive. "\ > >>>>> " Java thread priorities map over to the entire range of "\ > >>>>> " native thread priorities. Higher Java thread priorities map "\ > >>>>> " to higher native thread priorities. This policy should be "\ > >>>>> " used with care, as sometimes it can cause performance "\ > >>>>> " degradation in the application and/or the entire system. On "\ > >>>>> " Linux this policy requires root privilege.") \ > >>>>> > >>>>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp > (the > >>>> text in globals.hpp mentions only Linux which seems to be not fully > >> correct): > >>>>> > >>>>> if (geteuid() != 0) { ... } in function prio_init(). > >>>>> > >>>>> (looks like the check is not done for other platforms). > >>>>> > >>>>> However the check for root (e.g. on Linux) hinders users to set a > >>>> ****lower priority**** for a thread (== increase the "niceness" level) > >>>>> when running as a non-root user (there might be strange ways from > >>>> outside the VM with calling scripts and renice but .... ). > >>>>> > >>>>> > >>>>> In older JDKs (e.g. JDK8) there was a "workaround" to use for > example > >>>> ThreadPriorityPolicy=2 to avoid the root-check, > >>>>> but this is not possible any more in recent JDKs (10/11) after the range > >>>> check (0,1) has been introduced for the ThreadPriorityPolicy flag > >>>>> (and probably the old workaround was not a good one anyway > because > >> it > >>>> was undocumented). > >>>>> > >>>>> So do you think we could introduce another XX-flag ( > >>>> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) > >>>>> that allows using the "aggressive" mode for non-root users ? Another > >>>> option would be to add another mode 2 for ThreadPriorityPolicy > >>>>> that documents the behavior (like mode 1 but without root-user > check). > >>>>> > >>>>> If I get it right, even setting ***higher prios*** (lower niceness) is > >>>> possible for non-root users on systems configured in an appropriate > way > >>>>> (using the CAP_SYS_NICE capability). > >>>>> > >>>>> But setting lower prio / higher niceness is even possible for normal > users > >>>> NOW without special config, it is just disabled by the root-check > >>>>> which is very bad. > >>>>> > >>>>> Best regards, Matthias > >>>>> From david.holmes at oracle.com Fri Dec 21 12:47:11 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 22:47:11 +1000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> Message-ID: On 21/12/2018 9:24 pm, Lindenmaier, Goetz wrote: > Hi David, > > Yes, as Matthias stated we have at least one application that relies on > thread priorities. Their going to Java 11 is blocked by this. I'd like to hear more. I'd be very surprised that any actual functional correctness issue is addressed by playing with niceness. > Also are you aware that this is the implementation of Thread.setPriority()? There are no specified semantics for Thread priorities in Java, as they pertain to operating system priorities, or relative scheduling across processes. So there is nothing broken with regard to the specification with the default behaviour (ThreadPriorityPolicy=0). > So isn't it great we finally fix the old and bit-rotten code that implements > this official method? No. Thread priorities are a minefield. Giving threads priorities was one of the biggest mistakes in the Java thread API. People were advised to just forget about them 20+ years ago - and most took that advice. I think the only reason this wasn't all stripped out years ago was some corner cases with Solaris scheduling classes. If you want to just disable the root check to get back the erroneous ThreadPriorityPolicy=2 behaviour then file a bug and a CSR and propose a patch. I don't think we want yet another flag for this though. Just drop the root check for ThreadPriorityPolicy=1 and let the underlying system permissions control success or failure. David > Best regards, > Goetz. > > >> -----Original Message----- >> From: hotspot-dev On Behalf Of >> David Holmes >> Sent: Freitag, 21. Dezember 2018 11:56 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Subject: Re: ThreadPriorityPolicy settings for non-root users >> >> On 21/12/2018 6:39 pm, Baesken, Matthias wrote: >>> Hi David , it might be that the functionality is seen as not very helpful >> by some and removed or deprecated in some future Java release. >>> >>> However it is present in the current JDKs and should work there nicely . >> >> Sorry but that's a bit naive. The code is old and bit-rotted and in some >> cases (Mac port) likely never used, so the idea that "it's there so it >> should work" is just not realistic - sorry. >> >>> Currently I have some points I do not like about the current state : >>> >>> - the root-check is not consistent , it is present on Linux / BSD (Mac) but >> I don't see it on Solaris >> >> Wasn't needed on Solaris. User-level capabilities sufficed. >> >>> - It ignores currently the CAP_SYS_NICE capability >> >> It never supported it. AFAIK the linux code doesn't really support any >> capability based permissions. >> >>> - it ignores that setting a higher niceness works nicely on most OS >> (checked Linux/Solaris/BSD) without being root (or having special >> capabilities) >> >> The priority control was never really about tweaking niceness levels. >> >>> - the root check makes testing hard (maybe that's why the Mac version >> was a bit broken?) >> >> Running under sudo isn't that hard. >> >> Sorry I'm not very supportive here - this isn't something that needs >> some minor tweaking to bring back online, it's something that may never >> have worked well in the first place. >> >> Have you got real use cases for this? >> >> Cheers, >> David >> >>> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Donnerstag, 20. Dezember 2018 12:06 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Subject: Re: ThreadPriorityPolicy settings for non-root users >>>> >>>> Hi Matthias, >>>> >>>> The more I think about this the more I see it as a huge can of worms. >>>> There are very, very, limited usecases for managing the priority of >>>> individual threads within a running Java application. Adjusting the >>>> process priority/nice-ness is effective and much simpler. >>>> >>>> Cheers, >>>> David >>>> >>>> On 20/12/2018 3:13 am, Baesken, Matthias wrote: >>>>> >>>>> Hello , >>>>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 >> (normal, >>>> the default) and 1, >>>>> the so called "aggressive" mode : >>>>> >>>>> "1 : Aggressive. "\ >>>>> " Java thread priorities map over to the entire range of "\ >>>>> " native thread priorities. Higher Java thread priorities map "\ >>>>> " to higher native thread priorities. This policy should be "\ >>>>> " used with care, as sometimes it can cause performance "\ >>>>> " degradation in the application and/or the entire system. On "\ >>>>> " Linux this policy requires root privilege.") \ >>>>> >>>>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp (the >>>> text in globals.hpp mentions only Linux which seems to be not fully >> correct): >>>>> >>>>> if (geteuid() != 0) { ... } in function prio_init(). >>>>> >>>>> (looks like the check is not done for other platforms). >>>>> >>>>> However the check for root (e.g. on Linux) hinders users to set a >>>> ****lower priority**** for a thread (== increase the "niceness" level) >>>>> when running as a non-root user (there might be strange ways from >>>> outside the VM with calling scripts and renice but .... ). >>>>> >>>>> >>>>> In older JDKs (e.g. JDK8) there was a "workaround" to use for example >>>> ThreadPriorityPolicy=2 to avoid the root-check, >>>>> but this is not possible any more in recent JDKs (10/11) after the range >>>> check (0,1) has been introduced for the ThreadPriorityPolicy flag >>>>> (and probably the old workaround was not a good one anyway because >> it >>>> was undocumented). >>>>> >>>>> So do you think we could introduce another XX-flag ( >>>> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) >>>>> that allows using the "aggressive" mode for non-root users ? Another >>>> option would be to add another mode 2 for ThreadPriorityPolicy >>>>> that documents the behavior (like mode 1 but without root-user check). >>>>> >>>>> If I get it right, even setting ***higher prios*** (lower niceness) is >>>> possible for non-root users on systems configured in an appropriate way >>>>> (using the CAP_SYS_NICE capability). >>>>> >>>>> But setting lower prio / higher niceness is even possible for normal users >>>> NOW without special config, it is just disabled by the root-check >>>>> which is very bad. >>>>> >>>>> Best regards, Matthias >>>>> From david.holmes at oracle.com Fri Dec 21 12:48:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 22:48:39 +1000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> Message-ID: On 21/12/2018 10:44 pm, Baesken, Matthias wrote: >> I'd be interested to know what the actual usecase is. But I think a >> native method would be the best way to handle it. > > Hi David, sorry I am not sure what native method are you talking about ? > Do you mean the application - coding should be rewritten from Java into some C/C++-JNI + Java mixture where the setpriority - calls to adjust the threads > are done from the C/C++ coding part ? Yes - one native method to set the native priority according to the semantics actually desired by the application. David > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Freitag, 21. Dezember 2018 13:33 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Cc: Haug, Gunter >> Subject: Re: ThreadPriorityPolicy settings for non-root users >> >> On 21/12/2018 9:09 pm, Baesken, Matthias wrote: >>>> >>>> Have you got real use cases for this? >>>> >>> >>> Hi David, yes we have a user who was setting the "infamous" >> ThreadPriorityPolicy=2 (+ then different prios/niceness levels for different >> threads) , but that is now no longer working in current JDK11 . >> >> I'd be interested to know what the actual usecase is. But I think a >> native method would be the best way to handle it. >> >> David >> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Freitag, 21. Dezember 2018 11:56 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Subject: Re: ThreadPriorityPolicy settings for non-root users >>>> >>>> On 21/12/2018 6:39 pm, Baesken, Matthias wrote: >>>>> Hi David , it might be that the functionality is seen as not very helpful >> by >>>> some and removed or deprecated in some future Java release. >>>>> >>>>> However it is present in the current JDKs and should work there nicely . >>>> >>>> Sorry but that's a bit naive. The code is old and bit-rotted and in some >>>> cases (Mac port) likely never used, so the idea that "it's there so it >>>> should work" is just not realistic - sorry. >>>> >>>>> Currently I have some points I do not like about the current state : >>>>> >>>>> - the root-check is not consistent , it is present on Linux / BSD (Mac) >> but >>>> I don't see it on Solaris >>>> >>>> Wasn't needed on Solaris. User-level capabilities sufficed. >>>> >>>>> - It ignores currently the CAP_SYS_NICE capability >>>> >>>> It never supported it. AFAIK the linux code doesn't really support any >>>> capability based permissions. >>>> >>>>> - it ignores that setting a higher niceness works nicely on most OS >>>> (checked Linux/Solaris/BSD) without being root (or having special >>>> capabilities) >>>> >>>> The priority control was never really about tweaking niceness levels. >>>> >>>>> - the root check makes testing hard (maybe that's why the Mac version >>>> was a bit broken?) >>>> >>>> Running under sudo isn't that hard. >>>> >>>> Sorry I'm not very supportive here - this isn't something that needs >>>> some minor tweaking to bring back online, it's something that may never >>>> have worked well in the first place. >>>> >>>> Have you got real use cases for this? >>>> >>>> Cheers, >>>> David >>>> >>>>> >>>>> Best regards, Matthias >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Donnerstag, 20. Dezember 2018 12:06 >>>>>> To: Baesken, Matthias ; 'hotspot- >>>>>> dev at openjdk.java.net' >>>>>> Subject: Re: ThreadPriorityPolicy settings for non-root users >>>>>> >>>>>> Hi Matthias, >>>>>> >>>>>> The more I think about this the more I see it as a huge can of worms. >>>>>> There are very, very, limited usecases for managing the priority of >>>>>> individual threads within a running Java application. Adjusting the >>>>>> process priority/nice-ness is effective and much simpler. >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>> On 20/12/2018 3:13 am, Baesken, Matthias wrote: >>>>>>> >>>>>>> Hello , >>>>>>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 >>>> (normal, >>>>>> the default) and 1, >>>>>>> the so called "aggressive" mode : >>>>>>> >>>>>>> "1 : Aggressive. "\ >>>>>>> " Java thread priorities map over to the entire range of "\ >>>>>>> " native thread priorities. Higher Java thread priorities map "\ >>>>>>> " to higher native thread priorities. This policy should be "\ >>>>>>> " used with care, as sometimes it can cause performance "\ >>>>>>> " degradation in the application and/or the entire system. On "\ >>>>>>> " Linux this policy requires root privilege.") \ >>>>>>> >>>>>>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp >> (the >>>>>> text in globals.hpp mentions only Linux which seems to be not fully >>>> correct): >>>>>>> >>>>>>> if (geteuid() != 0) { ... } in function prio_init(). >>>>>>> >>>>>>> (looks like the check is not done for other platforms). >>>>>>> >>>>>>> However the check for root (e.g. on Linux) hinders users to set a >>>>>> ****lower priority**** for a thread (== increase the "niceness" level) >>>>>>> when running as a non-root user (there might be strange ways from >>>>>> outside the VM with calling scripts and renice but .... ). >>>>>>> >>>>>>> >>>>>>> In older JDKs (e.g. JDK8) there was a "workaround" to use for >> example >>>>>> ThreadPriorityPolicy=2 to avoid the root-check, >>>>>>> but this is not possible any more in recent JDKs (10/11) after the range >>>>>> check (0,1) has been introduced for the ThreadPriorityPolicy flag >>>>>>> (and probably the old workaround was not a good one anyway >> because >>>> it >>>>>> was undocumented). >>>>>>> >>>>>>> So do you think we could introduce another XX-flag ( >>>>>> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) >>>>>>> that allows using the "aggressive" mode for non-root users ? Another >>>>>> option would be to add another mode 2 for ThreadPriorityPolicy >>>>>>> that documents the behavior (like mode 1 but without root-user >> check). >>>>>>> >>>>>>> If I get it right, even setting ***higher prios*** (lower niceness) is >>>>>> possible for non-root users on systems configured in an appropriate >> way >>>>>>> (using the CAP_SYS_NICE capability). >>>>>>> >>>>>>> But setting lower prio / higher niceness is even possible for normal >> users >>>>>> NOW without special config, it is just disabled by the root-check >>>>>>> which is very bad. >>>>>>> >>>>>>> Best regards, Matthias >>>>>>> From sgehwolf at redhat.com Fri Dec 21 12:52:54 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 21 Dec 2018 13:52:54 +0100 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> <48cbf65f09472c2e233a8ee64105467f14679a2d.camel@redhat.com> Message-ID: <82335ba5d00fe8f73fde83fa0bb67d4c1d5270f6.camel@redhat.com> Hi Adrian, On Fri, 2018-12-21 at 13:27 +0100, John Paul Adrian Glaubitz wrote: > Hi Severin! > > On 12/21/18 10:12 AM, Severin Gehwolf wrote: > > It could be that you are hitting a GCC bug specific to linux sparc. > > Then again, why wouldn't you hit it for a server JVM. As a temporary > > work-around you could try this (compile without opt as prior JDK- > > 8210425, not tested): > > > > diff --git a/make/hotspot/lib/JvmOverrideFiles.gmk b/make/hotspot/lib/JvmOverrideFiles.gmk > > --- a/make/hotspot/lib/JvmOverrideFiles.gmk > > +++ b/make/hotspot/lib/JvmOverrideFiles.gmk > > @@ -49,6 +49,15 @@ > > # by using -ffp-contract=off on GCC/Clang platforms. > > ifneq ($(FDLIBM_CFLAGS), ) > > LIBJVM_FDLIBM_COPY_OPT_FLAG := $(CXX_O_FLAG_NORM) > > + # Don't optimize fdlibm-fork for Zero on Linux spark > > + # See JDK-XXXX > > + ifeq ($(call check-jvm-feature, zero), true) > > + ifeq ($(OPENJDK_TARGET_OS), linux) > > + ifeq ($(OPENJDK_TARGET_CPU_ARCH), spark) > > + LIBJVM_FDLIBM_COPY_OPT_FLAG := $(CXX_O_FLAG_NONE) > > + endif > > + endif > > + endif > > endif > > This indeed fixes the problem for me, with the spelling fix for "sparc" ;-). OK. It's rather curious. What are the different flags being used? Pre/post patch? It looks as if this happens (on Linux sparc): Zero and '-ffp-contract=off -O2' => bad build Zero and '-ffp-contract=off -O0' => good build Server and '-ffp-contract=off -O2' => good build Thanks, Severin From glaubitz at physik.fu-berlin.de Fri Dec 21 12:57:26 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 21 Dec 2018 13:57:26 +0100 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: <82335ba5d00fe8f73fde83fa0bb67d4c1d5270f6.camel@redhat.com> References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> <48cbf65f09472c2e233a8ee64105467f14679a2d.camel@redhat.com> <82335ba5d00fe8f73fde83fa0bb67d4c1d5270f6.camel@redhat.com> Message-ID: <022362a2-9634-1156-f542-12338ed85e2e@physik.fu-berlin.de> Hi Severin! On 12/21/18 1:52 PM, Severin Gehwolf wrote: > OK. It's rather curious. What are the different flags being used? > Pre/post patch? It looks as if this happens (on Linux sparc): > > Zero and '-ffp-contract=off -O2' => bad build > Zero and '-ffp-contract=off -O0' => good build > Server and '-ffp-contract=off -O2' => good build Here's the build log for openjdk-12 on Debian unstable (sparc64) for both the Server and Zero build - it first builds Server, then Zero: > https://buildd.debian.org/status/fetch.php?pkg=openjdk-12&arch=sparc64&ver=12%7E24-2&stamp=1544851016&raw=0 Will provide you the build log with your patch for Zero in a bit. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From christoph.langer at sap.com Fri Dec 21 13:12:06 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 21 Dec 2018 13:12:06 +0000 Subject: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check return code of pthread_getschedparam In-Reply-To: References: Message-ID: <670b89cded144b14aa66ecf345bfff4c@sap.com> Hi Matthias, looks good to me. Best regards Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Baesken, Matthias > Sent: Freitag, 21. Dezember 2018 08:01 > To: David Holmes ; 'hotspot- > dev at openjdk.java.net' > Subject: RFR [XS] 8215707: [macosx] fix pthread_getschedparam and > pthread_setschedparam calls - was : RE: RFR [XS] : 8215707: [macosx] check > return code of pthread_getschedparam > > Hello David, I adjusted the bug synopsis and description, the second > webrev uses the pthread id of the passed Thread* instead of > pthread_self() : > > > https://bugs.openjdk.java.net/browse/JDK-8215707 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215707.1/ > > > > Best regards, Matthias > > > > -----Original Message----- > > From: David Holmes > > Sent: Donnerstag, 20. Dezember 2018 13:19 > > To: Baesken, Matthias ; 'hotspot- > > dev at openjdk.java.net' > > Subject: Re: RFR [XS] : 8215707: [macosx] check return code of > > pthread_getschedparam > > > > On 20/12/2018 9:56 pm, Baesken, Matthias wrote: > > >> Note this code is likely never exercised due to the default > > >> ThreadPriorityPolicy of 0. > > > > > > Hi David, ThreadPriorityPolicy=1 is a valid setting / product flag. > > > > Yes it is. That doesn't mean it is actually used to any great extent. > > > > > You find quite a few people setting ThreadPriorityPolicy when you search > > for it, so the coding should better be fixed I guess . > > > > Quite a few? I could only find a couple (Cassandra, Gatling) and they > > were on Linux. I was surprised to find how many people knew about the > > bug that allowed the root check to be by-passed. > > > > The fact this code is incorrect on OS X indicates to me it is not > > actually being used - or by chance priority is only ever changed in the > > current thread. > > > > >> > > >> That seems more likely given that the code is incorrect: > > >> > > >> pthread_getschedparam(pthread_self(), &policy, &sp); > > >> > > >> this always gets the priority of the current thread, but the method need > > >> not be called on the current thread! > > > > > > Yes , true ! > > > Same for os::set_native_priority in os_bsd.cpp , there pthread_self() is > > also used instead of the real Thread . > > > > > > 2338OSReturn os::set_native_priority(Thread* thread, int newpri) { > > > 2339 if (!UseThreadPriorities || ThreadPriorityPolicy == 0) return OS_OK; > > > 2340 > > > .... > > > 2346#elif defined(__APPLE__) || defined(__NetBSD__) > > > 2347 struct sched_param sp; > > > 2348 int policy; > > > 2349 pthread_t self = pthread_self(); > > > 2350 > > > 2351 if (pthread_getschedparam(self, &policy, &sp) != 0) { > > > 2352 return OS_ERR; > > > 2353 } > > > > If you intend fixing this probably best to modify the bug synopsis and > > description. > > > > Cheers, > > David > > From glaubitz at physik.fu-berlin.de Fri Dec 21 13:34:45 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 21 Dec 2018 14:34:45 +0100 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: <82335ba5d00fe8f73fde83fa0bb67d4c1d5270f6.camel@redhat.com> References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> <48cbf65f09472c2e233a8ee64105467f14679a2d.camel@redhat.com> <82335ba5d00fe8f73fde83fa0bb67d4c1d5270f6.camel@redhat.com> Message-ID: On 12/21/18 1:52 PM, Severin Gehwolf wrote: > OK. It's rather curious. What are the different flags being used? > Pre/post patch? It looks as if this happens (on Linux sparc): > > Zero and '-ffp-contract=off -O2' => bad build > Zero and '-ffp-contract=off -O0' => good build > Server and '-ffp-contract=off -O2' => good build Logs here: https://people.debian.org/~glaubitz/openjdk-sparc/ Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From matthias.baesken at sap.com Fri Dec 21 13:38:06 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Dec 2018 13:38:06 +0000 Subject: ThreadPriorityPolicy settings for non-root users In-Reply-To: References: <196dbe97-31ca-03e5-4f0e-4018f5a85c0c@oracle.com> <1419088f-baf2-f0c1-3cde-9f66abed0e97@oracle.com> Message-ID: >If you want to just disable the root check to get back the erroneous >ThreadPriorityPolicy=2 behaviour then file a bug and a CSR and propose a >patch. I don't think we want yet another flag for this though. Just drop >the root check for ThreadPriorityPolicy=1 and let the underlying system >permissions control success or failure. Hi David, I think we will go this way ! ( However not the root check made the special case ThreadPriorityPolicy=2 fail, but the added range check for ThreadPriorityPolicy in globals.hpp ( 8122937 ) which introduced range(0, 1) ) . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Freitag, 21. Dezember 2018 13:47 > To: Lindenmaier, Goetz ; Baesken, Matthias > ; 'hotspot-dev at openjdk.java.net' dev at openjdk.java.net> > Subject: Re: ThreadPriorityPolicy settings for non-root users > > On 21/12/2018 9:24 pm, Lindenmaier, Goetz wrote: > > Hi David, > > > > Yes, as Matthias stated we have at least one application that relies on > > thread priorities. Their going to Java 11 is blocked by this. > > I'd like to hear more. I'd be very surprised that any actual functional > correctness issue is addressed by playing with niceness. > > > Also are you aware that this is the implementation of Thread.setPriority()? > > There are no specified semantics for Thread priorities in Java, as they > pertain to operating system priorities, or relative scheduling across > processes. So there is nothing broken with regard to the specification > with the default behaviour (ThreadPriorityPolicy=0). > > > So isn't it great we finally fix the old and bit-rotten code that implements > > this official method? > > No. Thread priorities are a minefield. Giving threads priorities was one > of the biggest mistakes in the Java thread API. People were advised to > just forget about them 20+ years ago - and most took that advice. I > think the only reason this wasn't all stripped out years ago was some > corner cases with Solaris scheduling classes. > > If you want to just disable the root check to get back the erroneous > ThreadPriorityPolicy=2 behaviour then file a bug and a CSR and propose a > patch. I don't think we want yet another flag for this though. Just drop > the root check for ThreadPriorityPolicy=1 and let the underlying system > permissions control success or failure. > > David > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: hotspot-dev On Behalf > Of > >> David Holmes > >> Sent: Freitag, 21. Dezember 2018 11:56 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Subject: Re: ThreadPriorityPolicy settings for non-root users > >> > >> On 21/12/2018 6:39 pm, Baesken, Matthias wrote: > >>> Hi David , it might be that the functionality is seen as not very helpful > >> by some and removed or deprecated in some future Java release. > >>> > >>> However it is present in the current JDKs and should work there nicely . > >> > >> Sorry but that's a bit naive. The code is old and bit-rotted and in some > >> cases (Mac port) likely never used, so the idea that "it's there so it > >> should work" is just not realistic - sorry. > >> > >>> Currently I have some points I do not like about the current state : > >>> > >>> - the root-check is not consistent , it is present on Linux / BSD (Mac) > but > >> I don't see it on Solaris > >> > >> Wasn't needed on Solaris. User-level capabilities sufficed. > >> > >>> - It ignores currently the CAP_SYS_NICE capability > >> > >> It never supported it. AFAIK the linux code doesn't really support any > >> capability based permissions. > >> > >>> - it ignores that setting a higher niceness works nicely on most OS > >> (checked Linux/Solaris/BSD) without being root (or having special > >> capabilities) > >> > >> The priority control was never really about tweaking niceness levels. > >> > >>> - the root check makes testing hard (maybe that's why the Mac version > >> was a bit broken?) > >> > >> Running under sudo isn't that hard. > >> > >> Sorry I'm not very supportive here - this isn't something that needs > >> some minor tweaking to bring back online, it's something that may never > >> have worked well in the first place. > >> > >> Have you got real use cases for this? > >> > >> Cheers, > >> David > >> > >>> > >>> Best regards, Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Donnerstag, 20. Dezember 2018 12:06 > >>>> To: Baesken, Matthias ; 'hotspot- > >>>> dev at openjdk.java.net' > >>>> Subject: Re: ThreadPriorityPolicy settings for non-root users > >>>> > >>>> Hi Matthias, > >>>> > >>>> The more I think about this the more I see it as a huge can of worms. > >>>> There are very, very, limited usecases for managing the priority of > >>>> individual threads within a running Java application. Adjusting the > >>>> process priority/nice-ness is effective and much simpler. > >>>> > >>>> Cheers, > >>>> David > >>>> > >>>> On 20/12/2018 3:13 am, Baesken, Matthias wrote: > >>>>> > >>>>> Hello , > >>>>> currently OpenJDK supports 2 ThreadPriorityPolicy settings, 0 > >> (normal, > >>>> the default) and 1, > >>>>> the so called "aggressive" mode : > >>>>> > >>>>> "1 : Aggressive. "\ > >>>>> " Java thread priorities map over to the entire range of "\ > >>>>> " native thread priorities. Higher Java thread priorities map "\ > >>>>> " to higher native thread priorities. This policy should be "\ > >>>>> " used with care, as sometimes it can cause performance "\ > >>>>> " degradation in the application and/or the entire system. On "\ > >>>>> " Linux this policy requires root privilege.") \ > >>>>> > >>>>> Currently we check directly for root in os_bsd.cpp and os_linux.cpp > (the > >>>> text in globals.hpp mentions only Linux which seems to be not fully > >> correct): > >>>>> > >>>>> if (geteuid() != 0) { ... } in function prio_init(). > >>>>> > >>>>> (looks like the check is not done for other platforms). > >>>>> > >>>>> However the check for root (e.g. on Linux) hinders users to set a > >>>> ****lower priority**** for a thread (== increase the "niceness" level) > >>>>> when running as a non-root user (there might be strange ways from > >>>> outside the VM with calling scripts and renice but .... ). > >>>>> > >>>>> > >>>>> In older JDKs (e.g. JDK8) there was a "workaround" to use for > example > >>>> ThreadPriorityPolicy=2 to avoid the root-check, > >>>>> but this is not possible any more in recent JDKs (10/11) after the range > >>>> check (0,1) has been introduced for the ThreadPriorityPolicy flag > >>>>> (and probably the old workaround was not a good one anyway > because > >> it > >>>> was undocumented). > >>>>> > >>>>> So do you think we could introduce another XX-flag ( > >>>> AllowAggressiveThreadPriorityPolicyForAllUsers, or some better name) > >>>>> that allows using the "aggressive" mode for non-root users ? Another > >>>> option would be to add another mode 2 for ThreadPriorityPolicy > >>>>> that documents the behavior (like mode 1 but without root-user > check). > >>>>> > >>>>> If I get it right, even setting ***higher prios*** (lower niceness) is > >>>> possible for non-root users on systems configured in an appropriate > way > >>>>> (using the CAP_SYS_NICE capability). > >>>>> > >>>>> But setting lower prio / higher niceness is even possible for normal > users > >>>> NOW without special config, it is just disabled by the root-check > >>>>> which is very bad. > >>>>> > >>>>> Best regards, Matthias > >>>>> From gunter.haug at sap.com Fri Dec 21 13:52:26 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Fri, 21 Dec 2018 13:52:26 +0000 Subject: RFR [XXS]: Tiny bug in VM monitoring/management In-Reply-To: <3185e4bb-be42-c899-cfe5-34b2edfb5e44@oracle.com> References: <3185e4bb-be42-c899-cfe5-34b2edfb5e44@oracle.com> Message-ID: <70CBF2DD-4442-4A1F-BC12-308BE60D714F@sap.com> Hi David, Thanks for looking into this. > But I have to wonder if this will suddenly turn this on or > off for some platform where it was off or on?? Ideally we will find both > flags are normally set the same. I don't think it's a problem. _thread_allocated_memory_enabled is always on, the VM takes care for that value regardless of the OS. _thread_cpu_time_enabled is turned on depending of the capabilities of the OS. Most systems support it, one exception is non-Apple BSD. In this case is_thread_allocated_memory_enabled() currently returns false even though it is actually supported by the VM. Best, Gunter ?On 21.12.18, 12:02, "David Holmes" wrote: Hi Gunter, On 21/12/2018 8:37 pm, Haug, Gunter wrote: > Hi all, > > can I please have reviews and a sponsor fort the following change: > > https://bugs.openjdk.java.net/browse/JDK-8215791 > http://cr.openjdk.java.net/~ghaug/webrevs/8215791/ > > This was obviously just a copy and paste mishap. Obviously! But I have to wonder if this will suddenly turn this on or off for some platform where it was off or on?? Ideally we will find both flags are normally set the same. > Thanks and happy holidays, Likewise! Cheers, David > Gunter > From volker.simonis at gmail.com Fri Dec 21 14:59:07 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 21 Dec 2018 15:59:07 +0100 Subject: RFR [XXS]: Tiny bug in VM monitoring/management In-Reply-To: <70CBF2DD-4442-4A1F-BC12-308BE60D714F@sap.com> References: <3185e4bb-be42-c899-cfe5-34b2edfb5e44@oracle.com> <70CBF2DD-4442-4A1F-BC12-308BE60D714F@sap.com> Message-ID: Hi Gunter, the change looks good to me. I can sponsor it. On Fri, Dec 21, 2018 at 2:52 PM Haug, Gunter wrote: > > Hi David, > > Thanks for looking into this. > > > But I have to wonder if this will suddenly turn this on or > > off for some platform where it was off or on?? Ideally we will find both > > flags are normally set the same. > > I don't think it's a problem. _thread_allocated_memory_enabled is always on, the VM takes care for that value regardless of the OS. _thread_cpu_time_enabled is turned on depending of the capabilities of the OS. Most systems support it, one exception is non-Apple BSD. In this case is_thread_allocated_memory_enabled() currently returns false even though it is actually supported by the VM. > It could also (unintentionally) have been switched off on Windows if the call to "GetThreadTimes()" in "os::is_thread_cpu_time_supported()" has failed. But I don't think that usually happened. And anyway, as Gunter wrote, "_thread_allocated_memory_enabled" is supported on any system anyway. So I think it is safe to push this. Regards, Volker > Best, > Gunter > > ?On 21.12.18, 12:02, "David Holmes" wrote: > > Hi Gunter, > > On 21/12/2018 8:37 pm, Haug, Gunter wrote: > > Hi all, > > > > can I please have reviews and a sponsor fort the following change: > > > > https://bugs.openjdk.java.net/browse/JDK-8215791 > > http://cr.openjdk.java.net/~ghaug/webrevs/8215791/ > > > > This was obviously just a copy and paste mishap. > > Obviously! But I have to wonder if this will suddenly turn this on or > off for some platform where it was off or on?? Ideally we will find both > flags are normally set the same. > > > Thanks and happy holidays, > > Likewise! > > Cheers, > David > > > Gunter > > > > From daniel.daugherty at oracle.com Fri Dec 21 21:35:50 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Dec 2018 16:35:50 -0500 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> Message-ID: David, Sorry for the delay in getting to this review. On 12/19/18 4:21 PM, David Holmes wrote: > Following on from the preliminary RFR: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html > > > I've merged with Kim's changes from 8215097. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 > Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ src/hotspot/os/linux/os_linux.cpp ??? No comments. src/hotspot/share/gc/parallel/gcTaskThread.cpp ??? Nit - needs a copyright year update. src/hotspot/share/gc/shared/concurrentGCThread.cpp ??? Nit - needs a copyright year update. src/hotspot/share/gc/shared/workgroup.cpp ??? No comments. src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp ??? No comments. src/hotspot/share/runtime/thread.cpp ??? L426: ? assert(_run_state == PRE_CALL_RUN || ??? L427: ???????? _run_state == POST_RUN, "Active Thread deleted before post_run()"); ??????? Adding a little more debug info would be helpful. Please consider: ????????? assert(_run_state == PRE_CALL_RUN || ???????????????? _run_state == POST_RUN, "Active Thread deleted before post_run(): " ???????????????? "_run_state=%d", (int)_run_state); ??? L1082: ? assert(_starting_thread == NULL, "already initialized"); ??????? Adding a little more debug info would be helpful. Please consider: ????????? assert(_starting_thread == NULL, "already initialized: " ???????????????? "_starting_thread=" INTPTR_FORMAT, _starting_thread); src/hotspot/share/runtime/thread.hpp ??? L112-L135: ??????? I like the content of the new comment block. I'm not fond of ? ? ? ? using yet another block comment style: ????????? /* ??????????? ?????????? */ ??????? We already have: ????????? /* ?????????? * ?????????? */ ??????? and (my preference): ????????? // ????????? // ????????? // src/hotspot/share/runtime/vmThread.cpp ??? No comments. src/hotspot/share/services/management.cpp ??? No comments. test/hotspot/gtest/threadHelper.inline.hpp ??? No comments. Thumbs up. I don't need to see another webrev if you choose to fix the few things I noted above. Dan > > Changes since previous discussion: > > - Replaced Kim's initial_thread_created assertion with a check of > Thread::current_or_null, with an explanation > > - Removed NJT list addition at construction time as it's not needed > after 8215097. > > - Moved the JavaThread call to this->exit(false); from the end of > thread_main_inner to the start of post_run - as Markus suggested. This > also necessitated a change to the gtest logic compared to the original > as it has to override post_run() now to avoid the exit call. > > - Made get_thread_name_string virtual so that the JavaThread > subclasses in the gtest can override it and allow their names to be > seen in hs_err files. (Makes debugging so much easier when you know > which thread crashed!). > > Testing: > ?- Mach 5 tiers 1 - 3 > ?- All runtime tests with: > ?? - default GC > ?? - ZGC > ?? - Shenandoah GC > - gc/g1 > - gc/serial > - gc/epsilon > - gc/shenandoah > > Thanks, > David From david.holmes at oracle.com Fri Dec 21 22:04:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 22 Dec 2018 08:04:03 +1000 Subject: RFR [XXS]: Tiny bug in VM monitoring/management In-Reply-To: <70CBF2DD-4442-4A1F-BC12-308BE60D714F@sap.com> References: <3185e4bb-be42-c899-cfe5-34b2edfb5e44@oracle.com> <70CBF2DD-4442-4A1F-BC12-308BE60D714F@sap.com> Message-ID: On 21/12/2018 11:52 pm, Haug, Gunter wrote: > Hi David, > > Thanks for looking into this. > >> But I have to wonder if this will suddenly turn this on or >> off for some platform where it was off or on?? Ideally we will find both >> flags are normally set the same. > > I don't think it's a problem. _thread_allocated_memory_enabled is always on, the VM takes care for that value regardless of the OS. _thread_cpu_time_enabled is turned on depending of the capabilities of the OS. Most systems support it, one exception is non-Apple BSD. In this case is_thread_allocated_memory_enabled() currently returns false even though it is actually supported by the VM. Sounds good. Thanks for checking. David > Best, > Gunter > > ?On 21.12.18, 12:02, "David Holmes" wrote: > > Hi Gunter, > > On 21/12/2018 8:37 pm, Haug, Gunter wrote: > > Hi all, > > > > can I please have reviews and a sponsor fort the following change: > > > > https://bugs.openjdk.java.net/browse/JDK-8215791 > > http://cr.openjdk.java.net/~ghaug/webrevs/8215791/ > > > > This was obviously just a copy and paste mishap. > > Obviously! But I have to wonder if this will suddenly turn this on or > off for some platform where it was off or on?? Ideally we will find both > flags are normally set the same. > > > Thanks and happy holidays, > > Likewise! > > Cheers, > David > > > Gunter > > > > From david.holmes at oracle.com Fri Dec 21 22:10:47 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 22 Dec 2018 08:10:47 +1000 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> Message-ID: <521b1a43-7be6-6327-2d1a-db571dd6441f@oracle.com> Hi Dan, On 22/12/2018 7:35 am, Daniel D. Daugherty wrote: > David, > > Sorry for the delay in getting to this review. No problem thanks for looking at it. I'm still waiting for Kim and he's already on Xmas vacation so no rush. I'll deal with this in January. I'll take up your suggestions below. Thanks, David > > On 12/19/18 4:21 PM, David Holmes wrote: >> Following on from the preliminary RFR: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html >> >> >> I've merged with Kim's changes from 8215097. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ > > src/hotspot/os/linux/os_linux.cpp > ??? No comments. > > src/hotspot/share/gc/parallel/gcTaskThread.cpp > ??? Nit - needs a copyright year update. > > src/hotspot/share/gc/shared/concurrentGCThread.cpp > ??? Nit - needs a copyright year update. > > src/hotspot/share/gc/shared/workgroup.cpp > ??? No comments. > > src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp > ??? No comments. > > src/hotspot/share/runtime/thread.cpp > ??? L426: ? assert(_run_state == PRE_CALL_RUN || > ??? L427: ???????? _run_state == POST_RUN, "Active Thread deleted > before post_run()"); > ??????? Adding a little more debug info would be helpful. Please consider: > > ????????? assert(_run_state == PRE_CALL_RUN || > ???????????????? _run_state == POST_RUN, "Active Thread deleted before > post_run(): " > ???????????????? "_run_state=%d", (int)_run_state); > > ??? L1082: ? assert(_starting_thread == NULL, "already initialized"); > ??????? Adding a little more debug info would be helpful. Please consider: > > ????????? assert(_starting_thread == NULL, "already initialized: " > ???????????????? "_starting_thread=" INTPTR_FORMAT, _starting_thread); > > src/hotspot/share/runtime/thread.hpp > ??? L112-L135: > ??????? I like the content of the new comment block. I'm not fond of > ? ? ? ? using yet another block comment style: > > ????????? /* > ??????????? > ?????????? */ > > ??????? We already have: > > ????????? /* > ?????????? * > ?????????? */ > > ??????? and (my preference): > > ????????? // > ????????? // > ????????? // > > > src/hotspot/share/runtime/vmThread.cpp > ??? No comments. > > src/hotspot/share/services/management.cpp > ??? No comments. > > test/hotspot/gtest/threadHelper.inline.hpp > ??? No comments. > > Thumbs up. I don't need to see another webrev if you choose to > fix the few things I noted above. > > Dan > > >> >> Changes since previous discussion: >> >> - Replaced Kim's initial_thread_created assertion with a check of >> Thread::current_or_null, with an explanation >> >> - Removed NJT list addition at construction time as it's not needed >> after 8215097. >> >> - Moved the JavaThread call to this->exit(false); from the end of >> thread_main_inner to the start of post_run - as Markus suggested. This >> also necessitated a change to the gtest logic compared to the original >> as it has to override post_run() now to avoid the exit call. >> >> - Made get_thread_name_string virtual so that the JavaThread >> subclasses in the gtest can override it and allow their names to be >> seen in hs_err files. (Makes debugging so much easier when you know >> which thread crashed!). >> >> Testing: >> ?- Mach 5 tiers 1 - 3 >> ?- All runtime tests with: >> ?? - default GC >> ?? - ZGC >> ?? - Shenandoah GC >> - gc/g1 >> - gc/serial >> - gc/epsilon >> - gc/shenandoah >> >> Thanks, >> David > From daniel.daugherty at oracle.com Fri Dec 21 22:21:45 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Dec 2018 17:21:45 -0500 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <01873a0f-a0fb-18b9-f7d4-98bb638e9b57@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> <70669453-e317-a30d-8d5a-e5b938b83c41@oracle.com> <4fb6cd22-cdd0-2419-c863-24b250ac0b16@oracle.com> <2a2679cc-b0e0-f8d0-7336-8666e1a42950@oracle.com> <01873a0f-a0fb-18b9-f7d4-98bb638e9b57@oracle.com> Message-ID: <26ebf8fe-f96e-7c27-3a69-f0159ba8227d@oracle.com> On 12/21/18 4:45 AM, Robbin Ehn wrote: > Hi David, > > On 2018-12-21 04:17, David Holmes wrote: >> >> Got it - subtle. >> >> Further this sounds like a race that could lead to bugs if not used >> very carefully ie. you can't assume between disarm() and wake() that >> all threads are blocked. > > I didn't realize how subtle this is. I think your original comment that > disarm/wake should be one operation was spot on. > Investigating... thinking... testing... yes I think this will work, > fixed! > Sorry for not looking more into this before. > >> >> I think perhaps this needs to be expanded to make this more obvious: >> >> ?? 68 //??? - A call to wait(tag) will block if the barrier is armed >> with the value >> ?? 69 //????? 'tag'; else it will return immediately. >> ?? 70 //??? - A blocked thread is eligible to execute again once the >> barrier is >> ?? 71 //????? disarmed and wake() has been called. >> +????????? - A call to wait(tag) that would block if it continued, >> but instead >> +??????????? is descheduled, may return immediately if scheduled after a >> +?????????? call to disarm(), but before the call to wake(). >> >> It also made me realize that in the general case (not when used with >> safepoints I think due to other state checks) a wake() may stall due >> to threads with a previous tag entering the wait() late. > > I added a double checking in the semaphore version, this means both > implementation should have progress guarantee. > > Making this v5 a bit large due to a lot of comments being changed. > > Inc: > http://cr.openjdk.java.net/~rehn/8214271/5/inc/webrev/ > Full: > http://cr.openjdk.java.net/~rehn/8214271/5/full/webrev/ It's been a while since I've reviewed this topic so I'm going with the full webrev. Sorry for the long delay in getting back to this: src/hotspot/os/linux/waitBarrier_linux.hpp ??? No comments. src/hotspot/os/linux/waitBarrier_linux.cpp ??? L35: ??? check_type(cond, "%s; error='%s' (errno=%s)", msg, os::strerror(err),?? \ ??????? nit - The '%s;' should probably be a '%s:' to match other mesg styles. ??? L41: static int futex(volatile int *addr, int futex_op, int op_arg) ??? L42: { ??????? nit - the '{' on L42 should be at the end of L41 (with a space in front of it). ??? L47: ? assert(_futex_barrier == 0, "Already armed"); ??????? Please consider: ????????? assert(_futex_barrier == 0, "Should not be already armed: " ???????????????? "_futex_barrier=%d", _futex_barrier); ??? L53: ? assert(_futex_barrier != 0, "Not armed"); ??????? Please consider: ????????? assert(_futex_barrier != 0, "Should be armed/non-zero."); ??? L58: ? guarantee_with_errno(s > -1, "futex FUTEX_WAKE"); ??????? Please consider: ?????????? guarantee_with_errno(s > -1, "futex FUTEX_WAKE failed: s=%d", s); ??? L72: ??? guarantee_with_errno((s == 0) || ??? L73: ???????????????????????? (s == -1 && errno == EAGAIN) || ??? L74: ???????????????????????? (s == -1 && errno == EINTR), ??? L75: ???????????????????????? "futex FUTEX_WAIT"); ??????? Please consider: ????????? "futex FUTEX_WAIT failed: s=%d", s); ??? L76: ??? // Return value 0: woken up, but re-check in case of spurious wakeup ??? L78: ??? // Error EAGAIN: we are already disarmed and so will pass the check ??????? Please add a period to the end of these two lines. src/hotspot/share/utilities/waitBarrier.hpp ??? I like the new header comment. ??? L102: ? // Returns implementation type. ??????? "type" or "description"? ??? L109-111: indent needs two more spaces. src/hotspot/share/utilities/waitBarrier_generic.hpp ??? L31: // Except for the barrier tag itself, it uses two counters to keep the semaphore ??????? Perhaps: ????????? // In addition to the barrier tag, it uses two counters to keep the semaphore src/hotspot/share/utilities/waitBarrier_generic.cpp ??? L61: ? // Loads of _barrier_threads/_waiters must not float above disarm store. ??? L62: ? OrderAccess::fence(); ??????? Since you are using fence() here, you should also say that "and disarm store ??????? must not float below." ??? L73: ? OrderAccess::fence(); ??????? Missing your rationale comment for this fence(). ??? L79: ??? OrderAccess::fence(); ??????? I was expecting this one to be OrderAccess::loadload(). ??????? Also, missing your rationale comment for this OrderAccess use. test/hotspot/gtest/utilities/test_waitBarrier.cpp ??? L59: ????? OrderAccess::storeload(); // Loads in WB must not float up. ?? ???? Not sure why this storeload() is here instead of being in wait(). ??????? Seems like wait() should do the "right thing". Thumbs up on the code! I don't need to see a new webrev if you choose to fix the minor things I pointed out above. Dan > > gtest passes thousands of loops locally and hundreds in mach5. > > Thanks, Robbin > >> >> Thanks, >> David >> >>>> >>>> s/Implementation/Implementations/ >>> >>> Fixed >>> >>>> >>>> The fourth line is no longer needed. >>> >>> Above is the reason I would like to keep the fourth line, since only >>> if you call >>> both disarm() and wake() you have that guarantee that waiter threads >>> will >>> return. >>> >>> Thanks, Robbin >>> >>>> >>>> Thanks, >>>> David >>>> >>>> >>>>> Inc: >>>>> http://cr.openjdk.java.net/~rehn/8214271/4/inc/webrev/ >>>>> >>>>> Full: >>>>> http://cr.openjdk.java.net/~rehn/8214271/4/full/webrev/ >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Otherwise this all looks good! >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> >>>>>>> Full: >>>>>>> http://cr.openjdk.java.net/~rehn/8214271/3/full/webrev/ >>>>>>> >>>>>>> Thanks, Robbin >>>>>>> >>>>>>> On 11/23/18 5:55 PM, Robbin Ehn wrote: >>>>>>>> Forgot RFR in subject. >>>>>>>> >>>>>>>> /Robbin >>>>>>>> >>>>>>>> On 2018-11-23 17:51, Robbin Ehn wrote: >>>>>>>>> Hi all, please review. >>>>>>>>> >>>>>>>>> When a safepoint is ended we need a way to get back to 100% >>>>>>>>> utilization as fast >>>>>>>>> as possible. 100% utilization means no idle cpu in the system >>>>>>>>> if there is a >>>>>>>>> JavaThread that could be executed. The traditional ways to >>>>>>>>> wake many, e.g. >>>>>>>>> semaphore, pthread_cond, is not implemented with a single >>>>>>>>> syscall instead they >>>>>>>>> typical do one syscall per thread to wake. >>>>>>>>> >>>>>>>>> This change-set contains that primitive, the WaitBarrier, and >>>>>>>>> a gtest for it. >>>>>>>>> No actual users, which is in coming patches. >>>>>>>>> >>>>>>>>> The WaitBarrier solves by doing a cooperative semaphore >>>>>>>>> posting, threads woken >>>>>>>>> will also post. On Linux we can instead directly use a futex >>>>>>>>> and with one >>>>>>>>> syscall wake all. Depending on how many threads and cpus the >>>>>>>>> performance vary, >>>>>>>>> but a good utilization of the machine, just on the edge of >>>>>>>>> saturated, the time to reach 100% utilization is around 3 >>>>>>>>> times faster with the WaitBarrier (where futex is faster than >>>>>>>>> semaphore). >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >>>>>>>>> >>>>>>>>> CR: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214271 >>>>>>>>> >>>>>>>>> Passes 100 iterations of gtest on our platforms, both >>>>>>>>> fastdebug and release. >>>>>>>>> And have been stable when used in safepoints (t1-8) (coming >>>>>>>>> patches). >>>>>>>>> >>>>>>>>> Thanks, Robbin From stefan.johansson at oracle.com Sat Dec 22 12:15:28 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Sat, 22 Dec 2018 13:15:28 +0100 Subject: [12] RFR: 8215898: Build broken on 32-bit after JDK-8211425 In-Reply-To: <7344df42-7fef-036d-e554-d4cd5d898875@oracle.com> References: <7344df42-7fef-036d-e554-d4cd5d898875@oracle.com> Message-ID: <9fa5b29d-6a81-bdf0-7987-b563ca4db7f0@oracle.com> Moving to hotspot-dev to reach a wider range of reviewers. -------- Forwarded Message -------- Subject: 8215898: Build broken on 32-bit after JDK-8211425 Date: Sat, 22 Dec 2018 11:52:34 +0100 From: Stefan Johansson Organization: Oracle Corporation To: hotspot-gc-dev at openjdk.java.net Hi all, Please review this small fix for a build failure: Bug: https://bugs.openjdk.java.net/browse/JDK-8215898 Webrev: http://cr.openjdk.java.net/~sjohanss/8215898/00/ Summary The build failure is caused by always setting UseCompressedOops and UseCompressedClassPointers to false when AllocateOldGenAt is used. This needs to be guarded with LP64_ONLY. Testing Built the previously failing platforms and they now pass. Thanks, Stefan From stefan.johansson at oracle.com Sat Dec 22 12:14:51 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Sat, 22 Dec 2018 13:14:51 +0100 Subject: [12] RFR: 8215897: Build broken on zero after JDK-8211424 In-Reply-To: References: Message-ID: <25e97a64-6d54-4184-ae45-508910df1539@oracle.com> Moving to hotspot-dev to reach a wider range of reviewers. -------- Forwarded Message -------- Subject: 8215897: Build broken on zero after JDK-8211424 Date: Sat, 22 Dec 2018 12:01:00 +0100 From: Stefan Johansson Organization: Oracle Corporation To: hotspot-gc-dev at openjdk.java.net Hi, Please review this fix for a build failure on Zero (or platforms excluding G1 or Parallel). Bug: https://bugs.openjdk.java.net/browse/JDK-8215897 Webrev: http://cr.openjdk.java.net/~sjohanss/8215897/00/ Summary The new whitebox methods for the AllocateOldGenAt feature was guarded by (G1 | Parallel), and this will not work if one of the GCs are not present in the build. Testing Built the previously failing platforms. Thanks, Stefan From thomas.schatzl at oracle.com Sat Dec 22 12:54:04 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sat, 22 Dec 2018 13:54:04 +0100 Subject: [12] RFR: 8215898: Build broken on 32-bit after JDK-8211425 In-Reply-To: <9fa5b29d-6a81-bdf0-7987-b563ca4db7f0@oracle.com> References: <7344df42-7fef-036d-e554-d4cd5d898875@oracle.com> <9fa5b29d-6a81-bdf0-7987-b563ca4db7f0@oracle.com> Message-ID: <2c5fb1ab56df1a1f3ece52baf38969557cde1678.camel@oracle.com> Hi Stefan, On Sat, 2018-12-22 at 13:15 +0100, Stefan Johansson wrote: > Moving to hotspot-dev to reach a wider range of reviewers. > > > -------- Forwarded Message -------- > Subject: 8215898: Build broken on 32-bit after JDK-8211425 > Date: Sat, 22 Dec 2018 11:52:34 +0100 > From: Stefan Johansson > Organization: Oracle Corporation > To: hotspot-gc-dev at openjdk.java.net < > hotspot-gc-dev at openjdk.java.net> > > > > Hi all, > > Please review this small fix for a build failure: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215898 > Webrev: http://cr.openjdk.java.net/~sjohanss/8215898/00/ > > Summary > The build failure is caused by always setting UseCompressedOops and > UseCompressedClassPointers to false when AllocateOldGenAt is used. > This > needs to be guarded with LP64_ONLY. > > Testing > Built the previously failing platforms and they now pass. looks good. Thomas From sangheon.kim at oracle.com Sat Dec 22 16:19:18 2018 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Sat, 22 Dec 2018 08:19:18 -0800 Subject: [12] RFR: 8215898: Build broken on 32-bit after JDK-8211425 In-Reply-To: <2c5fb1ab56df1a1f3ece52baf38969557cde1678.camel@oracle.com> References: <7344df42-7fef-036d-e554-d4cd5d898875@oracle.com> <9fa5b29d-6a81-bdf0-7987-b563ca4db7f0@oracle.com> <2c5fb1ab56df1a1f3ece52baf38969557cde1678.camel@oracle.com> Message-ID: Hi Stefan, > On Dec 22, 2018, at 4:54 AM, Thomas Schatzl wrote: > > Hi Stefan, > >> On Sat, 2018-12-22 at 13:15 +0100, Stefan Johansson wrote: >> Moving to hotspot-dev to reach a wider range of reviewers. >> >> >> -------- Forwarded Message -------- >> Subject: 8215898: Build broken on 32-bit after JDK-8211425 >> Date: Sat, 22 Dec 2018 11:52:34 +0100 >> From: Stefan Johansson >> Organization: Oracle Corporation >> To: hotspot-gc-dev at openjdk.java.net < >> hotspot-gc-dev at openjdk.java.net> >> >> >> >> Hi all, >> >> Please review this small fix for a build failure: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8215898 >> Webrev: http://cr.openjdk.java.net/~sjohanss/8215898/00/ >> >> Summary >> The build failure is caused by always setting UseCompressedOops and >> UseCompressedClassPointers to false when AllocateOldGenAt is used. >> This >> needs to be guarded with LP64_ONLY. >> >> Testing >> Built the previously failing platforms and they now pass. Looks good. Thanks for fixing this. Thanks, Sangheon > > looks good. > > Thomas > > From kim.barrett at oracle.com Thu Dec 27 00:18:26 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 26 Dec 2018 19:18:26 -0500 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> Message-ID: <3A419B8A-EB91-4C10-BC6B-86EB95807794@oracle.com> > On Dec 19, 2018, at 4:21 PM, David Holmes wrote: > > Following on from the preliminary RFR: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html > > I've merged with Kim's changes from 8215097. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 > Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ Looks good. A few minor things, for which I don't need a new webrev (unless you decide to make the normalization of JavaTestThreads part of this change; see comment for that class below). ------------------------------------------------------------------------------ src/hotspot/share/runtime/thread.hpp 857 // [...] There is a special case during VM startup 858 // when the BarrierSet has not yet been created, where we add to the 859 // list during the constructor. [...] That "special case" no longer exists. ------------------------------------------------------------------------------ test/hotspot/gtest/threadHelper.inline.hpp 51 VMThreadBlocker() { } Nit: addition of whitespace for the body is inconsistent with the immediately following destructor definition, as well as elsewhere in this file. ------------------------------------------------------------------------------ test/hotspot/gtest/threadHelper.inline.hpp 98 return (char*) "JavaTestThread"; Casting away const is unnecessary since the return type is "const char*". (And casting away const of a string literal isn't such a great idea anyway.) ------------------------------------------------------------------------------ test/hotspot/gtest/threadHelper.inline.hpp These JavaTestThreads are weird, and it seems like it would be better if they were more normal. But maybe that can be done in a later cleanup. The new, improved, life-cycle management might make that easier. ??????????????????????????????????????? I also like Dan's suggestions. From david.holmes at oracle.com Thu Dec 27 04:41:26 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Dec 2018 14:41:26 +1000 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <3A419B8A-EB91-4C10-BC6B-86EB95807794@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> <3A419B8A-EB91-4C10-BC6B-86EB95807794@oracle.com> Message-ID: <1cc49a10-110e-1831-ffd3-1c595fe811b8@oracle.com> Hi Kim, On 27/12/2018 10:18 am, Kim Barrett wrote: >> On Dec 19, 2018, at 4:21 PM, David Holmes wrote: >> >> Following on from the preliminary RFR: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html >> >> I've merged with Kim's changes from 8215097. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ > > Looks good. Thanks for looking at this again! > A few minor things, for which I don't need a new webrev (unless you > decide to make the normalization of JavaTestThreads part of this > change; see comment for that class below). > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/thread.hpp > 857 // [...] There is a special case during VM startup > 858 // when the BarrierSet has not yet been created, where we add to the > 859 // list during the constructor. [...] > > That "special case" no longer exists. There's still the special case of the initial thread. I'll double check the context and update as needed. > ------------------------------------------------------------------------------ > test/hotspot/gtest/threadHelper.inline.hpp > 51 VMThreadBlocker() { } > > Nit: addition of whitespace for the body is inconsistent with the > immediately following destructor definition, as well as elsewhere in > this file. Thanks. Leftover from changes where I was trying to set the thread name (and failing). Will fix. > ------------------------------------------------------------------------------ > test/hotspot/gtest/threadHelper.inline.hpp > 98 return (char*) "JavaTestThread"; > > Casting away const is unnecessary since the return type is "const > char*". (And casting away const of a string literal isn't such a > great idea anyway.) Okay. This was a direct copy of Thread::name, so for consistency I'll fix that too. > ------------------------------------------------------------------------------ > test/hotspot/gtest/threadHelper.inline.hpp > > These JavaTestThreads are weird, and it seems like it would be better > if they were more normal. But maybe that can be done in a later > cleanup. The new, improved, life-cycle management might make that > easier. I don't think they really can be "normal" JavaThreads as that requires they get a java.lang.Thread object and all the associated logic that goes along with that. I know little of the gtest mechanism and have no idea how much of the VM is initialized when these gtests run. Anyway this is out of scope for this change. > ??????????????????????????????????????? > > I also like Dan's suggestions. Yep will implement them. Thanks, David > From kim.barrett at oracle.com Thu Dec 27 09:34:27 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 27 Dec 2018 04:34:27 -0500 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <1cc49a10-110e-1831-ffd3-1c595fe811b8@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> <3A419B8A-EB91-4C10-BC6B-86EB95807794@oracle.com> <1cc49a10-110e-1831-ffd3-1c595fe811b8@oracle.com> Message-ID: <1971D982-312D-4E97-AECB-EFBA1CBE1930@oracle.com> > On Dec 26, 2018, at 11:41 PM, David Holmes wrote: > > Hi Kim, > > On 27/12/2018 10:18 am, Kim Barrett wrote: >>> On Dec 19, 2018, at 4:21 PM, David Holmes wrote: >>> >>> Following on from the preliminary RFR: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html >>> >>> I've merged with Kim's changes from 8215097. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >>> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ >> Looks good. > >> test/hotspot/gtest/threadHelper.inline.hpp >> These JavaTestThreads are weird, and it seems like it would be better >> if they were more normal. But maybe that can be done in a later >> cleanup. The new, improved, life-cycle management might make that >> easier. > > I don't think they really can be "normal" JavaThreads as that requires they get a java.lang.Thread object and all the associated logic that goes along with that. I know little of the gtest mechanism and have no idea how much of the VM is initialized when these gtests run. Anyway this is out of scope for this change. The gtests that use these threads are all TEST_VM, e.g. they require a fully initialized VM before the test can be run. I?m not sure why they were kind of wedged in the way they are, and looking at what needs to be done to make that work (and perhaps future work too), I think making them normal would be a good thing, or perhaps even a necessary thing eventually. But agree it?s out of scope for the change under review. File a followup RFE? From david.holmes at oracle.com Fri Dec 28 01:20:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Dec 2018 11:20:01 +1000 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <1cc49a10-110e-1831-ffd3-1c595fe811b8@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> <3A419B8A-EB91-4C10-BC6B-86EB95807794@oracle.com> <1cc49a10-110e-1831-ffd3-1c595fe811b8@oracle.com> Message-ID: <13706791-2590-107f-c943-0556c5256d0f@oracle.com> Hi Kim, A couple of follow ups below. Changes applied at: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v3/ I'll be pushing this once basic testing is complete. (Don't want to have to update all the copyright notices to 2019 :) ) On 27/12/2018 2:41 pm, David Holmes wrote: > Hi Kim, > > On 27/12/2018 10:18 am, Kim Barrett wrote: >>> On Dec 19, 2018, at 4:21 PM, David Holmes >>> wrote: >>> >>> Following on from the preliminary RFR: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-December/035737.html >>> >>> >>> I've merged with Kim's changes from 8215097. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214097 >>> Webrev: http://cr.openjdk.java.net/~dholmes/8214097/webrev.v2/ >> >> Looks good. > > Thanks for looking at this again! > >> A few minor things, for which I don't need a new webrev (unless you >> decide to make the normalization of JavaTestThreads part of this >> change; see comment for that class below). >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/runtime/thread.hpp >> ? 857 // [...] There is a special case during VM startup >> ? 858 // when the BarrierSet has not yet been created, where we add to >> the >> ? 859 // list during the constructor. [...] >> >> That "special case" no longer exists. > > There's still the special case of the initial thread. I'll double check > the context and update as needed. I was confusing my "special cases" - this is nothing to do with main thread. So I rewrote the comment to be much smaller and simpler. >> ------------------------------------------------------------------------------ >> >> test/hotspot/gtest/threadHelper.inline.hpp >> ?? 51?? VMThreadBlocker() { } >> >> Nit: addition of whitespace for the body is inconsistent with the >> immediately following destructor definition, as well as elsewhere in >> this file. > > Thanks. Leftover from changes where I was trying to set the thread name > (and failing). Will fix. > >> ------------------------------------------------------------------------------ >> >> test/hotspot/gtest/threadHelper.inline.hpp >> ?? 98???? return (char*) "JavaTestThread"; >> >> Casting away const is unnecessary since the return type is "const >> char*".? (And casting away const of a string literal isn't such a >> great idea anyway.) > > Okay. This was a direct copy of Thread::name, so for consistency I'll > fix that too. Actually the return type is only "char*" not "const char*" (but it is a const function), so I couldn't make this change. Trying to make the function return const char* also had a fan out affect, so I just left as-is. >> ------------------------------------------------------------------------------ >> >> test/hotspot/gtest/threadHelper.inline.hpp >> >> These JavaTestThreads are weird, and it seems like it would be better >> if they were more normal.? But maybe that can be done in a later >> cleanup.? The new, improved, life-cycle management might make that >> easier. > > I don't think they really can be "normal" JavaThreads as that requires > they get a java.lang.Thread object and all the associated logic that > goes along with that. I know little of the gtest mechanism and have no > idea how much of the VM is initialized when these gtests run. Anyway > this is out of scope for this change. This may actually be simpler than I thought if we just do as we do for the "service thread". I filed: https://bugs.openjdk.java.net/browse/JDK-8215948 and assigned to Robbin for consideration. Thanks, David ----- > >> ??????????????????????????????????????? >> >> I also like Dan's suggestions. > > Yep will implement them. > > Thanks, > David > >> From Nick.Gasson at arm.com Fri Dec 28 03:06:57 2018 From: Nick.Gasson at arm.com (Nick Gasson (Arm Technology China)) Date: Fri, 28 Dec 2018 03:06:57 +0000 Subject: RFR: AArch64: jtreg test vmTestbase/nsk/jvmti/PopFrame/popframe005 segfaults Message-ID: <492d6889-d5de-be2e-3acb-67b2b2ca4907@arm.com> Hi, Please review a one-line fix to prevent a JVMTI PopFrame crash on AArch64: Bug: https://bugs.openjdk.java.net/browse/JDK-8215951 Webrev: http://cr.openjdk.java.net/~njian/8215951/webrev.0/ This jtreg test started intermittently failing on AArch64 with a segfault after 8215425: http://hg.openjdk.java.net/jdk/jdk/rev/1e213c37befa (There is a full crash log attached to the bug.) The faulting instruction is the load of the dispatch table pointer from rdispatch (x21) in InterpreterMacroAssembler::dispatch_base. 0x0000ffff8c939ab8: ldrb w8, [x22, #0]! 0x0000ffff8c939abc: add w9, w8, #0x900 0x0000ffff8c939ac0: ldr x9, [x21, w9, uxtw #3] <------ 0x0000ffff8c939ac4: br x9 When deoptimising a frame that has a JVMTI PopFrame event pending, vframeArrayElement::unpack_on_stack will set the return PC of the deoptimised frame to the the interpreter entry point remove_activation_preserving_args_entry. After the PopFrame handling this calls dispatch_next to dispatch the next opcode, but at this point rdispatch may not have been initialised to point at the dispatch table. Fix by calling get_dispatch() to initialise rdispatch at the same time as the rest of the interpreter state is restored in remove_activation_preserving_args_entry. I guess this has always been a potential problem, but adding the prints in the changeset above changed the timing of when a method was compiled and exposed it? Thanks, Nick From kim.barrett at oracle.com Fri Dec 28 05:25:54 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 28 Dec 2018 00:25:54 -0500 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: <13706791-2590-107f-c943-0556c5256d0f@oracle.com> References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> <3A419B8A-EB91-4C10-BC6B-86EB95807794@oracle.com> <1cc49a10-110e-1831-ffd3-1c595fe811b8@oracle.com> <13706791-2590-107f-c943-0556c5256d0f@oracle.com> Message-ID: > On Dec 27, 2018, at 8:20 PM, David Holmes wrote: > > Hi Kim, > > A couple of follow ups below. > > Changes applied at: > > http://cr.openjdk.java.net/~dholmes/8214097/webrev.v3/ > > I'll be pushing this once basic testing is complete. (Don't want to have to update all the copyright notices to 2019 :) ) Good plan! > On 27/12/2018 2:41 pm, David Holmes wrote: >> Hi Kim, >> On 27/12/2018 10:18 am, Kim Barrett wrote: >>> src/hotspot/share/runtime/thread.hpp >>> 857 // [...] There is a special case during VM startup >>> 858 // when the BarrierSet has not yet been created, where we add to the >>> 859 // list during the constructor. [...] >>> >>> That "special case" no longer exists. >> There's still the special case of the initial thread. I'll double check the context and update as needed. > > I was confusing my "special cases" - this is nothing to do with main thread. So I rewrote the comment to be much smaller and simpler. Yes, that?s better. Thanks. >>> test/hotspot/gtest/threadHelper.inline.hpp >>> 98 return (char*) "JavaTestThread"; >>> >>> Casting away const is unnecessary since the return type is "const >>> char*". (And casting away const of a string literal isn't such a >>> great idea anyway.) >> Okay. This was a direct copy of Thread::name, so for consistency I'll fix that too. > > Actually the return type is only "char*" not "const char*" (but it is a const function), so I couldn't make this change. Trying to make the function return const char* also had a fan out affect, so I just left as-is. ??? The signature you just checked in is: 97: const char* get_thread_name_string(char* buf, int buflen) const { >>> test/hotspot/gtest/threadHelper.inline.hpp >>> >>> These JavaTestThreads are weird, and it seems like it would be better >>> if they were more normal. But maybe that can be done in a later >>> cleanup. The new, improved, life-cycle management might make that >>> easier. >> I don't think they really can be "normal" JavaThreads as that requires they get a java.lang.Thread object and all the associated logic that goes along with that. I know little of the gtest mechanism and have no idea how much of the VM is initialized when these gtests run. Anyway this is out of scope for this change. > > This may actually be simpler than I thought if we just do as we do for the "service thread". I filed: > > https://bugs.openjdk.java.net/browse/JDK-8215948 > > and assigned to Robbin for consideration. Thanks. From david.holmes at oracle.com Fri Dec 28 06:36:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Dec 2018 16:36:05 +1000 Subject: RFR: JDK-8214097: Rework thread initialization and teardown logic In-Reply-To: References: <8c479164-fc2d-12b8-cb3d-5d12a4a1c5dc@oracle.com> <3A419B8A-EB91-4C10-BC6B-86EB95807794@oracle.com> <1cc49a10-110e-1831-ffd3-1c595fe811b8@oracle.com> <13706791-2590-107f-c943-0556c5256d0f@oracle.com> Message-ID: On 28/12/2018 3:25 pm, Kim Barrett wrote: >> On Dec 27, 2018, at 8:20 PM, David Holmes wrote: >> >> Hi Kim, >> >> A couple of follow ups below. >> >> Changes applied at: >> >> http://cr.openjdk.java.net/~dholmes/8214097/webrev.v3/ >> >> I'll be pushing this once basic testing is complete. (Don't want to have to update all the copyright notices to 2019 :) ) > > Good plan! > >> On 27/12/2018 2:41 pm, David Holmes wrote: >>> Hi Kim, >>> On 27/12/2018 10:18 am, Kim Barrett wrote: >>>> src/hotspot/share/runtime/thread.hpp >>>> 857 // [...] There is a special case during VM startup >>>> 858 // when the BarrierSet has not yet been created, where we add to the >>>> 859 // list during the constructor. [...] >>>> >>>> That "special case" no longer exists. >>> There's still the special case of the initial thread. I'll double check the context and update as needed. >> >> I was confusing my "special cases" - this is nothing to do with main thread. So I rewrote the comment to be much smaller and simpler. > > Yes, that?s better. Thanks. > >>>> test/hotspot/gtest/threadHelper.inline.hpp >>>> 98 return (char*) "JavaTestThread"; >>>> >>>> Casting away const is unnecessary since the return type is "const >>>> char*". (And casting away const of a string literal isn't such a >>>> great idea anyway.) >>> Okay. This was a direct copy of Thread::name, so for consistency I'll fix that too. >> >> Actually the return type is only "char*" not "const char*" (but it is a const function), so I couldn't make this change. Trying to make the function return const char* also had a fan out affect, so I just left as-is. > > ??? The signature you just checked in is: > > 97: const char* get_thread_name_string(char* buf, int buflen) const { Sorry managed to totally confuse myself. I was in a time warp thinking I'd overridden Thread::name(), but that was the approach that doesn't work because for JavaThreads we call get_thread_name() and thus get_thread_name_string(). So I mistakenly tried to remove the cast in Thread::name first, which fails because it doesn't return const char*, and so left it as-is. I'll do a follow up. David ----- >>>> test/hotspot/gtest/threadHelper.inline.hpp >>>> >>>> These JavaTestThreads are weird, and it seems like it would be better >>>> if they were more normal. But maybe that can be done in a later >>>> cleanup. The new, improved, life-cycle management might make that >>>> easier. >>> I don't think they really can be "normal" JavaThreads as that requires they get a java.lang.Thread object and all the associated logic that goes along with that. I know little of the gtest mechanism and have no idea how much of the VM is initialized when these gtests run. Anyway this is out of scope for this change. >> >> This may actually be simpler than I thought if we just do as we do for the "service thread". I filed: >> >> https://bugs.openjdk.java.net/browse/JDK-8215948 >> >> and assigned to Robbin for consideration. > > Thanks. > From neil.stevenson at gmail.com Fri Dec 28 07:34:52 2018 From: neil.stevenson at gmail.com (Neil Stevenson) Date: Fri, 28 Dec 2018 07:34:52 +0000 Subject: SIGSEGV error on 11.0.1+13, libjvm.dylib Message-ID: Hi Hope this is the right way to submit bug notifications, new here A bug has been found on AdoptOpenJDK and Azul Zulu, that the AdoptOpenJDK folks thought should be submitted here. Occurs at least on Darwin Steps to recreate and fuller history is here https://github.com/AdoptOpenJDK/openjdk-build/issues/814. Gives a segmentation fault fairly consistently. # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x0000000104853d23, pid=23222, tid=953467 # # JRE version: OpenJDK Runtime Environment (11.0.1+13) (build 11.0.1+13) # Java VM: OpenJDK 64-Bit Server VM (11.0.1+13, mixed mode, tiered, compressed oops, g1 gc, bsd-amd64) # Problematic frame: # V [libjvm.dylib+0x20dd23] ClassLoaderData::loader_name_and_id() const+0x7 # Neil From david.holmes at oracle.com Fri Dec 28 08:07:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Dec 2018 18:07:35 +1000 Subject: SIGSEGV error on 11.0.1+13, libjvm.dylib In-Reply-To: References: Message-ID: Hi Neil, On 28/12/2018 5:34 pm, Neil Stevenson wrote: > Hi > Hope this is the right way to submit bug notifications, new here If you are an OpenJDK Contributor and have at least Author status then you can file a bug directly at: https://bugs.openjdk.java.net/ For AdoptOpenJDK issues I would have expected you to file the bug as you did per: # If you would like to submit a bug report, please visit: # https://github.com/AdoptOpenJDK/openjdk-build/issues and then I would have expected someone from AdoptOpenJDK to file the upstream bug at bugs.openjdk.java.net! Directing you to the mailing lists is not appropriate as these are for developer discussions not bug reporting. If this reproduces on Oracle JDK then you can file a bug at: https://bugreport.java.com/bugreport/ Thanks, David ----- > A bug has been found on AdoptOpenJDK and Azul Zulu, > that the AdoptOpenJDK folks thought should be submitted here. > Occurs at least on Darwin > > Steps to recreate and fuller history is here > https://github.com/AdoptOpenJDK/openjdk-build/issues/814. Gives a > segmentation fault fairly consistently. > > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGSEGV (0xb) at pc=0x0000000104853d23, pid=23222, tid=953467 > # > # JRE version: OpenJDK Runtime Environment (11.0.1+13) (build 11.0.1+13) > # Java VM: OpenJDK 64-Bit Server VM (11.0.1+13, mixed mode, tiered, > compressed oops, g1 gc, bsd-amd64) > # Problematic frame: > # V [libjvm.dylib+0x20dd23] ClassLoaderData::loader_name_and_id() const+0x7 > # > > > Neil > From matthias.baesken at sap.com Fri Dec 28 13:35:41 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 28 Dec 2018 13:35:41 +0000 Subject: RFR [XS]: 8215961: jdk/jfr/event/os/TestCPUInformation.java fails on AArch64 Message-ID: Hello, please review this small fix . At the moment, the test jdk/jfr/event/os/TestCPUInformation.java fails on AArch64 with the following error : java.lang.RuntimeException: Value not in (Intel, AMD, Unknown x86, SPARC, ARM, PPC, PowerPC, AArch64, zArch), field='description', value='0x50:0x0:0x000:1, simd' Reason is that the jdk.CPUInformation event misses a known CPU identifier value in the description, see the description part of it : Event: jdk.CPUInformation { .... description = "0x50:0x0:0x000:1, simd" sockets = 8 .... } The patch adds the CPU identifier info to the _cpu_desc string where it is taken from . Please compare also with the ppc - implementation where the info (PPC) is already added . vm_version_ext_ppc.cpp 50 snprintf(_cpu_desc, CPU_DETAILED_DESC_BUF_SIZE, "PPC %s", features_string()); Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8215961 http://cr.openjdk.java.net/~mbaesken/webrevs/8215961.0/ Thanks, Matthias From neil.stevenson at gmail.com Fri Dec 28 13:55:00 2018 From: neil.stevenson at gmail.com (Neil Stevenson) Date: Fri, 28 Dec 2018 13:55:00 +0000 Subject: SIGSEGV error on 11.0.1+13, libjvm.dylib In-Reply-To: References: Message-ID: Thanks and apologies I'll see if I can find someone with a copy of Oracle JDk to try with then progress Neil On Fri, 28 Dec 2018 at 08:07, David Holmes wrote: > Hi Neil, > > On 28/12/2018 5:34 pm, Neil Stevenson wrote: > > Hi > > Hope this is the right way to submit bug notifications, new here > > If you are an OpenJDK Contributor and have at least Author status then > you can file a bug directly at: > > https://bugs.openjdk.java.net/ > > For AdoptOpenJDK issues I would have expected you to file the bug as you > did per: > > # If you would like to submit a bug report, please visit: > # https://github.com/AdoptOpenJDK/openjdk-build/issues > > and then I would have expected someone from AdoptOpenJDK to file the > upstream bug at bugs.openjdk.java.net! Directing you to the mailing > lists is not appropriate as these are for developer discussions not bug > reporting. > > If this reproduces on Oracle JDK then you can file a bug at: > > https://bugreport.java.com/bugreport/ > > Thanks, > David > ----- > > > > A bug has been found on AdoptOpenJDK and Azul Zulu, > > that the AdoptOpenJDK folks thought should be submitted here. > > Occurs at least on Darwin > > > > Steps to recreate and fuller history is here > > https://github.com/AdoptOpenJDK/openjdk-build/issues/814. Gives a > > segmentation fault fairly consistently. > > > > # A fatal error has been detected by the Java Runtime Environment: > > # > > # SIGSEGV (0xb) at pc=0x0000000104853d23, pid=23222, tid=953467 > > # > > # JRE version: OpenJDK Runtime Environment (11.0.1+13) (build 11.0.1+13) > > # Java VM: OpenJDK 64-Bit Server VM (11.0.1+13, mixed mode, tiered, > > compressed oops, g1 gc, bsd-amd64) > > # Problematic frame: > > # V [libjvm.dylib+0x20dd23] ClassLoaderData::loader_name_and_id() > const+0x7 > > # > > > > > > Neil > > > From david.holmes at oracle.com Fri Dec 28 21:29:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 29 Dec 2018 07:29:12 +1000 Subject: RFR [XS]: 8215961: jdk/jfr/event/os/TestCPUInformation.java fails on AArch64 In-Reply-To: References: Message-ID: <24077e4c-7106-82d4-b67e-2653cc037198@oracle.com> Hi Matthias, On 28/12/2018 11:35 pm, Baesken, Matthias wrote: > Hello, please review this small fix . > > At the moment, the test jdk/jfr/event/os/TestCPUInformation.java fails on AArch64 with the following error : > > > java.lang.RuntimeException: Value not in (Intel, AMD, Unknown x86, SPARC, ARM, PPC, PowerPC, AArch64, zArch), field='description', value='0x50:0x0:0x000:1, simd' > > > Reason is that the jdk.CPUInformation event misses a known CPU identifier value in the description, see the description part of it : > > Event: jdk.CPUInformation { > .... > description = "0x50:0x0:0x000:1, simd" > sockets = 8 > .... > } > > > The patch adds the CPU identifier info to the _cpu_desc string where it is taken from . > Please compare also with the ppc - implementation where the info (PPC) is already added . > > vm_version_ext_ppc.cpp > > 50 snprintf(_cpu_desc, CPU_DETAILED_DESC_BUF_SIZE, "PPC %s", features_string()); > > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8215961 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215961.0/ Seems a reasonable approach. I would expect other ARM systems to also be affected by this. Thanks, David > > Thanks, Matthias > From felix.yang at huawei.com Sat Dec 29 08:45:10 2018 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Sat, 29 Dec 2018 08:45:10 +0000 Subject: AArch64: jtreg test vmTestbase/nsk/jvmti/PopFrame/popframe005 segfaults In-Reply-To: <492d6889-d5de-be2e-3acb-67b2b2ca4907@arm.com> References: <492d6889-d5de-be2e-3acb-67b2b2ca4907@arm.com> Message-ID: Hi, The change looks good to me (NOT a Reviewer). I think the aarch64 8u branches have the same issue. Thanks, Felix > > Hi, > > Please review a one-line fix to prevent a JVMTI PopFrame crash on AArch64: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215951 > Webrev: http://cr.openjdk.java.net/~njian/8215951/webrev.0/ > > This jtreg test started intermittently failing on AArch64 with a > segfault after 8215425: > http://hg.openjdk.java.net/jdk/jdk/rev/1e213c37befa > (There is a full crash log attached to the bug.) > > The faulting instruction is the load of the dispatch table pointer from > rdispatch (x21) in InterpreterMacroAssembler::dispatch_base. > > 0x0000ffff8c939ab8: ldrb w8, [x22, #0]! > 0x0000ffff8c939abc: add w9, w8, #0x900 > 0x0000ffff8c939ac0: ldr x9, [x21, w9, uxtw #3] <------ > 0x0000ffff8c939ac4: br x9 > > When deoptimising a frame that has a JVMTI PopFrame event pending, > vframeArrayElement::unpack_on_stack will set the return PC of the > deoptimised frame to the the interpreter entry point > remove_activation_preserving_args_entry. After the PopFrame handling > this calls dispatch_next to dispatch the next opcode, but at this point > rdispatch may not have been initialised to point at the dispatch table. > Fix by calling get_dispatch() to initialise rdispatch at the same time > as the rest of the interpreter state is restored in > remove_activation_preserving_args_entry. > > I guess this has always been a potential problem, but adding the prints > in the changeset above changed the timing of when a method was compiled > and exposed it? > > Thanks, > Nick From glaubitz at physik.fu-berlin.de Sat Dec 29 12:41:14 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Sat, 29 Dec 2018 13:41:14 +0100 Subject: Zero JVM segfaulting on linux-sparc In-Reply-To: References: <18c4b28e-9e94-9ee0-d2ac-cb8308607630@physik.fu-berlin.de> <401d0030-7681-1cf5-c557-024daf6d5e20@oracle.com> <48cbf65f09472c2e233a8ee64105467f14679a2d.camel@redhat.com> <82335ba5d00fe8f73fde83fa0bb67d4c1d5270f6.camel@redhat.com> Message-ID: Hi Severin! On 12/21/18 2:34 PM, John Paul Adrian Glaubitz wrote: > On 12/21/18 1:52 PM, Severin Gehwolf wrote: >> OK. It's rather curious. What are the different flags being used? >> Pre/post patch? It looks as if this happens (on Linux sparc): >> >> Zero and '-ffp-contract=off -O2' => bad build >> Zero and '-ffp-contract=off -O0' => good build >> Server and '-ffp-contract=off -O2' => good build > > Logs here: https://people.debian.org/~glaubitz/openjdk-sparc/ Any suggestions how to move forward with this? Should I open an RFR with your suggested patch which works fine for me? Or do you want to do that? I have filed a bug report in Jira now [1]. Shall I file a bug in gcc upstream as well? Adrian > [1] https://bugs.openjdk.java.net/browse/JDK-8215969 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From merkel05 at gmail.com Sun Dec 30 18:00:03 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Sun, 30 Dec 2018 19:00:03 +0100 Subject: [PATCH] hsdis installation documentation update Message-ID: Hi, I was looking through hsdis documentation to build and install it on the recent versions of JDK. Seems like there are few things in the installation section, that might be improved. Could anyone please review the patch below? diff --git a/src/utils/hsdis/README b/src/utils/hsdis/README --- a/src/utils/hsdis/README +++ b/src/utils/hsdis/README @@ -114,18 +114,16 @@ * Installing -Products are named like build/$OS-$LIBARCH/hsdis-$LIBARCH.so. You can -install them on your LD_LIBRARY_PATH, or inside of your JRE/JDK. The -search path in the JVM is: - -1. /jre/lib///libhsdis-.so -2. /jre/lib///hsdis-.so -3. /jre/lib//hsdis-.so +Products are named like build/$OS-$LIBARCH/hsdis-$LIBARCH.so. You can +install them next to your libjvm.so inside JRE/JDK or alternatively +put it anywhere on your LD_LIBRARY_PATH. JVM looks up several paths +derived from libjvm.so in the following order: + +1. /lib/server/libhsdis-.so +2. /lib/server/hsdis-.so +3. /lib/hsdis-.so 4. hsdis-.so (using LD_LIBRARY_PATH) -Note that there's a bug in hotspot versions prior to hs22 that causes -steps 2 and 3 to fail when used with JDK7. - Now test: export LD_LIBRARY_PATH .../hsdis/build/$OS-$LIBARCH:$LD_LIBRARY_PATH Thanks, Sergei From david.holmes at oracle.com Sun Dec 30 21:17:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 31 Dec 2018 07:17:56 +1000 Subject: [PATCH] hsdis installation documentation update In-Reply-To: References: Message-ID: <79927d73-9592-4510-0157-93dd01f95536@oracle.com> On 31/12/2018 4:00 am, Sergei Ustimenko wrote: > Hi, > > I was looking through hsdis documentation to build and install > it on the recent versions of JDK. Seems like there are few things > in the installation section, that might be improved. > > Could anyone please review the patch below? > > > > diff --git a/src/utils/hsdis/README b/src/utils/hsdis/README > --- a/src/utils/hsdis/README > +++ b/src/utils/hsdis/README > @@ -114,18 +114,16 @@ > > * Installing > > -Products are named like build/$OS-$LIBARCH/hsdis-$LIBARCH.so. You can > -install them on your LD_LIBRARY_PATH, or inside of your JRE/JDK. The > -search path in the JVM is: > - > -1. /jre/lib///libhsdis-.so > -2. /jre/lib///hsdis-.so > -3. /jre/lib//hsdis-.so > +Products are named like build/$OS-$LIBARCH/hsdis-$LIBARCH.so. You can > +install them next to your libjvm.so inside JRE/JDK or alternatively > +put it anywhere on your LD_LIBRARY_PATH. JVM looks up several paths > +derived from libjvm.so in the following order: > + > +1. /lib/server/libhsdis-.so > +2. /lib/server/hsdis-.so "server" should still say . While most platforms only have sevrer VM these days some do still have client and there is also minimal VM. David ----- > +3. /lib/hsdis-.so > 4. hsdis-.so (using LD_LIBRARY_PATH) > > -Note that there's a bug in hotspot versions prior to hs22 that causes > -steps 2 and 3 to fail when used with JDK7. > - > Now test: > > export LD_LIBRARY_PATH .../hsdis/build/$OS-$LIBARCH:$LD_LIBRARY_PATH > > > > Thanks, > Sergei > From merkel05 at gmail.com Sun Dec 30 22:42:04 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Sun, 30 Dec 2018 23:42:04 +0100 Subject: [PATCH] hsdis installation documentation update In-Reply-To: <79927d73-9592-4510-0157-93dd01f95536@oracle.com> References: <79927d73-9592-4510-0157-93dd01f95536@oracle.com> Message-ID: Hi David, > "server" should still say . While most platforms only have sevrer VM > these days some do still have client and there is also minimal VM Ah, right. I though there are no client VMs out there already. I've updated the patch, hope it is good now. diff --git a/src/utils/hsdis/README b/src/utils/hsdis/README --- a/src/utils/hsdis/README +++ b/src/utils/hsdis/README @@ -114,18 +114,16 @@ * Installing -Products are named like build/$OS-$LIBARCH/hsdis-$LIBARCH.so. You can -install them on your LD_LIBRARY_PATH, or inside of your JRE/JDK. The -search path in the JVM is: - -1. /jre/lib///libhsdis-.so -2. /jre/lib///hsdis-.so -3. /jre/lib//hsdis-.so +Products are named like build/$OS-$LIBARCH/hsdis-$LIBARCH.so. You can +install them next to your libjvm.so inside JRE/JDK or alternatively +put it anywhere on your LD_LIBRARY_PATH. JVM looks up several paths +derived from libjvm.so in the following order: + +1. /lib//libhsdis-.so +2. /lib//hsdis-.so +3. /lib/hsdis-.so 4. hsdis-.so (using LD_LIBRARY_PATH) -Note that there's a bug in hotspot versions prior to hs22 that causes -steps 2 and 3 to fail when used with JDK7. - Now test: export LD_LIBRARY_PATH .../hsdis/build/$OS-$LIBARCH:$LD_LIBRARY_PATH Thanks, Sergei From aph at redhat.com Mon Dec 31 12:39:43 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 31 Dec 2018 12:39:43 +0000 Subject: RFR: AArch64: jtreg test vmTestbase/nsk/jvmti/PopFrame/popframe005 segfaults In-Reply-To: <492d6889-d5de-be2e-3acb-67b2b2ca4907@arm.com> References: <492d6889-d5de-be2e-3acb-67b2b2ca4907@arm.com> Message-ID: Hi, On 12/28/18 3:06 AM, Nick Gasson (Arm Technology China) wrote: > Please review a one-line fix to prevent a JVMTI PopFrame crash on AArch64: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215951 > Webrev: http://cr.openjdk.java.net/~njian/8215951/webrev.0/ > > This jtreg test started intermittently failing on AArch64 with a > segfault after 8215425: > http://hg.openjdk.java.net/jdk/jdk/rev/1e213c37befa > (There is a full crash log attached to the bug.) > > The faulting instruction is the load of the dispatch table pointer from > rdispatch (x21) in InterpreterMacroAssembler::dispatch_base. > > 0x0000ffff8c939ab8: ldrb w8, [x22, #0]! > 0x0000ffff8c939abc: add w9, w8, #0x900 > 0x0000ffff8c939ac0: ldr x9, [x21, w9, uxtw #3] <------ > 0x0000ffff8c939ac4: br x9 > > When deoptimising a frame that has a JVMTI PopFrame event pending, > vframeArrayElement::unpack_on_stack will set the return PC of the > deoptimised frame to the the interpreter entry point > remove_activation_preserving_args_entry. After the PopFrame handling > this calls dispatch_next to dispatch the next opcode, but at this point > rdispatch may not have been initialised to point at the dispatch table. > Fix by calling get_dispatch() to initialise rdispatch at the same time > as the rest of the interpreter state is restored in > remove_activation_preserving_args_entry. > > I guess this has always been a potential problem, but adding the prints > in the changeset above changed the timing of when a method was compiled > and exposed it? That's hard to say without a lot of analysis. In any case, your patch looks fine. We'll want to add it to the list of jdk8 backports. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671