From david.holmes at oracle.com Wed Feb 1 00:14:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 1 Feb 2017 10:14:14 +1000 Subject: JDK 10 RFR: 8173421: Obsolete and expired flags for JDK 10 need to be removed and related tests updated In-Reply-To: <4025f109-3f01-2a73-55f0-a12c54092708@oracle.com> References: <588B4F43.4040907@oracle.com> <29a0c6ad-ac7a-ac6f-61b9-76b63fabd94d@oracle.com> <3253D1E0-7E1B-4C5C-967B-49B0DDB179F7@oracle.com> <54ec44f0-4787-c97b-d544-1ad622b7fb56@oracle.com> <599f24e6-193e-a25e-dbd8-09ce2c167d25@oracle.com> <4025f109-3f01-2a73-55f0-a12c54092708@oracle.com> Message-ID: Thanks for the review Dan! MinSleepInterval should have been deprecated in 9 but we missed it. I don't think it is worth trying to do that now given where we are in the release cycle. David On 1/02/2017 8:14 AM, Daniel D. Daugherty wrote: > On 1/30/17 5:47 PM, David Holmes wrote: >> Just realized I still need a JDK 10 Reviewer to look at this. >> >> http://cr.openjdk.java.net/~dholmes/8173421/webrev.hotspot.v2/ > > src/cpu/aarch64/vm/c1_globals_aarch64.hpp > No comments. > > src/cpu/aarch64/vm/c2_globals_aarch64.hpp > No comments. > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java > No comments. > > src/os/solaris/vm/os_solaris.cpp > No comments. > > src/share/vm/gc/cms/concurrentMarkSweepGeneration.cpp > No comments. > > src/share/vm/oops/method.cpp > No comments. > > src/share/vm/prims/jvm.cpp > The JVM_Sleep() changes had me confused for second, > but I think they're okay... > > No other comments. > > src/share/vm/runtime/arguments.cpp > MinSleepInterval - are you going to deprecate this in JDK9? > > No other comments. > > src/share/vm/runtime/arguments.hpp > No comments. > > src/share/vm/runtime/globals.hpp > No comments. > > test/TEST.groups > No comments. > > test/gc/arguments/TestSelectDefaultGC.java > No comments. > > test/runtime/CommandLine/ObsoleteFlagErrorMessage.java > No comments. > > test/runtime/CommandLine/VMAliasOptions.java > No comments. > > test/runtime/CommandLine/VMDeprecatedOptions.java > No comments. > > test/gc/arguments/TestExplicitGCInvokesConcurrentAndUnloadsClasses.java > test/gc/startup_warnings/TestDefNewCMS.java > test/gc/startup_warnings/TestParNewCMS.java > test/gc/startup_warnings/TestParNewSerialOld.java > test/gc/startup_warnings/TestUseAutoGCSelectPolicy.java > test/runtime/NMT/AutoshutdownNMT.java > No comments on the deleted files. > > Thumbs up! > > Dan > > >> >> Thanks, >> David >> >> On 31/01/2017 10:42 AM, David Holmes wrote: >>> Forgot to add that I'm no longer deleting the test: >>> >>> test/runtime/CommandLine/PermGenFlagsTest.java >>> >>> David >>> >>> On 31/01/2017 10:35 AM, David Holmes wrote: >>>> Thanks for reviewing Mikael! >>>> >>>> On 31/01/2017 10:12 AM, Mikael Vidstedt wrote: >>>>> >>>>> Looks like the UseFastAccessorMethods and UseFastEmptyMethods are >>>>> still in active use on ZERO, so they should not be removed. Apart from >>>>> that it looks good. >>>> >>>> Good catch! Yes I have reverted the Zero changes, other than deleting >>>> the flags from the table. So the following files no longer appear in >>>> the >>>> webrev: >>>> >>>> src/cpu/zero/vm/cppInterpreterGenerator_zero.cpp >>>> src/cpu/zero/vm/globals_zero.hpp >>>> src/share/vm/prims/jvmtiManageCapabilities.cpp >>>> >>>> I've also changed the PermSize and MaxPermSize so that they have an >>>> unspecified expiration version. >>>> >>>> Webrev updated at: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8173421/webrev.hotspot.v2/ >>>> >>>> but I intend to push the changes as described >>>> >>>>> And thanks for taking over the work! >>>> >>>> Didn't know what I'd let myself in for :) >>>> >>>> Thanks, >>>> David >>>> >>>>> Cheers, >>>>> Mikael >>>>> >>>>>> On Jan 30, 2017, at 12:37 PM, David Holmes >>>>>> wrote: >>>>>> >>>>>> Thanks for the Review Lois. For the public record there's a >>>>>> conversation happening around what to do with the PermSize flags. >>>>>> >>>>>> Can I please get additional Reviews - this affects code in all teams. >>>>>> >>>>>> We need to get the JDK 10 forest buildable as JDK 10 ASAP. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> On 27/01/2017 11:46 PM, Lois Foltan wrote: >>>>>>> >>>>>>> On 1/27/2017 12:13 AM, David Holmes wrote: >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8173421 >>>>>>>> >>>>>>>> When we try to bump the JDK version to 10 we trigger all the >>>>>>>> warnings >>>>>>>> (and an assert) surrounding the VM deprecated/obsolete/expired flag >>>>>>>> handling. In short anything that expires in 10 must now be removed; >>>>>>>> anything now obsolete in 10 must go in a different table. Related >>>>>>>> changes required to source code and tests. Gory details below. >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8173421/webrev/ >>>>>>>> >>>>>>>> Testing: JPRT (in progress) >>>>>>>> Manual runs of affected regression tests >>>>>>>> >>>>>>>> This will be pushed to jdk10/jdk10 along with the actual change to >>>>>>>> bump the major version number. >>>>>>>> >>>>>>>> There are some Zero and Aarch64 code tweaks that should have been >>>>>>>> handled in JDK 9 but got overlooked somehow. Nothing significant. >>>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> This looks good. However, please do not remove PermSize and >>>>>>> MaxPermSize >>>>>>> in the special flag table in src/share/vm/runtime/arguments.cpp. We >>>>>>> recently added these options back in JDK 9 for ease of migration, >>>>>>> see >>>>>>> JDK-8167446. I would rather see these two options put back in the >>>>>>> table >>>>>>> and have their expiration bumped to 11 until we decide going forward >>>>>>> what is the correct course of action. Feel free to enter a new >>>>>>> JDK 10 >>>>>>> specific bug to address the issue of these two options specifically. >>>>>>> >>>>>>> Thanks, >>>>>>> Lois >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> --- >>>>>>>> >>>>>>>> Non-aliased flags: >>>>>>>> >>>>>>>> AutoGCSelectPauseMillis (def: 5000) has now expired and is removed >>>>>>>> completely from the source code. >>>>>>>> >>>>>>>> UseAutoGCSelectPolicy (def: false) has now expired and is removed >>>>>>>> completely from the source code as-if replaced by false. >>>>>>>> >>>>>>>> UseParNewGC (def: false) expired. However it is >>>>>>>> expected/required to >>>>>>>> be true whenever UseConcMarkSweepGC is true, and UseConcMarkSweepGC >>>>>>>> has not been deprecated. So we must remove all uses as-if it were >>>>>>>> true, not false. >>>>>>>> >>>>>>>> ExplicitGCInvokesConcurrentAndUnloadsClasses (def: false) has now >>>>>>>> expired and is removed completely from the source as-if replaced by >>>>>>>> false. >>>>>>>> >>>>>>>> ConvertSleepToYield (def: true) is now obsolete. It is moved to the >>>>>>>> obsolete table and is removed completely from the source code as-if >>>>>>>> replaced by true. >>>>>>>> >>>>>>>> ConvertYieldToSleep (def: false) is now obsolete. It is moved to >>>>>>>> the >>>>>>>> obsolete table and is removed completely from the source code as-if >>>>>>>> replaced by false. >>>>>>>> >>>>>>>> As a result of the above MinSleepInterval is no longer used (it >>>>>>>> should >>>>>>>> have also been deprecated in 9) and is also marked as obsolete. >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> Aliased options that have expired: >>>>>>>> >>>>>>>> CMSMarkStackSizeMax is an alias and has expired. It is removed from >>>>>>>> the alias and obsolete flag tables. >>>>>>>> >>>>>>>> CMSMarkStackSize is an alias and has expired. It is removed from >>>>>>>> the >>>>>>>> alias and obsolete flag tables. >>>>>>>> >>>>>>>> G1MarkStackSize is an alias and has expired. It is removed from the >>>>>>>> alias and obsolete flag tables. >>>>>>>> >>>>>>>> ParallelMarkingThreads is an alias and has expired. It is removed >>>>>>>> from >>>>>>>> the alias and obsolete flag tables. >>>>>>>> >>>>>>>> ParallelCMSThreads is an alias and has expired. It is removed from >>>>>>>> the >>>>>>>> alias and obsolete flag tables. >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> The following flags all expire in 10 and exist (except where noted) >>>>>>>> only in the special-flag table (from which they are now >>>>>>>> removed). Any >>>>>>>> other uses of the flag are removed. >>>>>>>> >>>>>>>> UseOldInlining >>>>>>>> SafepointPollOffset (defined in >>>>>>>> ./cpu/aarch64/vm/c1_globals_aarch64.hpp) >>>>>>>> UseBoundThreads (comments in os_solaris.cpp) >>>>>>>> DefaultThreadPriority >>>>>>>> NoYieldsInMicrolock >>>>>>>> BackEdgeThreshold (defined in c1/2_globals_aarch64.hpp) >>>>>>>> UseNewReflection >>>>>>>> ReflectionWrapResolutionErrors >>>>>>>> VerifyReflectionBytecodes >>>>>>>> AutoShutdownNMT >>>>>>>> NmethodSweepFraction >>>>>>>> NmethodSweepCheckInterval >>>>>>>> CodeCacheMinimumFreeSpace >>>>>>>> UseFastAccessorMethods (ZERO) (still used in >>>>>>>> ccpInterpreterGenerator_zero.cpp, jvmtiManageCapabilities.cpp) >>>>>>>> UseFastEmptyMethods (ZERO) (still used in >>>>>>>> ccpInterpreterGenerator_zero.cpp, jvmtiManageCapabilities.cpp) >>>>>>>> UseCompilerSafepoints >>>>>>>> AdaptiveSizePausePolicy >>>>>>>> ParallelGCRetainPLAB >>>>>>>> ThreadSafetyMargin >>>>>>>> LazyBootClassLoader >>>>>>>> StarvationMonitorInterval >>>>>>>> PreInflateSpin >>>>>>>> JNIDetachReleasesMonitors >>>>>>>> UseAltSigs >>>>>>>> SegmentedHeapDumpThreshold >>>>>>>> PrintOopAddress (comment in method.cpp) >>>>>>>> PermSize >>>>>>>> MaxPermSize >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> Tests modified: >>>>>>>> >>>>>>>> runtime/CommandLine/ >>>>>>>> ObsoleteFlagErrorMessage.java >>>>>>>> VMDeprecatedOptions.java >>>>>>>> VMAliasOptions.java >>>>>>>> >>>>>>>> gc/arguments/TestSelectDefaultGC.java >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> Tests removed (TEST.groups updated as needed): >>>>>>>> >>>>>>>> gc/startup_warnings/ >>>>>>>> TestUseAutoGCSelectPolicy.java >>>>>>>> TestParNewSerialOld.java >>>>>>>> TestParNewCMS.java >>>>>>>> TestDefNewCMS.java >>>>>>>> gc/arguments/TestExplicitGCInvokesConcurrentAndUnloadsClasses.java >>>>>>>> >>>>>>>> runtime/CommandLine/PermGenFlagsTest.java >>>>>>>> runtime/NMT/AutoshutdownNMT.java >>>>>>> >>>>> >> > From daniel.daugherty at oracle.com Wed Feb 1 00:15:12 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 31 Jan 2017 17:15:12 -0700 Subject: JDK 10 RFR: 8173421: Obsolete and expired flags for JDK 10 need to be removed and related tests updated In-Reply-To: References: <588B4F43.4040907@oracle.com> <29a0c6ad-ac7a-ac6f-61b9-76b63fabd94d@oracle.com> <3253D1E0-7E1B-4C5C-967B-49B0DDB179F7@oracle.com> <54ec44f0-4787-c97b-d544-1ad622b7fb56@oracle.com> <599f24e6-193e-a25e-dbd8-09ce2c167d25@oracle.com> <4025f109-3f01-2a73-55f0-a12c54092708@oracle.com> Message-ID: Sounds good to me. Dan On 1/31/17 5:14 PM, David Holmes wrote: > Thanks for the review Dan! > > MinSleepInterval should have been deprecated in 9 but we missed it. I > don't think it is worth trying to do that now given where we are in > the release cycle. > > David > > On 1/02/2017 8:14 AM, Daniel D. Daugherty wrote: >> On 1/30/17 5:47 PM, David Holmes wrote: >>> Just realized I still need a JDK 10 Reviewer to look at this. >>> >>> http://cr.openjdk.java.net/~dholmes/8173421/webrev.hotspot.v2/ >> >> src/cpu/aarch64/vm/c1_globals_aarch64.hpp >> No comments. >> >> src/cpu/aarch64/vm/c2_globals_aarch64.hpp >> No comments. >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java >> >> No comments. >> >> src/os/solaris/vm/os_solaris.cpp >> No comments. >> >> src/share/vm/gc/cms/concurrentMarkSweepGeneration.cpp >> No comments. >> >> src/share/vm/oops/method.cpp >> No comments. >> >> src/share/vm/prims/jvm.cpp >> The JVM_Sleep() changes had me confused for second, >> but I think they're okay... >> >> No other comments. >> >> src/share/vm/runtime/arguments.cpp >> MinSleepInterval - are you going to deprecate this in JDK9? >> >> No other comments. >> >> src/share/vm/runtime/arguments.hpp >> No comments. >> >> src/share/vm/runtime/globals.hpp >> No comments. >> >> test/TEST.groups >> No comments. >> >> test/gc/arguments/TestSelectDefaultGC.java >> No comments. >> >> test/runtime/CommandLine/ObsoleteFlagErrorMessage.java >> No comments. >> >> test/runtime/CommandLine/VMAliasOptions.java >> No comments. >> >> test/runtime/CommandLine/VMDeprecatedOptions.java >> No comments. >> >> test/gc/arguments/TestExplicitGCInvokesConcurrentAndUnloadsClasses.java >> test/gc/startup_warnings/TestDefNewCMS.java >> test/gc/startup_warnings/TestParNewCMS.java >> test/gc/startup_warnings/TestParNewSerialOld.java >> test/gc/startup_warnings/TestUseAutoGCSelectPolicy.java >> test/runtime/NMT/AutoshutdownNMT.java >> No comments on the deleted files. >> >> Thumbs up! >> >> Dan >> >> >>> >>> Thanks, >>> David >>> >>> On 31/01/2017 10:42 AM, David Holmes wrote: >>>> Forgot to add that I'm no longer deleting the test: >>>> >>>> test/runtime/CommandLine/PermGenFlagsTest.java >>>> >>>> David >>>> >>>> On 31/01/2017 10:35 AM, David Holmes wrote: >>>>> Thanks for reviewing Mikael! >>>>> >>>>> On 31/01/2017 10:12 AM, Mikael Vidstedt wrote: >>>>>> >>>>>> Looks like the UseFastAccessorMethods and UseFastEmptyMethods are >>>>>> still in active use on ZERO, so they should not be removed. Apart >>>>>> from >>>>>> that it looks good. >>>>> >>>>> Good catch! Yes I have reverted the Zero changes, other than deleting >>>>> the flags from the table. So the following files no longer appear in >>>>> the >>>>> webrev: >>>>> >>>>> src/cpu/zero/vm/cppInterpreterGenerator_zero.cpp >>>>> src/cpu/zero/vm/globals_zero.hpp >>>>> src/share/vm/prims/jvmtiManageCapabilities.cpp >>>>> >>>>> I've also changed the PermSize and MaxPermSize so that they have an >>>>> unspecified expiration version. >>>>> >>>>> Webrev updated at: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8173421/webrev.hotspot.v2/ >>>>> >>>>> but I intend to push the changes as described >>>>> >>>>>> And thanks for taking over the work! >>>>> >>>>> Didn't know what I'd let myself in for :) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Cheers, >>>>>> Mikael >>>>>> >>>>>>> On Jan 30, 2017, at 12:37 PM, David Holmes >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Thanks for the Review Lois. For the public record there's a >>>>>>> conversation happening around what to do with the PermSize flags. >>>>>>> >>>>>>> Can I please get additional Reviews - this affects code in all >>>>>>> teams. >>>>>>> >>>>>>> We need to get the JDK 10 forest buildable as JDK 10 ASAP. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> On 27/01/2017 11:46 PM, Lois Foltan wrote: >>>>>>>> >>>>>>>> On 1/27/2017 12:13 AM, David Holmes wrote: >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8173421 >>>>>>>>> >>>>>>>>> When we try to bump the JDK version to 10 we trigger all the >>>>>>>>> warnings >>>>>>>>> (and an assert) surrounding the VM deprecated/obsolete/expired >>>>>>>>> flag >>>>>>>>> handling. In short anything that expires in 10 must now be >>>>>>>>> removed; >>>>>>>>> anything now obsolete in 10 must go in a different table. Related >>>>>>>>> changes required to source code and tests. Gory details below. >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8173421/webrev/ >>>>>>>>> >>>>>>>>> Testing: JPRT (in progress) >>>>>>>>> Manual runs of affected regression tests >>>>>>>>> >>>>>>>>> This will be pushed to jdk10/jdk10 along with the actual >>>>>>>>> change to >>>>>>>>> bump the major version number. >>>>>>>>> >>>>>>>>> There are some Zero and Aarch64 code tweaks that should have been >>>>>>>>> handled in JDK 9 but got overlooked somehow. Nothing significant. >>>>>>>> >>>>>>>> Hi David, >>>>>>>> >>>>>>>> This looks good. However, please do not remove PermSize and >>>>>>>> MaxPermSize >>>>>>>> in the special flag table in >>>>>>>> src/share/vm/runtime/arguments.cpp. We >>>>>>>> recently added these options back in JDK 9 for ease of migration, >>>>>>>> see >>>>>>>> JDK-8167446. I would rather see these two options put back in the >>>>>>>> table >>>>>>>> and have their expiration bumped to 11 until we decide going >>>>>>>> forward >>>>>>>> what is the correct course of action. Feel free to enter a new >>>>>>>> JDK 10 >>>>>>>> specific bug to address the issue of these two options >>>>>>>> specifically. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Lois >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> --- >>>>>>>>> >>>>>>>>> Non-aliased flags: >>>>>>>>> >>>>>>>>> AutoGCSelectPauseMillis (def: 5000) has now expired and is >>>>>>>>> removed >>>>>>>>> completely from the source code. >>>>>>>>> >>>>>>>>> UseAutoGCSelectPolicy (def: false) has now expired and is removed >>>>>>>>> completely from the source code as-if replaced by false. >>>>>>>>> >>>>>>>>> UseParNewGC (def: false) expired. However it is >>>>>>>>> expected/required to >>>>>>>>> be true whenever UseConcMarkSweepGC is true, and >>>>>>>>> UseConcMarkSweepGC >>>>>>>>> has not been deprecated. So we must remove all uses as-if it were >>>>>>>>> true, not false. >>>>>>>>> >>>>>>>>> ExplicitGCInvokesConcurrentAndUnloadsClasses (def: false) has now >>>>>>>>> expired and is removed completely from the source as-if >>>>>>>>> replaced by >>>>>>>>> false. >>>>>>>>> >>>>>>>>> ConvertSleepToYield (def: true) is now obsolete. It is moved >>>>>>>>> to the >>>>>>>>> obsolete table and is removed completely from the source code >>>>>>>>> as-if >>>>>>>>> replaced by true. >>>>>>>>> >>>>>>>>> ConvertYieldToSleep (def: false) is now obsolete. It is moved to >>>>>>>>> the >>>>>>>>> obsolete table and is removed completely from the source code >>>>>>>>> as-if >>>>>>>>> replaced by false. >>>>>>>>> >>>>>>>>> As a result of the above MinSleepInterval is no longer used (it >>>>>>>>> should >>>>>>>>> have also been deprecated in 9) and is also marked as obsolete. >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> Aliased options that have expired: >>>>>>>>> >>>>>>>>> CMSMarkStackSizeMax is an alias and has expired. It is removed >>>>>>>>> from >>>>>>>>> the alias and obsolete flag tables. >>>>>>>>> >>>>>>>>> CMSMarkStackSize is an alias and has expired. It is removed from >>>>>>>>> the >>>>>>>>> alias and obsolete flag tables. >>>>>>>>> >>>>>>>>> G1MarkStackSize is an alias and has expired. It is removed >>>>>>>>> from the >>>>>>>>> alias and obsolete flag tables. >>>>>>>>> >>>>>>>>> ParallelMarkingThreads is an alias and has expired. It is removed >>>>>>>>> from >>>>>>>>> the alias and obsolete flag tables. >>>>>>>>> >>>>>>>>> ParallelCMSThreads is an alias and has expired. It is removed >>>>>>>>> from >>>>>>>>> the >>>>>>>>> alias and obsolete flag tables. >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> The following flags all expire in 10 and exist (except where >>>>>>>>> noted) >>>>>>>>> only in the special-flag table (from which they are now >>>>>>>>> removed). Any >>>>>>>>> other uses of the flag are removed. >>>>>>>>> >>>>>>>>> UseOldInlining >>>>>>>>> SafepointPollOffset (defined in >>>>>>>>> ./cpu/aarch64/vm/c1_globals_aarch64.hpp) >>>>>>>>> UseBoundThreads (comments in os_solaris.cpp) >>>>>>>>> DefaultThreadPriority >>>>>>>>> NoYieldsInMicrolock >>>>>>>>> BackEdgeThreshold (defined in c1/2_globals_aarch64.hpp) >>>>>>>>> UseNewReflection >>>>>>>>> ReflectionWrapResolutionErrors >>>>>>>>> VerifyReflectionBytecodes >>>>>>>>> AutoShutdownNMT >>>>>>>>> NmethodSweepFraction >>>>>>>>> NmethodSweepCheckInterval >>>>>>>>> CodeCacheMinimumFreeSpace >>>>>>>>> UseFastAccessorMethods (ZERO) (still used in >>>>>>>>> ccpInterpreterGenerator_zero.cpp, jvmtiManageCapabilities.cpp) >>>>>>>>> UseFastEmptyMethods (ZERO) (still used in >>>>>>>>> ccpInterpreterGenerator_zero.cpp, jvmtiManageCapabilities.cpp) >>>>>>>>> UseCompilerSafepoints >>>>>>>>> AdaptiveSizePausePolicy >>>>>>>>> ParallelGCRetainPLAB >>>>>>>>> ThreadSafetyMargin >>>>>>>>> LazyBootClassLoader >>>>>>>>> StarvationMonitorInterval >>>>>>>>> PreInflateSpin >>>>>>>>> JNIDetachReleasesMonitors >>>>>>>>> UseAltSigs >>>>>>>>> SegmentedHeapDumpThreshold >>>>>>>>> PrintOopAddress (comment in method.cpp) >>>>>>>>> PermSize >>>>>>>>> MaxPermSize >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> Tests modified: >>>>>>>>> >>>>>>>>> runtime/CommandLine/ >>>>>>>>> ObsoleteFlagErrorMessage.java >>>>>>>>> VMDeprecatedOptions.java >>>>>>>>> VMAliasOptions.java >>>>>>>>> >>>>>>>>> gc/arguments/TestSelectDefaultGC.java >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> Tests removed (TEST.groups updated as needed): >>>>>>>>> >>>>>>>>> gc/startup_warnings/ >>>>>>>>> TestUseAutoGCSelectPolicy.java >>>>>>>>> TestParNewSerialOld.java >>>>>>>>> TestParNewCMS.java >>>>>>>>> TestDefNewCMS.java >>>>>>>>> gc/arguments/TestExplicitGCInvokesConcurrentAndUnloadsClasses.java >>>>>>>>> >>>>>>>>> >>>>>>>>> runtime/CommandLine/PermGenFlagsTest.java >>>>>>>>> runtime/NMT/AutoshutdownNMT.java >>>>>>>> >>>>>> >>> >> From david.holmes at oracle.com Wed Feb 1 04:47:16 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 1 Feb 2017 14:47:16 +1000 Subject: RFR(S): 8171848: ObjectMonitor verify() and print() methods are empty In-Reply-To: <41183671-ebd6-312d-34d4-bbde9e950f70@oracle.com> References: <585A5B15.80607@oracle.com> <83afe5b7-abb5-a960-b92b-655284a54531@oracle.com> <0f2829ac-a404-b256-41fc-0bd6d1b56656@oracle.com> <41183671-ebd6-312d-34d4-bbde9e950f70@oracle.com> Message-ID: Hi Kirill, On 1/02/2017 2:11 AM, Kirill Zhaldybin wrote: > David, > > Here are a new WebRev: > http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8171848/webrev.01/ > I deleted ObjectSynchronizer::verify too since I failed to find any > calls to it. > > Could you please let me know your opinion? Looks fine. Thanks, David ----- > Thank you. > > Regards, Kirill > > On 21.12.2016 16:24, Kirill Zhaldybin wrote: >> David, >> >> Thank you for review! >> >> On 21.12.2016 15:46, David Holmes wrote: >>> Hi Kirill, >>> >>> This cleanup is considered an enhancement and as such is no longer >>> acceptable in JDK 9 - sorry. The CR has been re-targetted to JDK 10. >>> >>> The removal itself seems okay. I wonder whether >>> ObjectSynchronizer::verify is itself ever called? >> I did not find any calls to it but it was after I sent WebRev. >> I will double-check and will send new one if we can just delete >> ObjectSynchronizer::verify. >> >> Regards, Kirill >>> >>> David >>> >>> On 21/12/2016 8:36 PM, Kirill Zhaldybin wrote: >>>> Dear all, >>>> >>>> Could you please review this fix for 8171848? >>>> I deleted both methods from ObjectMonitor and the code which was not >>>> doing anything from ObjectSynchronizer::verify(). >>>> I did local and rbt testing to be sure that I found all calls to >>>> deleted >>>> methods. >>>> >>>> WebRev: >>>> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8171848/webrev.00/ >>>> CR: https://bugs.openjdk.java.net/browse/JDK-8171848 >>>> >>>> Thank you. >>>> >>>> Regards, Kirill >> > From rwestrel at redhat.com Wed Feb 1 09:32:20 2017 From: rwestrel at redhat.com (Roland Westrelin) Date: Wed, 01 Feb 2017 10:32:20 +0100 Subject: RFR: 8173472: AArch64: C1 comparisons with null only use 32-bit instructions In-Reply-To: References: <8b9200d0-b5f4-8441-47f4-e32a630b4f1f@redhat.com> <42e977b5-db0b-73fa-5ba8-a144731bad47@oracle.com> <3073a060-f737-64cf-7b1a-74480aca8fe5@redhat.com> Message-ID: Hi David, > Well that's what I'm questioning. Maybe one of the compiler folk > (Roland?) can chime in. C1 was 32-bit only but there was a proprietary > 64-bit version. I don't know if OpenJDK ever supported a 64-bit C1. The > code would have to have been adapted for tiered compilation. Tiered runs 64-bit c1 so the code is well tested. Building a client only 64 bit VM is forbidden by the build scripts and is as far as I know considered unsupported but given how much testing it gets with tiered compilation, in practice, it's on par with 32 bit c1. Roland. From stuart.monteith at linaro.org Wed Feb 1 11:56:57 2017 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Wed, 1 Feb 2017 11:56:57 +0000 Subject: RFR: 8173472: AArch64: C1 comparisons with null only use 32-bit instructions In-Reply-To: References: <8b9200d0-b5f4-8441-47f4-e32a630b4f1f@redhat.com> <42e977b5-db0b-73fa-5ba8-a144731bad47@oracle.com> <3073a060-f737-64cf-7b1a-74480aca8fe5@redhat.com> Message-ID: We're testing with "-XX:+TiedCompilation -XX:TieredStopAtLevel=1" to try and catch C1 errors. I switched from building a special client build to this as it seemed more sustainable and supported. I've been trawling through the code, and there is definite confusion. For example: hotspot/src/share/vm/c1/c1_LIRGenerator.cpp 1290:LIRGenerator::do_getClass(Intrinsic *x) // FIXME T_ADDRESS should actually be T_METADATA but it can't because the // meaning of these two is mixed up (see JDK-8026837). See https://bugs.openjdk.java.net/browse/JDK-8026837 "Currently metadata klass pointers (Klass*) are of type T_ADDRESS which is wrong; the type should be T_METADATA." There are places where T_ADDRESS type arguments are used as an array element pointer, and others where it is used as metadata pointers/offsets. Why treating T_ADDRESS types as jints works, I've yet to find out, but I'm working from the assumption that is is as much through luck as design. BR, Stuart On 1 February 2017 at 09:32, Roland Westrelin wrote: > > Hi David, > >> Well that's what I'm questioning. Maybe one of the compiler folk >> (Roland?) can chime in. C1 was 32-bit only but there was a proprietary >> 64-bit version. I don't know if OpenJDK ever supported a 64-bit C1. The >> code would have to have been adapted for tiered compilation. > > Tiered runs 64-bit c1 so the code is well tested. Building a client only > 64 bit VM is forbidden by the build scripts and is as far as I know > considered unsupported but given how much testing it gets with tiered > compilation, in practice, it's on par with 32 bit c1. > > Roland. From david.holmes at oracle.com Wed Feb 1 12:34:11 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 1 Feb 2017 22:34:11 +1000 Subject: RFR: 8173472: AArch64: C1 comparisons with null only use 32-bit instructions In-Reply-To: References: <8b9200d0-b5f4-8441-47f4-e32a630b4f1f@redhat.com> <42e977b5-db0b-73fa-5ba8-a144731bad47@oracle.com> <3073a060-f737-64cf-7b1a-74480aca8fe5@redhat.com> Message-ID: Hi Roland, On 1/02/2017 7:32 PM, Roland Westrelin wrote: > > Hi David, > >> Well that's what I'm questioning. Maybe one of the compiler folk >> (Roland?) can chime in. C1 was 32-bit only but there was a proprietary >> 64-bit version. I don't know if OpenJDK ever supported a 64-bit C1. The >> code would have to have been adapted for tiered compilation. > > Tiered runs 64-bit c1 so the code is well tested. Building a client only > 64 bit VM is forbidden by the build scripts and is as far as I know > considered unsupported but given how much testing it gets with tiered > compilation, in practice, it's on par with 32 bit c1. Thanks for clarifying that. So can you also clarify why int is used for Address even in 64-bit? David > Roland. > From kirill.zhaldybin at oracle.com Wed Feb 1 13:29:20 2017 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Wed, 1 Feb 2017 16:29:20 +0300 Subject: RFR(S): 8171848: ObjectMonitor verify() and print() methods are empty In-Reply-To: References: <585A5B15.80607@oracle.com> <83afe5b7-abb5-a960-b92b-655284a54531@oracle.com> <0f2829ac-a404-b256-41fc-0bd6d1b56656@oracle.com> <41183671-ebd6-312d-34d4-bbde9e950f70@oracle.com> Message-ID: <0eb7870c-008d-1e3e-3230-cc40f980cf9d@oracle.com> David, Thank you for review! Regards, Kirill On 01.02.2017 07:47, David Holmes wrote: > Hi Kirill, > > On 1/02/2017 2:11 AM, Kirill Zhaldybin wrote: >> David, >> >> Here are a new WebRev: >> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8171848/webrev.01/ >> I deleted ObjectSynchronizer::verify too since I failed to find any >> calls to it. >> >> Could you please let me know your opinion? > > Looks fine. > > Thanks, > David > ----- > >> Thank you. >> >> Regards, Kirill >> >> On 21.12.2016 16:24, Kirill Zhaldybin wrote: >>> David, >>> >>> Thank you for review! >>> >>> On 21.12.2016 15:46, David Holmes wrote: >>>> Hi Kirill, >>>> >>>> This cleanup is considered an enhancement and as such is no longer >>>> acceptable in JDK 9 - sorry. The CR has been re-targetted to JDK 10. >>>> >>>> The removal itself seems okay. I wonder whether >>>> ObjectSynchronizer::verify is itself ever called? >>> I did not find any calls to it but it was after I sent WebRev. >>> I will double-check and will send new one if we can just delete >>> ObjectSynchronizer::verify. >>> >>> Regards, Kirill >>>> >>>> David >>>> >>>> On 21/12/2016 8:36 PM, Kirill Zhaldybin wrote: >>>>> Dear all, >>>>> >>>>> Could you please review this fix for 8171848? >>>>> I deleted both methods from ObjectMonitor and the code which was not >>>>> doing anything from ObjectSynchronizer::verify(). >>>>> I did local and rbt testing to be sure that I found all calls to >>>>> deleted >>>>> methods. >>>>> >>>>> WebRev: >>>>> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8171848/webrev.00/ >>>>> CR: https://bugs.openjdk.java.net/browse/JDK-8171848 >>>>> >>>>> Thank you. >>>>> >>>>> Regards, Kirill >>> >> From aph at redhat.com Wed Feb 1 16:04:05 2017 From: aph at redhat.com (Andrew Haley) Date: Wed, 1 Feb 2017 16:04:05 +0000 Subject: RFR: 8173472: AArch64: C1 comparisons with null only use 32-bit instructions In-Reply-To: References: <8b9200d0-b5f4-8441-47f4-e32a630b4f1f@redhat.com> <42e977b5-db0b-73fa-5ba8-a144731bad47@oracle.com> <3073a060-f737-64cf-7b1a-74480aca8fe5@redhat.com> Message-ID: On 01/02/17 11:56, Stuart Monteith wrote: > There are places where T_ADDRESS type arguments are used as an array > element pointer, and others where it is used as metadata > pointers/offsets. Why treating T_ADDRESS types as jints works, I've > yet to find out, but I'm working from the assumption that is is as > much through luck as design. Sure. Please get back to me when you know a bit more, especially if you have a reproducible test case. Thanks, Andrew. From rwestrel at redhat.com Wed Feb 1 16:42:52 2017 From: rwestrel at redhat.com (Roland Westrelin) Date: Wed, 01 Feb 2017 17:42:52 +0100 Subject: RFR: 8173472: AArch64: C1 comparisons with null only use 32-bit instructions In-Reply-To: References: <8b9200d0-b5f4-8441-47f4-e32a630b4f1f@redhat.com> <42e977b5-db0b-73fa-5ba8-a144731bad47@oracle.com> <3073a060-f737-64cf-7b1a-74480aca8fe5@redhat.com> Message-ID: > Thanks for clarifying that. So can you also clarify why int is used for > Address even in 64-bit? The use of T_ADDRESS is quite limited in c1. It's supposed to be for anything that's nor a java object or a metadata pointer (but as Stuart found out it's not that clean). So I suppose we never hit a constant T_ADDRESS and we're just lucky. Roland. From matthias.baesken at sap.com Thu Feb 2 14:44:10 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 2 Feb 2017 14:44:10 +0000 Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp Message-ID: Hello, could I please have a review for the following small adjustment to the comment of UseAES in globals.hpp . UseAES is available now on more than x86/x64, for example s390x and ppc64le so the comment in globals.hpp needs to be adjusted. I also need a sponsor for this. Bug : https://bugs.openjdk.java.net/browse/JDK-8173825 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.0/ Thanks, Matthias From christoph.langer at sap.com Thu Feb 2 14:47:21 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 2 Feb 2017 14:47:21 +0000 Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp In-Reply-To: References: Message-ID: <7d5cce3ce5b54651b92bf812e4e121cc@derote13de46.global.corp.sap> Hi Matthias, looks good. I'm wondering whether this textual change could still make it into JDK9...? Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 2. Februar 2017 15:44 To: hotspot-dev at openjdk.java.net Cc: Schmidt, Lutz ; Langer, Christoph ; Simonis, Volker Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp Hello, could I please have a review for the following small adjustment to the comment of UseAES in globals.hpp . UseAES is available now on more than x86/x64, for example s390x and ppc64le so the comment in globals.hpp needs to be adjusted. I also need a sponsor for this. Bug : https://bugs.openjdk.java.net/browse/JDK-8173825 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.0/ Thanks, Matthias From vladimir.kozlov at oracle.com Thu Feb 2 16:41:05 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 2 Feb 2017 08:41:05 -0800 Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp In-Reply-To: <7d5cce3ce5b54651b92bf812e4e121cc@derote13de46.global.corp.sap> References: <7d5cce3ce5b54651b92bf812e4e121cc@derote13de46.global.corp.sap> Message-ID: On 2/2/17 6:47 AM, Langer, Christoph wrote: > Hi Matthias, > > looks good. +1 > > I'm wondering whether this textual change could still make it into JDK9...? Make it P3 and change "Affects" and "Fix" version to 9. Thanks, Vladimir > > Best regards > Christoph > > > From: Baesken, Matthias > Sent: Donnerstag, 2. Februar 2017 15:44 > To: hotspot-dev at openjdk.java.net > Cc: Schmidt, Lutz ; Langer, Christoph ; Simonis, Volker > Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp > > Hello, > could I please have a review for the following small adjustment to the comment of UseAES in globals.hpp . > UseAES is available now on more than x86/x64, for example s390x and ppc64le so the comment in globals.hpp needs to be adjusted. > I also need a sponsor for this. > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8173825 > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.0/ > > > Thanks, Matthias > From matthias.baesken at sap.com Fri Feb 3 11:52:00 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 3 Feb 2017 11:52:00 +0000 Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp Message-ID: >> Hi Matthias, >> >> looks good. > >+1 > >> >> I'm wondering whether this textual change could still make it into JDK9...? > >Make it P3 and change "Affects" and "Fix" version to 9. > > Thanks, > Vladimir Hi Vladimir, I adjusted the bug as suggested. Additional JDK9 webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.1/ Can you please sponsor it ? Thanks, Matthias >> >> Best regards >> Christoph >> >> >> From: Baesken, Matthias >> Sent: Donnerstag, 2. Februar 2017 15:44 >> To: hotspot-dev at openjdk.java.net >> Cc: Schmidt, Lutz >; Langer, Christoph >; Simonis, Volker > >> Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp >> >> Hello, >> could I please have a review for the following small adjustment to the comment of UseAES in globals.hpp . >> UseAES is available now on more than x86/x64, for example s390x and ppc64le so the comment in globals.hpp needs to be adjusted. >> I also need a sponsor for this. >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8173825 >> >> Webrev : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.0/ >> From vladimir.kozlov at oracle.com Fri Feb 3 17:46:32 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 3 Feb 2017 09:46:32 -0800 Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp In-Reply-To: References: Message-ID: <58e1e6b4-5296-139d-e4df-6e5d6269f3f6@oracle.com> Hi Matthias, I looked on other flags and I want to remove line 667 too if you are okay with it: 665 product(bool, UseSHA, false, \ 666 "Control whether SHA instructions can be used " \ 667 "on SPARC, on ARM and on x86") \ Thanks, Vladimir On 2/3/17 3:52 AM, Baesken, Matthias wrote: >>>/Hi Matthias,/ > >>>/ / > >>>/looks good./ > >> > >>+1 > >> > >>>/ / > >>>/I'm wondering whether this textual change could still make it into > JDK9...?/ > >> > >>Make it P3 and change "Affects" and "Fix" version to 9. > >> > >> Thanks, > >> Vladimir > > > > Hi Vladimir, I adjusted the bug as suggested. > > > > Additional JDK9 webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.1/ > > > > Can you please sponsor it ? > > > > Thanks, Matthias > > > > > >>>/ / > >>>/Best regards/ > >>>/Christoph/ > >>>/ / > >>>/ / > >>>/From: Baesken, Matthias/ > >>>/Sent: Donnerstag, 2. Februar 2017 15:44/ > >>>/To: //hotspot-dev at openjdk.java.net > /// > >>>/Cc: Schmidt, Lutz //>; Langer, Christoph //>; Simonis, Volker //>/ > >>>/Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp/ > >>>/ / > >>>/Hello,/ > >>>/ could I please have a review for the following small adjustment > to the comment of UseAES in globals.hpp ./ > >>>/UseAES is available now on more than x86/x64, for example s390x and > ppc64le so the comment in globals.hpp needs to be adjusted./ > >>>/I also need a sponsor for this./ > >>>/ / > >>>/Bug :/ > >>>/ / > >>>///https://bugs.openjdk.java.net/browse/JDK-8173825/// > >>>/ / > >>>/Webrev :/ > >>>/ / > >>>///http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.0//// > >>>/ / > > > From matthias.baesken at sap.com Mon Feb 6 07:43:48 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 6 Feb 2017 07:43:48 +0000 Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp In-Reply-To: <58e1e6b4-5296-139d-e4df-6e5d6269f3f6@oracle.com> References: <58e1e6b4-5296-139d-e4df-6e5d6269f3f6@oracle.com> Message-ID: Hi Vladimir, that's a good idea , thanks for adjusting it. Best regards, Matthias > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Freitag, 3. Februar 2017 18:47 > To: Baesken, Matthias ; hotspot- > dev at openjdk.java.net > Cc: Langer, Christoph > Subject: Re: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp > > Hi Matthias, > > I looked on other flags and I want to remove line 667 too if you are > okay with it: > > 665 product(bool, UseSHA, false, \ > 666 "Control whether SHA instructions can be used " \ > 667 "on SPARC, on ARM and on x86") \ > > Thanks, > Vladimir > > On 2/3/17 3:52 AM, Baesken, Matthias wrote: > >>>/Hi Matthias,/ > > > >>>/ / > > > >>>/looks good./ > > > >> > > > >>+1 > > > >> > > > >>>/ / > > > >>>/I'm wondering whether this textual change could still make it into > > JDK9...?/ > > > >> > > > >>Make it P3 and change "Affects" and "Fix" version to 9. > > > >> > > > >> Thanks, > > > >> Vladimir > > > > > > > > Hi Vladimir, I adjusted the bug as suggested. > > > > > > > > Additional JDK9 webrev : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.1/ > > > > > > > > Can you please sponsor it ? > > > > > > > > Thanks, Matthias > > > > > > > > > > > >>>/ / > > > >>>/Best regards/ > > > >>>/Christoph/ > > > >>>/ / > > > >>>/ / > > > >>>/From: Baesken, Matthias/ > > > >>>/Sent: Donnerstag, 2. Februar 2017 15:44/ > > > >>>/To: //hotspot-dev at openjdk.java.net > > /// > > > >>>/Cc: Schmidt, Lutz > //>; Langer, > Christoph > //>; Simonis, > Volker > //>/ > > > >>>/Subject: RFR [XXS] : Adjust the comment for flag UseAES in globals.hpp/ > > > >>>/ / > > > >>>/Hello,/ > > > >>>/ could I please have a review for the following small adjustment > > to the comment of UseAES in globals.hpp ./ > > > >>>/UseAES is available now on more than x86/x64, for example s390x and > > ppc64le so the comment in globals.hpp needs to be adjusted./ > > > >>>/I also need a sponsor for this./ > > > >>>/ / > > > >>>/Bug :/ > > > >>>/ / > > > >>>///https://bugs.openjdk.java.net/browse/JDK-8173825/// > > > >>>/ / > > > >>>/Webrev :/ > > > >>>/ / > > > >>>///http://cr.openjdk.java.net/~mbaesken/webrevs/8173825.0//// > > > >>>/ / > > > > > > From gromero at linux.vnet.ibm.com Mon Feb 6 21:50:12 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Mon, 6 Feb 2017 19:50:12 -0200 Subject: Linux/PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used Message-ID: <5898EF94.5060904@linux.vnet.ibm.com> Hi, On Linux/PPC64 I'm getting a series of "mbind: Invalid argument" that seems exactly the same as reported for x64 [1]: [root at spocfire3 ~]# java -XX:+UseNUMA -version mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument openjdk version "1.8.0_121" OpenJDK Runtime Environment (build 1.8.0_121-b13) OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) [root at spocfire3 ~]# uname -a Linux spocfire3.aus.stglabs.ibm.com 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux [root at spocfire3 ~]# lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 160 On-line CPU(s) list: 0-159 Thread(s) per core: 8 Core(s) per socket: 10 Socket(s): 2 NUMA node(s): 2 Model: 2.0 (pvr 004d 0200) Model name: POWER8 (raw), altivec supported L1d cache: 64K L1i cache: 32K L2 cache: 512K L3 cache: 8192K NUMA node0 CPU(s): 0-79 NUMA node8 CPU(s): 80-159 On chasing down it, looks like it comes from PSYoungGen::initialize() in src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp that calls initialize_work(), that calls the MutableNUMASpace() constructor if UseNUMA is set: http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/567e410935e5/src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp#l77 MutableNUMASpace() then calls os::numa_make_local(), that in the end calls numa_set_bind_policy() in libnuma.so.1 [2]. I've traced some values for which mbind() syscall fails: http://termbin.com/ztfs (search for "Invalid argument" in the log). Assuming it's the same bug as reported in [1] and so it's not fixed on 9 and 10: - Is there any WIP or known workaround? - Should I append this output in [1] description or open a new one and make it related to" [1]? Thank you. Best regards, Gustavo [1] https://bugs.openjdk.java.net/browse/JDK-8163796 [2] https://da.gd/4vXF From sangheon.kim at oracle.com Mon Feb 6 22:23:35 2017 From: sangheon.kim at oracle.com (sangheon) Date: Mon, 6 Feb 2017 14:23:35 -0800 Subject: Linux/PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: <5898EF94.5060904@linux.vnet.ibm.com> References: <5898EF94.5060904@linux.vnet.ibm.com> Message-ID: <8a0d9151-8ed7-9409-20a1-10258023ae7d@oracle.com> Hi Gustavo, On 02/06/2017 01:50 PM, Gustavo Romero wrote: > Hi, > > On Linux/PPC64 I'm getting a series of "mbind: Invalid argument" that seems > exactly the same as reported for x64 [1]: > > [root at spocfire3 ~]# java -XX:+UseNUMA -version > mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument > openjdk version "1.8.0_121" > OpenJDK Runtime Environment (build 1.8.0_121-b13) > OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) > > [root at spocfire3 ~]# uname -a > Linux spocfire3.aus.stglabs.ibm.com 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux > > [root at spocfire3 ~]# lscpu > Architecture: ppc64le > Byte Order: Little Endian > CPU(s): 160 > On-line CPU(s) list: 0-159 > Thread(s) per core: 8 > Core(s) per socket: 10 > Socket(s): 2 > NUMA node(s): 2 > Model: 2.0 (pvr 004d 0200) > Model name: POWER8 (raw), altivec supported > L1d cache: 64K > L1i cache: 32K > L2 cache: 512K > L3 cache: 8192K > NUMA node0 CPU(s): 0-79 > NUMA node8 CPU(s): 80-159 > > On chasing down it, looks like it comes from PSYoungGen::initialize() in > src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp that calls > initialize_work(), that calls the MutableNUMASpace() constructor if > UseNUMA is set: > http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/567e410935e5/src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp#l77 > > MutableNUMASpace() then calls os::numa_make_local(), that in the end calls > numa_set_bind_policy() in libnuma.so.1 [2]. > > I've traced some values for which mbind() syscall fails: > http://termbin.com/ztfs (search for "Invalid argument" in the log). > > Assuming it's the same bug as reported in [1] and so it's not fixed on 9 and 10: > > - Is there any WIP or known workaround? There's no progress on JDK-8163796 and no workaround found yet. And unfortunately, I'm not planning to fix it soon. > - Should I append this output in [1] description or open a new one and make it > related to" [1]? I think your problem seems same as JDK-8163796, so adding your output on the CR seems good. And please add logs as well. I recommend to enabling something like "-Xlog:gc*,gc+heap*=trace". IIRC, the problem was only occurred when the -Xmx was small in my case. Thanks, Sangheon > > Thank you. > > > Best regards, > Gustavo > > [1] https://bugs.openjdk.java.net/browse/JDK-8163796 > [2] https://da.gd/4vXF > From kim.barrett at oracle.com Tue Feb 7 01:03:44 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 6 Feb 2017 20:03:44 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Apologies for taking so long to respond to earlier comments. I've been distracted by some other tasks, and it took a while to figure out what to do about the problem Mikael reported. Martin Doerr - Thanks for the ppc and s390 native call generator modifications. Do you want a contributed-by credit when pushed? Andrew Haley - Thanks for the syntax fix. The new version with always tagged jweaks (see below) makes testing easier, and there's a new test included. Testing the G1-specific behavior still needs the whitebox extension. I've updated the G1-specific test (which is still not part of the changeset for JDK 9). Per Liden - I tried your suggestion of making jweak unconditionally tagged. It doesn't eliminate much of the INCLUDE_ALL_GCS conditionals, since those are still needed to protect references to G1-specific code. (The littering of otherwise generic runtime code with this G1-specific stuff should be a cleanup task for JDK 10.) However, it does simplify some of the code, and simplifies testing. The cost is a "bit-test and (predicted taken) conditional branch" in the handling of an oop result from a native call. I agree that's unlikely to be noticable, and the benefits seem worthwhile, so I've made that change. I've attempted to update the platform-specific native call return handling changes for all of the platforms, including those I don't have access to (aarch64, ppc, s390). I also fixed the zero change I'd made earlier. And I made shark error, to force anyone using it to fix the handling of an oop result by SharkNativeWrapper::initialize(). There might be other zero/shark changes that I've missed. All of these should be checked by appropriate platform experts. I had to fix a JVMTI bug as part of this; FollowReferences wasn't doing as much argument checking as expected. This was unconvered by a closed test that unexpectedly asserted. This test would have similarly asserted previously if CheckJNICalls was true; I removed that conditionalization in JNIHandles::resolve, as it was only there to avoid an expensive check for the jobject being a jweak, and in the new code we already know it isn't. There might be other JVMTI functions with similar issues. Mikael - There are some ugly worms lurking under that rock! I think I've dealt with the problem, tripping over some other bugs along the way. Thanks for the test case. I tweaked it a bit and included it in the updated changeset. The passing of arguments involves collecting them into a JavaCallArguments object. oop arguments are in some handlized form (either jobject or Handle). The existing code heavily cheats; for example, it assumes one can correctly cast from a jobject to an oop* and dereference that pointer. Obviously, a tagged jweak breaks that assumption. Even worse, there are casts from oop* to oop and back! (These are so nasty they *must* use C-style cast syntax.) I'm thinking about some followup cleanup bugs in that area. To address this, I changed JavaCallArguments to avoid conversions from jobject to Handle/oop*. To support that, it now keeps more accurate track of the state of the object references in the collected arguments. It also delays resolving jobjects to their designated oops, avoiding intermediate Handle objects. (This was a place where cheating was going on, in order to avoid intermediate Handle allocations. We're now accomplishing that without crashing through the JNIHandles abstraction.) CR: https://bugs.openjdk.java.net/browse/JDK-8166188 New webrevs: full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.inc The full webrev may be more useful than the incremental; some of the incremental changes are pretty hard to read. Some additional webrevs that might be helpful: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.other-platforms Incremental changes for aarch64, ppc, and s390 supplied by Andrew and Martin. http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.always_tag[.full] Incremental [full] changes to always tag jweaks. The full webrev may be more useful than the incremental; some of the incremental changes are pretty hard to read. http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.java_call Incremental changes to address the java call problem found by Mikael. hotspot.04 == hotspot.02 + hotspot.04.inc hotspot.04 == hotspot.04.always_tag.full + hotspot.04.java_call hotspot.04 == hotspot.02 + hotspot.04.other-platforms + hotspot.04.always_tag + hotspot.04.java_call The updated G1-specific test requiring the whitebox extension is http://cr.openjdk.java.net/~kbarrett/8166188/test.04 The whitebox extension has not changed: http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ From martin.doerr at sap.com Tue Feb 7 13:59:16 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 7 Feb 2017 13:59:16 +0000 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: <4f5a4b9c9ea4422b95b49a7824c2b1db@DEWDFE13DE02.global.corp.sap> Hi Kim, thanks for adding and adapting my code. There's only one minor bug on PPC64. Please use the correct label name for the conditional branch as shown in the diff below. If you like, you can add simonis and mdoerr as contributors, but I don't insist on it. I guess SignatureChekker was written with "kk" to distinguish it from something else? Besides that, the change looks good to me. Thanks and best regards, Martin --- a/src/cpu/ppc/vm/sharedRuntime_ppc.cpp Tue Feb 07 12:48:05 2017 +0100 +++ b/src/cpu/ppc/vm/sharedRuntime_ppc.cpp Tue Feb 07 14:30:48 2017 +0100 @@ -2483,7 +2483,7 @@ if (ret_type == T_OBJECT || ret_type == T_ARRAY) { Label done, not_weak; __ cmpdi(CCR0, R3_RET, 0); - __ beq(CCR0, null_result); // Use NULL as-is. + __ beq(CCR0, done); // Use NULL as-is. __ andi_(r_temp_1, R3_RET, JNIHandles::weak_tag_mask); __ beq(CCR0, not_weak); // Test for jweak tag. -----Original Message----- From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett Sent: Dienstag, 7. Februar 2017 02:04 To: hotspot-dev developers Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles Apologies for taking so long to respond to earlier comments. I've been distracted by some other tasks, and it took a while to figure out what to do about the problem Mikael reported. Martin Doerr - Thanks for the ppc and s390 native call generator modifications. Do you want a contributed-by credit when pushed? Andrew Haley - Thanks for the syntax fix. The new version with always tagged jweaks (see below) makes testing easier, and there's a new test included. Testing the G1-specific behavior still needs the whitebox extension. I've updated the G1-specific test (which is still not part of the changeset for JDK 9). Per Liden - I tried your suggestion of making jweak unconditionally tagged. It doesn't eliminate much of the INCLUDE_ALL_GCS conditionals, since those are still needed to protect references to G1-specific code. (The littering of otherwise generic runtime code with this G1-specific stuff should be a cleanup task for JDK 10.) However, it does simplify some of the code, and simplifies testing. The cost is a "bit-test and (predicted taken) conditional branch" in the handling of an oop result from a native call. I agree that's unlikely to be noticable, and the benefits seem worthwhile, so I've made that change. I've attempted to update the platform-specific native call return handling changes for all of the platforms, including those I don't have access to (aarch64, ppc, s390). I also fixed the zero change I'd made earlier. And I made shark error, to force anyone using it to fix the handling of an oop result by SharkNativeWrapper::initialize(). There might be other zero/shark changes that I've missed. All of these should be checked by appropriate platform experts. I had to fix a JVMTI bug as part of this; FollowReferences wasn't doing as much argument checking as expected. This was unconvered by a closed test that unexpectedly asserted. This test would have similarly asserted previously if CheckJNICalls was true; I removed that conditionalization in JNIHandles::resolve, as it was only there to avoid an expensive check for the jobject being a jweak, and in the new code we already know it isn't. There might be other JVMTI functions with similar issues. Mikael - There are some ugly worms lurking under that rock! I think I've dealt with the problem, tripping over some other bugs along the way. Thanks for the test case. I tweaked it a bit and included it in the updated changeset. The passing of arguments involves collecting them into a JavaCallArguments object. oop arguments are in some handlized form (either jobject or Handle). The existing code heavily cheats; for example, it assumes one can correctly cast from a jobject to an oop* and dereference that pointer. Obviously, a tagged jweak breaks that assumption. Even worse, there are casts from oop* to oop and back! (These are so nasty they *must* use C-style cast syntax.) I'm thinking about some followup cleanup bugs in that area. To address this, I changed JavaCallArguments to avoid conversions from jobject to Handle/oop*. To support that, it now keeps more accurate track of the state of the object references in the collected arguments. It also delays resolving jobjects to their designated oops, avoiding intermediate Handle objects. (This was a place where cheating was going on, in order to avoid intermediate Handle allocations. We're now accomplishing that without crashing through the JNIHandles abstraction.) CR: https://bugs.openjdk.java.net/browse/JDK-8166188 New webrevs: full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.inc The full webrev may be more useful than the incremental; some of the incremental changes are pretty hard to read. Some additional webrevs that might be helpful: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.other-platforms Incremental changes for aarch64, ppc, and s390 supplied by Andrew and Martin. http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.always_tag[.full] Incremental [full] changes to always tag jweaks. The full webrev may be more useful than the incremental; some of the incremental changes are pretty hard to read. http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.java_call Incremental changes to address the java call problem found by Mikael. hotspot.04 == hotspot.02 + hotspot.04.inc hotspot.04 == hotspot.04.always_tag.full + hotspot.04.java_call hotspot.04 == hotspot.02 + hotspot.04.other-platforms + hotspot.04.always_tag + hotspot.04.java_call The updated G1-specific test requiring the whitebox extension is http://cr.openjdk.java.net/~kbarrett/8166188/test.04 The whitebox extension has not changed: http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ From stuart.monteith at linaro.org Tue Feb 7 14:12:18 2017 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Tue, 7 Feb 2017 14:12:18 +0000 Subject: PrintCFGToFile crashes VM Message-ID: When running with: -XX:PrintCFGToFile It is very straightforward for the VM to SIGSEGV. The single CFGPrinterOutput instance isn't serialized, and so multiple threads are setting its _do_print_HIR and _do_print_LIR flags. This causes a crash when one compilation thread is trying to print LIRs, even when there aren't any. For example: #10 0x000003ffa769df14 in LIR_List::length (this=0x0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_LIR.hpp:2009 #11 0x000003ffa769cf10 in CFGPrinterOutput::print_LIR (this=0x3fdf4031910, block=0x3fdb8004fc0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_CFGPrinter.cpp:267 #12 0x000003ffa769d4a4 in CFGPrinterOutput::print_block (this=0x3fdf4031910, block=0x3fdb8004fc0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_CFGPrinter.cpp:338 #13 0x000003ffa769ef14 in CFGPrinterOutput::PrintBlockClosure::block_do (this=0x3fdcfffdcb0, block=0x3fdb8004fc0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_CFGPrinter.cpp:45 #14 0x000003ffa76f168c in BlockBegin::iterate_preorder (this=0x3fdb8004fc0, mark=..., closure=0x3fdcfffdcb0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_Instruction.cpp:706 #15 0x000003ffa76f1708 in BlockBegin::iterate_preorder (this=0x3fdb8006bc0, mark=..., closure=0x3fdcfffdcb0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_Instruction.cpp:709 #16 0x000003ffa76f18dc in BlockBegin::iterate_preorder (this=0x3fdb8006bc0, closure=0x3fdcfffdcb0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_Instruction.cpp:728 #17 0x000003ffa76e9cec in IR::iterate_preorder (this=0x3fdb8004ac0, closure=0x3fdcfffdcb0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_IR.cpp:1200 #18 0x000003ffa769d5e0 in CFGPrinterOutput::print_cfg (this=0x3fdf4031910, blocks=0x3fdb8004ac0, name=0x3ffa83017d8 "After Generation of HIR") at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_CFGPrinter.cpp:362 #19 0x000003ffa769c310 in CFGPrinter::print_cfg (blocks=0x3fdb8004ac0, name=0x3ffa83017d8 "After Generation of HIR", do_print_HIR=true, do_print_LIR=false) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_CFGPrinter.cpp:98 #20 0x000003ffa76b1264 in Compilation::build_hir (this=0x3fdcfffe0d0) at /home/stuart/repos/jdk9dev/hotspot/src/share/vm/c1/c1_Compilation.cpp:164 I've constructed a straw-man proposal for how this might be handled (see below), which is to force there to be one compilation thread if you use this flags (or two if C2 is enabled). Another possibility is to explicitly serialize access to the CFGPrinterOutput object. In principle we could have 1 thread for C1 and as many as are normal for C2, but that may be complicating things unnecessarily. I'm keen to hear opinions. This is low priority, as it is only present in debug builds, and setting the CICompileCount=1 is sufficient to get correct behaviour. Stuart # HG changeset patch # User smonteith # Date 1486476452 0 # Tue Feb 07 14:07:32 2017 +0000 # Node ID 530f3652e974874118e90f13639c8e528452f334 # Parent 0ae983a3af0759530a6b59fff35f18e8ac88816e Option PrintCFGToFile forces single threaded compilation Running with --XX:PrintCFGToFile causes a SIGSEGV as code is not thread-safe. Overrides CICompilerCount settings to use one cpu. diff -r 0ae983a3af07 -r 530f3652e974 src/share/vm/runtime/arguments.cpp --- a/src/share/vm/runtime/arguments.cpp Wed Jan 11 16:32:35 2017 +0000 +++ b/src/share/vm/runtime/arguments.cpp Tue Feb 07 14:07:32 2017 +0000 @@ -2582,6 +2582,23 @@ FLAG_SET_CMDLINE(bool, PostLoopMultiversioning, false); } #endif + +#ifndef PRODUCT + // C1's CFGPrinter must run on a single thread. If PrintCFGToFile is enabled, this + // code forces the compilers to use one or two threads. + if (PrintCFGToFile) { + // C1 and C2 each require 1 thread. + if(TieredCompilation && TieredStopAtLevel == CompLevel_full_optimization) { + warning("PrintCFGToFile set, CICompilerCount set to 2, -CICompilerCountPerCPU "); + FLAG_SET_CMDLINE(intx, CICompilerCount, 2); + FLAG_SET_CMDLINE(bool, CICompilerCountPerCPU, false); + } else { + warning("PrintCFGToFile set, CICompilerCount set to 1, -CICompilerCountPerCPU "); + FLAG_SET_CMDLINE(intx, CICompilerCount, 1); + FLAG_SET_CMDLINE(bool, CICompilerCountPerCPU, false); + } + } +#endif return status; } From kim.barrett at oracle.com Tue Feb 7 22:29:22 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 7 Feb 2017 17:29:22 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <4f5a4b9c9ea4422b95b49a7824c2b1db@DEWDFE13DE02.global.corp.sap> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <4f5a4b9c9ea4422b95b49a7824c2b1db@DEWDFE13DE02.global.corp.sap> Message-ID: <840D74CC-0ABA-43C4-BDC8-9A012E44CA79@oracle.com> > On Feb 7, 2017, at 8:59 AM, Doerr, Martin wrote: > > Hi Kim, > > thanks for adding and adapting my code. There's only one minor bug on PPC64. Please use the correct label name for the conditional branch as shown in the diff below. > > If you like, you can add simonis and mdoerr as contributors, but I don't insist on it. > > I guess SignatureChekker was written with "kk" to distinguish it from something else? > > Besides that, the change looks good to me. > > Thanks and best regards, > Martin > > > --- a/src/cpu/ppc/vm/sharedRuntime_ppc.cpp Tue Feb 07 12:48:05 2017 +0100 > +++ b/src/cpu/ppc/vm/sharedRuntime_ppc.cpp Tue Feb 07 14:30:48 2017 +0100 > @@ -2483,7 +2483,7 @@ > if (ret_type == T_OBJECT || ret_type == T_ARRAY) { > Label done, not_weak; > __ cmpdi(CCR0, R3_RET, 0); > - __ beq(CCR0, null_result); // Use NULL as-is. > + __ beq(CCR0, done); // Use NULL as-is. > > __ andi_(r_temp_1, R3_RET, JNIHandles::weak_tag_mask); > __ beq(CCR0, not_weak); // Test for jweak tag. Oops. Thanks for the fix. And thanks for the review. From dean.long at oracle.com Tue Feb 7 23:56:31 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 7 Feb 2017 15:56:31 -0800 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: <08918c3d-395c-2e29-36e5-d06128f45122@oracle.com> Hi Kim. I took a look at the sharedRuntime_ and templateInterpreterGenerator_ for arm, sparc, and x86. In sharedRuntime_arm.cpp, I suggest simplifying: 1750 #ifdef AARCH64 1751 __ cbz(R0, done); // Use NULL as-is. 1752 STATIC_ASSERT(JNIHandles::weak_tag_mask == 1u); 1753 __ tbz(R0, 0, not_weak); // Test for jweak tag. 1754 #else // !AARCH64 1755 __ cmp(R0, 0); 1756 __ b(done, eq); // Use NULL as-is. 1757 __ tst(R0, JNIHandles::weak_tag_mask); // Test for jweak tag. 1758 __ b(not_weak, eq); 1759 #endif // !AARCH64 to the equivalent: 1751 __ cbz(R0, done); // Use NULL as-is. 1752 STATIC_ASSERT(JNIHandles::weak_tag_mask == 1u); 1753 __ tbz(R0, 0, not_weak); // Test for jweak tag. as cbz and tbz expand to cmp/b and tst/b on arm32. The others look good. dl On 2/6/17 5:03 PM, Kim Barrett wrote: > Apologies for taking so long to respond to earlier comments. I've > been distracted by some other tasks, and it took a while to figure out > what to do about the problem Mikael reported. > > Martin Doerr - Thanks for the ppc and s390 native call generator > modifications. Do you want a contributed-by credit when pushed? > > Andrew Haley - Thanks for the syntax fix. The new version with always > tagged jweaks (see below) makes testing easier, and there's a new test > included. Testing the G1-specific behavior still needs the whitebox > extension. I've updated the G1-specific test (which is still not part > of the changeset for JDK 9). > > Per Liden - I tried your suggestion of making jweak unconditionally > tagged. It doesn't eliminate much of the INCLUDE_ALL_GCS > conditionals, since those are still needed to protect references to > G1-specific code. (The littering of otherwise generic runtime code > with this G1-specific stuff should be a cleanup task for JDK 10.) > However, it does simplify some of the code, and simplifies testing. > The cost is a "bit-test and (predicted taken) conditional branch" in > the handling of an oop result from a native call. I agree that's > unlikely to be noticable, and the benefits seem worthwhile, so I've > made that change. > > I've attempted to update the platform-specific native call return > handling changes for all of the platforms, including those I don't > have access to (aarch64, ppc, s390). I also fixed the zero change I'd > made earlier. And I made shark error, to force anyone using it to fix > the handling of an oop result by SharkNativeWrapper::initialize(). > There might be other zero/shark changes that I've missed. All of > these should be checked by appropriate platform experts. > > I had to fix a JVMTI bug as part of this; FollowReferences wasn't > doing as much argument checking as expected. This was unconvered by a > closed test that unexpectedly asserted. This test would have > similarly asserted previously if CheckJNICalls was true; I removed > that conditionalization in JNIHandles::resolve, as it was only there > to avoid an expensive check for the jobject being a jweak, and in the > new code we already know it isn't. There might be other JVMTI > functions with similar issues. > > Mikael - There are some ugly worms lurking under that rock! I think > I've dealt with the problem, tripping over some other bugs along the > way. Thanks for the test case. I tweaked it a bit and included it in > the updated changeset. > > The passing of arguments involves collecting them into a > JavaCallArguments object. oop arguments are in some handlized form > (either jobject or Handle). The existing code heavily cheats; for > example, it assumes one can correctly cast from a jobject to an oop* > and dereference that pointer. Obviously, a tagged jweak breaks that > assumption. Even worse, there are casts from oop* to oop and back! > (These are so nasty they *must* use C-style cast syntax.) I'm thinking > about some followup cleanup bugs in that area. > > To address this, I changed JavaCallArguments to avoid conversions > from jobject to Handle/oop*. To support that, it now keeps more > accurate track of the state of the object references in the collected > arguments. It also delays resolving jobjects to their designated > oops, avoiding intermediate Handle objects. (This was a place where > cheating was going on, in order to avoid intermediate Handle > allocations. We're now accomplishing that without crashing through > the JNIHandles abstraction.) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.inc > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > Some additional webrevs that might be helpful: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.other-platforms > Incremental changes for aarch64, ppc, and s390 supplied by Andrew and > Martin. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.always_tag[.full] > Incremental [full] changes to always tag jweaks. > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.java_call > Incremental changes to address the java call problem found by Mikael. > > hotspot.04 == hotspot.02 + hotspot.04.inc > hotspot.04 == hotspot.04.always_tag.full + hotspot.04.java_call > hotspot.04 == hotspot.02 + hotspot.04.other-platforms + > hotspot.04.always_tag + hotspot.04.java_call > > The updated G1-specific test requiring the whitebox extension is > http://cr.openjdk.java.net/~kbarrett/8166188/test.04 > > The whitebox extension has not changed: > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ > > From david.holmes at oracle.com Wed Feb 8 02:32:23 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 8 Feb 2017 12:32:23 +1000 Subject: Hotspot for BSD / opinion request In-Reply-To: References: <1488ccc1-e00c-42e7-3cfc-9fd0b5699a63@oracle.com> <3c580abc-9333-8803-abc1-1b0863123060@oracle.com> Message-ID: <64159c85-d58d-64b0-6250-8b0f3f052a5b@oracle.com> Hi David, Sorry for the long lag on this. I've been looking into the mutex attribute defaults and it is unfortunate that POSIX allowed PTHREAD_MUTEX_DEFAULT to be an implementation specific alias for one of the other three mutex types: PTHREAD_MUTEX_NORMAL, PTHREAD_MUTEX_ERRORCHECK, PTHREAD_MUTEX_RECURSIVE. But even more unfortunate that FreeBSD decided to map it to ERRORCHECK instead of NORMAL as every other platform seems to do! (I note NetBSD defines a DEFAULT that is compatible with NORMAL, not ERRORCHECK.) Fixing this for FreeBSD when it is not an officially supported platform is a little messy because we don't want to penalize all the other platforms with extra initialization code. Right now this could be confined to os_bsd.* but this code should be refactored into os_posix.* to be more widely shared. That aside we do not need to keep an attr object per mutex as they will all share the same attributes. So we would have a single static instance and initialize it once during VM initialization, then use it when creating mutexes. Even then I would structure this so that the use of the attr object is hidden behind a macro so that it is a no-op on platforms that use NORMAL as the default. I will file a RFE for sharing POSIX-based PlatformEvent/Parker code and include the mutex type issue as part of that. Thanks, David H. ------- On 10/01/2017 3:37 AM, David CARLIER wrote: > Hi sure thing > > diff --git a/src/os/bsd/vm/os_bsd.hpp b/src/os/bsd/vm/os_bsd.hpp > --- a/src/os/bsd/vm/os_bsd.hpp > +++ b/src/os/bsd/vm/os_bsd.hpp > @@ -175,6 +175,7 @@ > volatile int _Event; > volatile int _nParked; > pthread_mutex_t _mutex[1]; > + pthread_mutexattr_t _attr[1]; > pthread_cond_t _cond[1]; > double PostPad[2]; > Thread * _Assoc; > @@ -187,7 +188,11 @@ > int status; > status = pthread_cond_init(_cond, NULL); > assert_status(status == 0, status, "cond_init"); > - status = pthread_mutex_init(_mutex, NULL); > + status = pthread_mutexattr_init(_attr); > + assert_status(status == 0, status, "attr_init"); > + status = pthread_mutexattr_settype(_attr, PTHREAD_MUTEX_NORMAL); > + assert_status(status == 0, status, "attr_settype"); > + status = pthread_mutex_init(_mutex, _attr); > assert_status(status == 0, status, "mutex_init"); > _Event = 0; > _nParked = 0; > @@ -206,6 +211,7 @@ > class PlatformParker : public CHeapObj { > protected: > pthread_mutex_t _mutex[1]; > + pthread_mutexattr_t _attr[1]; > pthread_cond_t _cond[1]; > > public: // TODO-FIXME: make dtor private > @@ -216,7 +222,11 @@ > int status; > status = pthread_cond_init(_cond, NULL); > assert_status(status == 0, status, "cond_init"); > - status = pthread_mutex_init(_mutex, NULL); > + status = pthread_mutexattr_init(_attr); > + assert_status(status == 0, status, "attr_init"); > + status = pthread_mutexattr_settype(_attr, PTHREAD_MUTEX_NORMAL); > + assert_status(status == 0, status, "attr_settype"); > + status = pthread_mutex_init(_mutex, _attr); > assert_status(status == 0, status, "mutex_init"); > } > }; > > > > Kind regards. > > Thanks. > > On 9 January 2017 at 16:48, Daniel D. Daugherty > wrote: >> David C., >> >> If you sent the patch as an attachment, then you should be advised that >> the OpenJDK email servers strip attachments. If the patch is relatively >> small, then you can send it in-line... >> >> Dan >> >> >> >> On 1/6/17 3:23 PM, David CARLIER wrote: >>> >>> Thanks, here a patch proposal. Kindest regards. >>> >>> On 24 December 2016 at 02:16, David Holmes >>> wrote: >>>> >>>> If there is a platform, we support, where the default settings are >>>> inappropriate then setting them explicitly would be the right thing to >>>> do. >>>> >>>> David >>>> >>>> >>>> On 24/12/2016 10:23 AM, Vladimir Kozlov wrote: >>>>> >>>>> Hi, >>>>> >>>>> Forwarding to mailing lists since I can't answer. >>>>> May be someone can answer your question on these lists. >>>>> >>>>> Regards, >>>>> Vladimir >>>>> >>>>> On 12/23/16 2:17 AM, David CARLIER wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> this is my first mail to the maiing list, I have difficulties to push >>>>>> messages to any mailing lists, but as BSD user/dev I was >>>>>> looking at code's part like this one >>>>>> >>>>>> >>>>>> >>>>>> http://hg.openjdk.java.net/jdk9/dev/hotspot/file/70c6fae64754/src/os/bsd/vm/os_bsd.hpp >>>>>> >>>>>> >>>>>> I was wondering if it would be better to explicitally set the attr >>>>>> type to PTHREAD_MUTEX_NORMAL to insure consistency with AIX and Linux >>>>>> (on FreeBSD it is PTHREAD_MUTEX_ERRORCHECK by default). >>>>>> >>>>>> What do you think ? >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>>> Kindest regards. >>>>>> >> From kim.barrett at oracle.com Wed Feb 8 03:02:31 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 7 Feb 2017 22:02:31 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <08918c3d-395c-2e29-36e5-d06128f45122@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <08918c3d-395c-2e29-36e5-d06128f45122@oracle.com> Message-ID: > On Feb 7, 2017, at 6:56 PM, dean.long at oracle.com wrote: > > Hi Kim. I took a look at the sharedRuntime_ and templateInterpreterGenerator_ for arm, sparc, and x86. > > In sharedRuntime_arm.cpp, I suggest simplifying: > > 1750 #ifdef AARCH64 > 1751 __ cbz(R0, done); // Use NULL as-is. > 1752 STATIC_ASSERT(JNIHandles::weak_tag_mask == 1u); > 1753 __ tbz(R0, 0, not_weak); // Test for jweak tag. > 1754 #else // !AARCH64 > > 1755 __ cmp(R0, 0); > > 1756 __ b(done, eq); // Use NULL as-is. > 1757 __ tst(R0, JNIHandles::weak_tag_mask); // Test for jweak tag. > 1758 __ b(not_weak, eq); > 1759 #endif // !AARCH64 > to the equivalent: > > 1751 __ cbz(R0, done); // Use NULL as-is. > 1752 STATIC_ASSERT(JNIHandles::weak_tag_mask == 1u); > 1753 __ tbz(R0, 0, not_weak); // Test for jweak tag. > as cbz and tbz expand to cmp/b and tst/b on arm32. The others look good. Nice! That will make it much more readable. Thanks. From per.liden at oracle.com Wed Feb 8 08:40:24 2017 From: per.liden at oracle.com (Per Liden) Date: Wed, 8 Feb 2017 09:40:24 +0100 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: Hi Kim, On 2017-02-07 02:03, Kim Barrett wrote: > Apologies for taking so long to respond to earlier comments. I've > been distracted by some other tasks, and it took a while to figure out > what to do about the problem Mikael reported. > > Martin Doerr - Thanks for the ppc and s390 native call generator > modifications. Do you want a contributed-by credit when pushed? > > Andrew Haley - Thanks for the syntax fix. The new version with always > tagged jweaks (see below) makes testing easier, and there's a new test > included. Testing the G1-specific behavior still needs the whitebox > extension. I've updated the G1-specific test (which is still not part > of the changeset for JDK 9). > > Per Liden - I tried your suggestion of making jweak unconditionally > tagged. It doesn't eliminate much of the INCLUDE_ALL_GCS > conditionals, since those are still needed to protect references to > G1-specific code. (The littering of otherwise generic runtime code > with this G1-specific stuff should be a cleanup task for JDK 10.) > However, it does simplify some of the code, and simplifies testing. > The cost is a "bit-test and (predicted taken) conditional branch" in > the handling of an oop result from a native call. I agree that's > unlikely to be noticable, and the benefits seem worthwhile, so I've > made that change. Thanks for doing that. > > I've attempted to update the platform-specific native call return > handling changes for all of the platforms, including those I don't > have access to (aarch64, ppc, s390). I also fixed the zero change I'd > made earlier. And I made shark error, to force anyone using it to fix > the handling of an oop result by SharkNativeWrapper::initialize(). > There might be other zero/shark changes that I've missed. All of > these should be checked by appropriate platform experts. > > I had to fix a JVMTI bug as part of this; FollowReferences wasn't > doing as much argument checking as expected. This was unconvered by a > closed test that unexpectedly asserted. This test would have > similarly asserted previously if CheckJNICalls was true; I removed > that conditionalization in JNIHandles::resolve, as it was only there > to avoid an expensive check for the jobject being a jweak, and in the > new code we already know it isn't. There might be other JVMTI > functions with similar issues. > > Mikael - There are some ugly worms lurking under that rock! I think > I've dealt with the problem, tripping over some other bugs along the > way. Thanks for the test case. I tweaked it a bit and included it in > the updated changeset. > > The passing of arguments involves collecting them into a > JavaCallArguments object. oop arguments are in some handlized form > (either jobject or Handle). The existing code heavily cheats; for > example, it assumes one can correctly cast from a jobject to an oop* > and dereference that pointer. Obviously, a tagged jweak breaks that > assumption. Even worse, there are casts from oop* to oop and back! > (These are so nasty they *must* use C-style cast syntax.) I'm thinking > about some followup cleanup bugs in that area. > > To address this, I changed JavaCallArguments to avoid conversions > from jobject to Handle/oop*. To support that, it now keeps more > accurate track of the state of the object references in the collected > arguments. It also delays resolving jobjects to their designated > oops, avoiding intermediate Handle objects. (This was a place where > cheating was going on, in order to avoid intermediate Handle > allocations. We're now accomplishing that without crashing through > the JNIHandles abstraction.) That looks really fragile, thanks for trying to clean that up. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 Looks good to me. cheers, Per > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.inc > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > Some additional webrevs that might be helpful: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.other-platforms > Incremental changes for aarch64, ppc, and s390 supplied by Andrew and > Martin. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.always_tag[.full] > Incremental [full] changes to always tag jweaks. > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.java_call > Incremental changes to address the java call problem found by Mikael. > > hotspot.04 == hotspot.02 + hotspot.04.inc > hotspot.04 == hotspot.04.always_tag.full + hotspot.04.java_call > hotspot.04 == hotspot.02 + hotspot.04.other-platforms + > hotspot.04.always_tag + hotspot.04.java_call > > The updated G1-specific test requiring the whitebox extension is > http://cr.openjdk.java.net/~kbarrett/8166188/test.04 > > The whitebox extension has not changed: > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ > > From aph at redhat.com Wed Feb 8 09:56:57 2017 From: aph at redhat.com (Andrew Haley) Date: Wed, 8 Feb 2017 09:56:57 +0000 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: On 08/02/17 08:40, Per Liden wrote: > Some additional webrevs that might be helpful: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.other-platforms > Incremental changes for aarch64, ppc, and s390 supplied by Andrew and > Martin. OK, thanks. Andrew. From kim.barrett at oracle.com Wed Feb 8 16:55:41 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 8 Feb 2017 11:55:41 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: <89366917-2453-41AD-914F-4C426CC42AC6@oracle.com> > On Feb 8, 2017, at 3:40 AM, Per Liden wrote: > On 2017-02-07 02:03, Kim Barrett wrote: >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 > > Looks good to me. > > cheers, > Per Thanks Per. From bob.vandette at oracle.com Wed Feb 8 19:29:19 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 8 Feb 2017 14:29:19 -0500 Subject: RFR: JDK-8172670: AOT Platform Support for Windows and Mac OS X x64 (Linux and Solaris too) Message-ID: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> SUMMARY: Please review the following set of changes that adds Ahead-of-time compilation support for Mac OSX and Windows x64 in JDK 10. Linux and Solaris x64 AOT support has also been updated to be consistent with the new 100% Java based binary container support included in this changeset. This change also removes the build and runtime dependency on the external libelf library and header files. Once this change is integrated Ahead of time support will be available for the following set of Platforms: - Linux x64 - Windows x64 - MacOSX x64 - Solaris x64 RFE: https://bugs.openjdk.java.net/browse/JDK-8172670 WEBREVS: TOP LEVEL ????? http://cr.openjdk.java.net/~bobv/8172670/webrev.01/ JDK ?? http://cr.openjdk.java.net/~bobv/8172670/jdk/webrev.01/ HOTSPOT ????? Full Hotspot Webrev: http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev.01/ If you?d prefer to review things in smaller chunks, here are the hotspot changes for Linux and Solaris including the libelf removal: http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-linux.01/ Files added to support Mac OSX: http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-mac.01/ Files added to provide Windows support: http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-win.01/ Changes to the jtreg tests for AOT: http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-test.01/ TESTING: 1. JPRT has been run and passes with these changes 2. jtreg AOT tests pass on all supported platforms 3. AOT compiling of java.base completes and can run basic Java programs on all supported platforms NOTE: 1. Although these test run correctly on Windows, jtreg AOT tests have been temporarily disabled for this platform. This is due to an internal JPRT system configuration issue which will hopefully get resolved soon. AOT requires access to a local toolchain and our JPRT systems do not regularly install Visual Studio. Thanks, Bob. From daniel.daugherty at oracle.com Wed Feb 8 22:13:20 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 8 Feb 2017 15:13:20 -0700 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 make/test/JtregNative.gmk No comments. src/share/vm/runtime/javaCalls.hpp No comments. src/share/vm/runtime/javaCalls.cpp L526: "signature does not matched pushed arguments: %u", state); Typo: 'matched' -> 'match' For the guarantees on L522 and L525, would the (_pos - 1) value be useful for diagnostic purposes? L569: if (v != 0 ) { Not your bug, but there's an extra blank before the ")". L571: if (t < (size_t)os::vm_page_size()) { Again, not your issue, but that size < page size check means the oop is bad is screaming for a short comment. src/share/vm/runtime/jniHandles.hpp (old) L208: *((oop*)handle) = deleted_handle(); // Mark the handle as deleted, allocate will reuse it The old comment was too obvious so you didn't keep it? :-) It threw me for a second when JNIHandles::resolve_non_null() was switched to use resolve_impl() instead of guard_value(), then I figured out that resolve_impl() uses guard_value(). src/share/vm/runtime/jniHandles.cpp (old) L115 assert(!CheckJNICalls || is_weak_global_handle(handle), "Invalid delete of weak global JNI handle"); Don't like the old assert? Is is_weak_global_handle() now obsolete? src/share/vm/prims/jni.cpp No comments. src/share/vm/prims/jvmtiEnv.cpp Need a copyright year update in this file. src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp No comments. src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp No comments. src/cpu/arm/vm/interp_masm_arm.cpp No comments on the code deletion. src/cpu/arm/vm/interp_masm_arm.hpp No comments on the code deletion. src/cpu/arm/vm/macroAssembler_arm.cpp OK, the code from interp_masm_arm.cpp moved here. Did not try to compare the two versions. src/cpu/arm/vm/macroAssembler_arm.hpp OK, the code from interp_masm_arm.hpp moved here. Did not try to compare the two versions. src/cpu/arm/vm/sharedRuntime_arm.cpp No comments. src/cpu/arm/vm/templateInterpreterGenerator_arm.cpp No comments. src/cpu/ppc/vm/frame_ppc.cpp No comments. src/cpu/ppc/vm/sharedRuntime_ppc.cpp No comments. src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp No comments. src/cpu/s390/vm/sharedRuntime_s390.cpp No comments. src/cpu/s390/vm/templateInterpreterGenerator_s390.cpp (old) L1705: __ verify_oop(Rlresult); The delete of the verify_oop() call after the store_oop_result label threw me for a few minutes. Then I realized that the only uses of the store_oop_result label appear to be when we have a NULL oop. New code is cleaner. src/cpu/sparc/vm/sharedRuntime_sparc.cpp No comments. src/cpu/sparc/vm/templateInterpreterGenerator_sparc.cpp Not your bug, but unlike sharedRuntime_sparc.cpp, there are no verify_oop calls here. src/cpu/x86/vm/sharedRuntime_x86_32.cpp No comments. src/cpu/x86/vm/sharedRuntime_x86_64.cpp No comments. src/cpu/x86/vm/templateInterpreterGenerator_x86.cpp Not your bug, but unlike sharedRuntime_x86{32,64}.cpp, there are no verify_oop calls here. src/cpu/zero/vm/cppInterpreter_zero.cpp No comments. src/share/vm/shark/sharkNativeWrapper.cpp No comments. test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java jcheck will likely complain about the empty line at the end of the file. test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c Same empty line at the end of the file comment. test/runtime/jni/ReturnJNIWeak/ReturnJNIWeak.java No comments. test/runtime/jni/ReturnJNIWeak/libReturnJNIWeak.c Nit - libCallWithJNIWeak.c uses an indent of 4 spaces; this file uses an indent of 2 spaces. Should probably pick one... Thumbs up! If you choose to fix any of the nits above, I don't need to see another webrev. Dan On 2/6/17 6:03 PM, Kim Barrett wrote: > Apologies for taking so long to respond to earlier comments. I've > been distracted by some other tasks, and it took a while to figure out > what to do about the problem Mikael reported. > > Martin Doerr - Thanks for the ppc and s390 native call generator > modifications. Do you want a contributed-by credit when pushed? > > Andrew Haley - Thanks for the syntax fix. The new version with always > tagged jweaks (see below) makes testing easier, and there's a new test > included. Testing the G1-specific behavior still needs the whitebox > extension. I've updated the G1-specific test (which is still not part > of the changeset for JDK 9). > > Per Liden - I tried your suggestion of making jweak unconditionally > tagged. It doesn't eliminate much of the INCLUDE_ALL_GCS > conditionals, since those are still needed to protect references to > G1-specific code. (The littering of otherwise generic runtime code > with this G1-specific stuff should be a cleanup task for JDK 10.) > However, it does simplify some of the code, and simplifies testing. > The cost is a "bit-test and (predicted taken) conditional branch" in > the handling of an oop result from a native call. I agree that's > unlikely to be noticable, and the benefits seem worthwhile, so I've > made that change. > > I've attempted to update the platform-specific native call return > handling changes for all of the platforms, including those I don't > have access to (aarch64, ppc, s390). I also fixed the zero change I'd > made earlier. And I made shark error, to force anyone using it to fix > the handling of an oop result by SharkNativeWrapper::initialize(). > There might be other zero/shark changes that I've missed. All of > these should be checked by appropriate platform experts. > > I had to fix a JVMTI bug as part of this; FollowReferences wasn't > doing as much argument checking as expected. This was unconvered by a > closed test that unexpectedly asserted. This test would have > similarly asserted previously if CheckJNICalls was true; I removed > that conditionalization in JNIHandles::resolve, as it was only there > to avoid an expensive check for the jobject being a jweak, and in the > new code we already know it isn't. There might be other JVMTI > functions with similar issues. > > Mikael - There are some ugly worms lurking under that rock! I think > I've dealt with the problem, tripping over some other bugs along the > way. Thanks for the test case. I tweaked it a bit and included it in > the updated changeset. > > The passing of arguments involves collecting them into a > JavaCallArguments object. oop arguments are in some handlized form > (either jobject or Handle). The existing code heavily cheats; for > example, it assumes one can correctly cast from a jobject to an oop* > and dereference that pointer. Obviously, a tagged jweak breaks that > assumption. Even worse, there are casts from oop* to oop and back! > (These are so nasty they *must* use C-style cast syntax.) I'm thinking > about some followup cleanup bugs in that area. > > To address this, I changed JavaCallArguments to avoid conversions > from jobject to Handle/oop*. To support that, it now keeps more > accurate track of the state of the object references in the collected > arguments. It also delays resolving jobjects to their designated > oops, avoiding intermediate Handle objects. (This was a place where > cheating was going on, in order to avoid intermediate Handle > allocations. We're now accomplishing that without crashing through > the JNIHandles abstraction.) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.inc > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > Some additional webrevs that might be helpful: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.other-platforms > Incremental changes for aarch64, ppc, and s390 supplied by Andrew and > Martin. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.always_tag[.full] > Incremental [full] changes to always tag jweaks. > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.java_call > Incremental changes to address the java call problem found by Mikael. > > hotspot.04 == hotspot.02 + hotspot.04.inc > hotspot.04 == hotspot.04.always_tag.full + hotspot.04.java_call > hotspot.04 == hotspot.02 + hotspot.04.other-platforms + > hotspot.04.always_tag + hotspot.04.java_call > > The updated G1-specific test requiring the whitebox extension is > http://cr.openjdk.java.net/~kbarrett/8166188/test.04 > > The whitebox extension has not changed: > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ > > > From kim.barrett at oracle.com Thu Feb 9 00:00:44 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 8 Feb 2017 19:00:44 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> On Feb 8, 2017, at 5:13 PM, Daniel D. Daugherty wrote: > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 > src/share/vm/runtime/javaCalls.cpp > L526: "signature does not matched pushed arguments: %u", state); > Typo: 'matched' -> 'match' > > For the guarantees on L522 and L525, would the (_pos - 1) value > be useful for diagnostic purposes? Done. > L569: if (v != 0 ) { > Not your bug, but there's an extra blank before the ")". Done. > L571: if (t < (size_t)os::vm_page_size()) { > Again, not your issue, but that size < page size check means > the oop is bad is screaming for a short comment. Added comment. Also removed the unnecessary intermediate variable t, thereby eliminating some casts. I'm not completly convinced by the comment, but remaining true to the existing code. Here's the diff: @@ -566,9 +568,10 @@ uint value_state = _value_state[p]; if (is_value_state_indirect_oop(value_state)) { intptr_t v = _value[p]; - if (v != 0 ) { - size_t t = (size_t)v; - if (t < (size_t)os::vm_page_size()) { + if (v != 0) { + if (v < os::vm_page_size()) { + // v is a "handle" referring to an oop, cast to integral type. + // There ought not be any handles in very low memory. bad = true; } else if (!resolve_indirect_oop(v, value_state)->is_oop_or_null(true)) { bad = true; > src/share/vm/runtime/jniHandles.cpp > (old) L115 assert(!CheckJNICalls || is_weak_global_handle(handle), "Invalid delete of weak global JNI handle"); > > Don't like the old assert? Is is_weak_global_handle() now obsolete? There is an is_jweak assert in jweak_ref. That's cheap (unlike is_weak_global_handle), so not conditional on CheckJNICalls. There are still a few calls to is_weak_global_handle. They all occur as alternatives with is_global_handle &etc, so I was reluctant to replace them with is_jweak (which would also require making that public). > src/cpu/s390/vm/templateInterpreterGenerator_s390.cpp > (old) L1705: __ verify_oop(Rlresult); > The delete of the verify_oop() call after the store_oop_result > label threw me for a few minutes. Then I realized that the > only uses of the store_oop_result label appear to be when we > have a NULL oop. New code is cleaner. There's another, after the G1 block. But it's preceeded by a verify_oop of its own. > src/cpu/sparc/vm/templateInterpreterGenerator_sparc.cpp > Not your bug, but unlike sharedRuntime_sparc.cpp, there are no > verify_oop calls here. > > src/cpu/x86/vm/templateInterpreterGenerator_x86.cpp > Not your bug, but unlike sharedRuntime_x86{32,64}.cpp, > there are no verify_oop calls here. I didn't add them where they weren't already present. I could be convinced to change that. > test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java > jcheck will likely complain about the empty line at the end > of the file. > > test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c > Same empty line at the end of the file comment. Okay. I don't use jcheck until preparing to push, because it doesn't understand mq patches. > Thumbs up! If you choose to fix any of the nits above, I don't > need to see another webrev. Thanks, Dan. From vladimir.kozlov at oracle.com Thu Feb 9 03:04:51 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 8 Feb 2017 19:04:51 -0800 Subject: RFR: JDK-8172670: AOT Platform Support for Windows and Mac OS X x64 (Linux and Solaris too) In-Reply-To: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> References: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> Message-ID: On 2/8/17 11:29 AM, Bob Vandette wrote: > SUMMARY: > > Please review the following set of changes that adds Ahead-of-time compilation support for Mac OSX and > Windows x64 in JDK 10. Linux and Solaris x64 AOT support has also been updated to be consistent with > the new 100% Java based binary container support included in this changeset. > > This change also removes the build and runtime dependency on the external libelf library and header files. > > Once this change is integrated Ahead of time support will be available for the following set of Platforms: > > - Linux x64 > - Windows x64 > - MacOSX x64 > - Solaris x64 > > RFE: > > https://bugs.openjdk.java.net/browse/JDK-8172670 > > WEBREVS: > > TOP LEVEL > ????? > > http://cr.openjdk.java.net/~bobv/8172670/webrev.01/ Good. > > JDK > ?? > > http://cr.openjdk.java.net/~bobv/8172670/jdk/webrev.01/ Looks good to me but need to be reviewed by build jigsaw team. > > HOTSPOT > ????? > > Full Hotspot Webrev: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev.01/ Both AOT tool and JVM changes looks good to me. Thanks, Vladimir > > > If you?d prefer to review things in smaller chunks, here are the hotspot changes for Linux and > Solaris including the libelf removal: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-linux.01/ > > Files added to support Mac OSX: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-mac.01/ > > Files added to provide Windows support: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-win.01/ > > Changes to the jtreg tests for AOT: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-test.01/ > > > TESTING: > > 1. JPRT has been run and passes with these changes > 2. jtreg AOT tests pass on all supported platforms > 3. AOT compiling of java.base completes and can run basic Java programs on all supported platforms > > NOTE: > > 1. Although these test run correctly on Windows, jtreg AOT tests have been temporarily disabled for this platform. > This is due to an internal JPRT system configuration issue which will hopefully get resolved soon. AOT requires > access to a local toolchain and our JPRT systems do not regularly install Visual Studio. > > > Thanks, > Bob. > From david.holmes at oracle.com Thu Feb 9 03:18:31 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Feb 2017 13:18:31 +1000 Subject: Hotspot for BSD / opinion request In-Reply-To: <64159c85-d58d-64b0-6250-8b0f3f052a5b@oracle.com> References: <1488ccc1-e00c-42e7-3cfc-9fd0b5699a63@oracle.com> <3c580abc-9333-8803-abc1-1b0863123060@oracle.com> <64159c85-d58d-64b0-6250-8b0f3f052a5b@oracle.com> Message-ID: <5affb3f8-f5d0-accd-68e2-a3c2e057af2f@oracle.com> Filed: https://bugs.openjdk.java.net/browse/JDK-8174231 Thanks, David H. On 8/02/2017 12:32 PM, David Holmes wrote: > Hi David, > > Sorry for the long lag on this. I've been looking into the mutex > attribute defaults and it is unfortunate that POSIX allowed > PTHREAD_MUTEX_DEFAULT to be an implementation specific alias for one of > the other three mutex types: PTHREAD_MUTEX_NORMAL, > PTHREAD_MUTEX_ERRORCHECK, PTHREAD_MUTEX_RECURSIVE. But even more > unfortunate that FreeBSD decided to map it to ERRORCHECK instead of > NORMAL as every other platform seems to do! (I note NetBSD defines a > DEFAULT that is compatible with NORMAL, not ERRORCHECK.) > > Fixing this for FreeBSD when it is not an officially supported platform > is a little messy because we don't want to penalize all the other > platforms with extra initialization code. Right now this could be > confined to os_bsd.* but this code should be refactored into os_posix.* > to be more widely shared. > > That aside we do not need to keep an attr object per mutex as they will > all share the same attributes. So we would have a single static instance > and initialize it once during VM initialization, then use it when > creating mutexes. Even then I would structure this so that the use of > the attr object is hidden behind a macro so that it is a no-op on > platforms that use NORMAL as the default. > > I will file a RFE for sharing POSIX-based PlatformEvent/Parker code and > include the mutex type issue as part of that. > > Thanks, > David H. > ------- > > > On 10/01/2017 3:37 AM, David CARLIER wrote: >> Hi sure thing >> >> diff --git a/src/os/bsd/vm/os_bsd.hpp b/src/os/bsd/vm/os_bsd.hpp >> --- a/src/os/bsd/vm/os_bsd.hpp >> +++ b/src/os/bsd/vm/os_bsd.hpp >> @@ -175,6 +175,7 @@ >> volatile int _Event; >> volatile int _nParked; >> pthread_mutex_t _mutex[1]; >> + pthread_mutexattr_t _attr[1]; >> pthread_cond_t _cond[1]; >> double PostPad[2]; >> Thread * _Assoc; >> @@ -187,7 +188,11 @@ >> int status; >> status = pthread_cond_init(_cond, NULL); >> assert_status(status == 0, status, "cond_init"); >> - status = pthread_mutex_init(_mutex, NULL); >> + status = pthread_mutexattr_init(_attr); >> + assert_status(status == 0, status, "attr_init"); >> + status = pthread_mutexattr_settype(_attr, PTHREAD_MUTEX_NORMAL); >> + assert_status(status == 0, status, "attr_settype"); >> + status = pthread_mutex_init(_mutex, _attr); >> assert_status(status == 0, status, "mutex_init"); >> _Event = 0; >> _nParked = 0; >> @@ -206,6 +211,7 @@ >> class PlatformParker : public CHeapObj { >> protected: >> pthread_mutex_t _mutex[1]; >> + pthread_mutexattr_t _attr[1]; >> pthread_cond_t _cond[1]; >> >> public: // TODO-FIXME: make dtor private >> @@ -216,7 +222,11 @@ >> int status; >> status = pthread_cond_init(_cond, NULL); >> assert_status(status == 0, status, "cond_init"); >> - status = pthread_mutex_init(_mutex, NULL); >> + status = pthread_mutexattr_init(_attr); >> + assert_status(status == 0, status, "attr_init"); >> + status = pthread_mutexattr_settype(_attr, PTHREAD_MUTEX_NORMAL); >> + assert_status(status == 0, status, "attr_settype"); >> + status = pthread_mutex_init(_mutex, _attr); >> assert_status(status == 0, status, "mutex_init"); >> } >> }; >> >> >> >> Kind regards. >> >> Thanks. >> >> On 9 January 2017 at 16:48, Daniel D. Daugherty >> wrote: >>> David C., >>> >>> If you sent the patch as an attachment, then you should be advised that >>> the OpenJDK email servers strip attachments. If the patch is relatively >>> small, then you can send it in-line... >>> >>> Dan >>> >>> >>> >>> On 1/6/17 3:23 PM, David CARLIER wrote: >>>> >>>> Thanks, here a patch proposal. Kindest regards. >>>> >>>> On 24 December 2016 at 02:16, David Holmes >>>> wrote: >>>>> >>>>> If there is a platform, we support, where the default settings are >>>>> inappropriate then setting them explicitly would be the right thing to >>>>> do. >>>>> >>>>> David >>>>> >>>>> >>>>> On 24/12/2016 10:23 AM, Vladimir Kozlov wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> Forwarding to mailing lists since I can't answer. >>>>>> May be someone can answer your question on these lists. >>>>>> >>>>>> Regards, >>>>>> Vladimir >>>>>> >>>>>> On 12/23/16 2:17 AM, David CARLIER wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> this is my first mail to the maiing list, I have difficulties to >>>>>>> push >>>>>>> messages to any mailing lists, but as BSD user/dev I was >>>>>>> looking at code's part like this one >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk9/dev/hotspot/file/70c6fae64754/src/os/bsd/vm/os_bsd.hpp >>>>>>> >>>>>>> >>>>>>> >>>>>>> I was wondering if it would be better to explicitally set the attr >>>>>>> type to PTHREAD_MUTEX_NORMAL to insure consistency with AIX and >>>>>>> Linux >>>>>>> (on FreeBSD it is PTHREAD_MUTEX_ERRORCHECK by default). >>>>>>> >>>>>>> What do you think ? >>>>>>> >>>>>>> Thanks in advance. >>>>>>> >>>>>>> Kindest regards. >>>>>>> >>> From mandy.chung at oracle.com Thu Feb 9 03:36:28 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 8 Feb 2017 19:36:28 -0800 Subject: RFR: JDK-8172670: AOT Platform Support for Windows and Mac OS X x64 (Linux and Solaris too) In-Reply-To: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> References: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> Message-ID: <95FC69EC-2F6C-4816-9B5C-BA6D99F2A67B@oracle.com> > On Feb 8, 2017, at 11:29 AM, Bob Vandette wrote: > > JDK > ?? > > http://cr.openjdk.java.net/~bobv/8172670/jdk/webrev.01/ > The jdk change looks fine. If any change to module-info.java.extra in JDK 9 for jdk.vm.compiler, we should make sure the same change applies to windows version. Mandy From erik.joelsson at oracle.com Thu Feb 9 08:39:04 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 9 Feb 2017 09:39:04 +0100 Subject: RFR: JDK-8172670: AOT Platform Support for Windows and Mac OS X x64 (Linux and Solaris too) In-Reply-To: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> References: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> Message-ID: <4d383bab-959e-f789-53d3-61f3bf40ae08@oracle.com> Hello, Looks good, just one minor comment below. On 2017-02-08 20:29, Bob Vandette wrote: > SUMMARY: > > Please review the following set of changes that adds Ahead-of-time compilation support for Mac OSX and > Windows x64 in JDK 10. Linux and Solaris x64 AOT support has also been updated to be consistent with > the new 100% Java based binary container support included in this changeset. > > This change also removes the build and runtime dependency on the external libelf library and header files. > > Once this change is integrated Ahead of time support will be available for the following set of Platforms: > > - Linux x64 > - Windows x64 > - MacOSX x64 > - Solaris x64 > > RFE: > > https://bugs.openjdk.java.net/browse/JDK-8172670 > > WEBREVS: > > TOP LEVEL > ????? > > http://cr.openjdk.java.net/~bobv/8172670/webrev.01/ hotspot.m4: The variable OPENJDK_TARGET_CPU can never have the value "amd64" so no need to test for it. (if it does, it's a bug) > > JDK > ?? > > http://cr.openjdk.java.net/~bobv/8172670/jdk/webrev.01/ > > HOTSPOT > ????? > > Full Hotspot Webrev: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev.01/ > > > If you?d prefer to review things in smaller chunks, here are the hotspot changes for Linux and > Solaris including the libelf removal: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-linux.01/ > > Files added to support Mac OSX: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-mac.01/ > > Files added to provide Windows support: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-win.01/ > > Changes to the jtreg tests for AOT: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-test.01/ > > > TESTING: > > 1. JPRT has been run and passes with these changes > 2. jtreg AOT tests pass on all supported platforms > 3. AOT compiling of java.base completes and can run basic Java programs on all supported platforms > > NOTE: > > 1. Although these test run correctly on Windows, jtreg AOT tests have been temporarily disabled for this platform. > This is due to an internal JPRT system configuration issue which will hopefully get resolved soon. AOT requires > access to a local toolchain and our JPRT systems do not regularly install Visual Studio. We need to discuss this, will send a separate mail. /Erik > > Thanks, > Bob. From rwestrel at redhat.com Thu Feb 9 16:26:23 2017 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 09 Feb 2017 17:26:23 +0100 Subject: Result: New hotspot Group Member: Aleksey Shipilev Message-ID: The vote for Aleksey Shipilev [1] is now closed. Yes: 12 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Roland. [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-January/025709.html From coleen.phillimore at oracle.com Fri Feb 10 02:34:23 2017 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 9 Feb 2017 21:34:23 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/cpu/x86/vm/sharedRuntime_x86_32.cpp.udiff.html http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/cpu/x86/vm/sharedRuntime_x86_64.cpp.udiff.html http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/cpu/x86/vm/templateInterpreterGenerator_x86.cpp.udiff.html The code to unpack the oop result is the same at least for the x86 implementations, which unfortunately still have 32/64 bit variants for sharedRuntime_x86.cpp. So in the x86 case, there are 3. I think these should be refactored into a function in macroAssembler_x86.cpp, something like unpack_native_oop_result(). It's not technically unboxing because boxing refers to making primitives into java.lang.Integer, etc objects. Sorry that this might be more work and tricky for the other platforms because I didn't diff them, but it would be a lot better. Maintaining separate copies of assembly code is a lot of work too. You can file an RFE/bug to clean this up if there was not a reason for the duplication because of ZBB. http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/share/vm/prims/jni.cpp.udiff.html Can you change 'l' to obj? never mind, I see why you picked 'l' but it looks like a 1. http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/share/vm/runtime/javaCalls.hpp.udiff.html I see what you've done. These are terrible casts. + JNITypes::put_obj((oop)handle, _value, size); // Updates size. I think as a future cleanup, we should make JNITypes::put_obj take (intptr_t) or (jobject,Handle) instead of dealing with the oop class for CheckUnhandledOops. I didn't see any other callers except this one. Why is value_state uint when it's being stored into uchar? + static const uint value_state_primitive = 0; Could they be enums near where _value_state_buffer is declared? http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/share/vm/runtime/jniHandles.hpp.udiff.html I don't think it's good to have to include +#include "gc/g1/g1SATBCardTableModRefBS.hpp" in jniHandles.hpp header file. could you refactor resolve_impl to put the jweak case in the .cpp file? I don't think performance is going to be important for jweak. All this work for an indirection. With the template code, it's hard to see that's all it is. :( http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/test/runtime/jni/ReturnJNIWeak/ReturnJNIWeak.java.html Someone else asked this question but how can you assume that GC cleared the weak reference? 115 System.gc(); 116 // Verify weak ref cleared as expected. That's all. The change looks good. Thanks, Coleen On 2/6/17 8:03 PM, Kim Barrett wrote: > Apologies for taking so long to respond to earlier comments. I've > been distracted by some other tasks, and it took a while to figure out > what to do about the problem Mikael reported. > > Martin Doerr - Thanks for the ppc and s390 native call generator > modifications. Do you want a contributed-by credit when pushed? > > Andrew Haley - Thanks for the syntax fix. The new version with always > tagged jweaks (see below) makes testing easier, and there's a new test > included. Testing the G1-specific behavior still needs the whitebox > extension. I've updated the G1-specific test (which is still not part > of the changeset for JDK 9). > > Per Liden - I tried your suggestion of making jweak unconditionally > tagged. It doesn't eliminate much of the INCLUDE_ALL_GCS > conditionals, since those are still needed to protect references to > G1-specific code. (The littering of otherwise generic runtime code > with this G1-specific stuff should be a cleanup task for JDK 10.) > However, it does simplify some of the code, and simplifies testing. > The cost is a "bit-test and (predicted taken) conditional branch" in > the handling of an oop result from a native call. I agree that's > unlikely to be noticable, and the benefits seem worthwhile, so I've > made that change. > > I've attempted to update the platform-specific native call return > handling changes for all of the platforms, including those I don't > have access to (aarch64, ppc, s390). I also fixed the zero change I'd > made earlier. And I made shark error, to force anyone using it to fix > the handling of an oop result by SharkNativeWrapper::initialize(). > There might be other zero/shark changes that I've missed. All of > these should be checked by appropriate platform experts. > > I had to fix a JVMTI bug as part of this; FollowReferences wasn't > doing as much argument checking as expected. This was unconvered by a > closed test that unexpectedly asserted. This test would have > similarly asserted previously if CheckJNICalls was true; I removed > that conditionalization in JNIHandles::resolve, as it was only there > to avoid an expensive check for the jobject being a jweak, and in the > new code we already know it isn't. There might be other JVMTI > functions with similar issues. > > Mikael - There are some ugly worms lurking under that rock! I think > I've dealt with the problem, tripping over some other bugs along the > way. Thanks for the test case. I tweaked it a bit and included it in > the updated changeset. > > The passing of arguments involves collecting them into a > JavaCallArguments object. oop arguments are in some handlized form > (either jobject or Handle). The existing code heavily cheats; for > example, it assumes one can correctly cast from a jobject to an oop* > and dereference that pointer. Obviously, a tagged jweak breaks that > assumption. Even worse, there are casts from oop* to oop and back! > (These are so nasty they *must* use C-style cast syntax.) I'm thinking > about some followup cleanup bugs in that area. > > To address this, I changed JavaCallArguments to avoid conversions > from jobject to Handle/oop*. To support that, it now keeps more > accurate track of the state of the object references in the collected > arguments. It also delays resolving jobjects to their designated > oops, avoiding intermediate Handle objects. (This was a place where > cheating was going on, in order to avoid intermediate Handle > allocations. We're now accomplishing that without crashing through > the JNIHandles abstraction.) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.inc > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > Some additional webrevs that might be helpful: > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.other-platforms > Incremental changes for aarch64, ppc, and s390 supplied by Andrew and > Martin. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.always_tag[.full] > Incremental [full] changes to always tag jweaks. > The full webrev may be more useful than the incremental; some of the > incremental changes are pretty hard to read. > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04.java_call > Incremental changes to address the java call problem found by Mikael. > > hotspot.04 == hotspot.02 + hotspot.04.inc > hotspot.04 == hotspot.04.always_tag.full + hotspot.04.java_call > hotspot.04 == hotspot.02 + hotspot.04.other-platforms + > hotspot.04.always_tag + hotspot.04.java_call > > The updated G1-specific test requiring the whitebox extension is > http://cr.openjdk.java.net/~kbarrett/8166188/test.04 > > The whitebox extension has not changed: > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hsroot.01/ > http://cr.openjdk.java.net/~kbarrett/8166188/whitebox/hotspot.01/ > > From thomas.schatzl at oracle.com Fri Feb 10 10:49:50 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 10 Feb 2017 11:49:50 +0100 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: <1486723790.2702.39.camel@oracle.com> Hi Kim, On Mon, 2017-02-06 at 20:03 -0500, Kim Barrett wrote: > Apologies for taking so long to respond to earlier comments.??I've > been distracted by some other tasks, and it took a while to figure > out what to do about the problem Mikael reported. > > [...] > CR: > https://bugs.openjdk.java.net/browse/JDK-8166188 > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 ?- ReturnJNIWeak.java: please make sure that the test is not called with -XX:+ExplicitGCInvokesConcurrent. It will most likely fail in this case. ?- jniHandles.hpp: it might be useful to explain the external_guard template parameter to the new guard_value/resolve_impl methods. This might be due to my unfamiliarity to the code, so skip this if you think it is "obvious" what this means. >From the callers of JNIHandles::resolve() I am actually not sure if the extra performance due to templatizing the parameter has any impact. Again, your call. ?- it would be nice if JNIHandles::weak_oops_do(BoolObjectClosure, OopClosure) would have an assert_at_safepoint_or_JNI_lock_locked() - but that's probably another issue. Same with the _global_handles array. ?- jniHandles.hpp:58. Not sure why the "int" type specifier for weak_tag_value has an extra space in front of it. ?- isn't is_global_handle() thread-unsafe too? The assert uses JNIHandleBlock::chain_contains(), which iterates over the JNIHandles array unguarded, but it seems possible to add entries concurrently using e.g. make_global()? I.e. one thread adding a global handle, another thread calling e.g. jni_GetObjectRefType? That may be another issue. Same with the accesses to _weak_global_handle value. ?- javaCalls.hpp: in line 135 I would prefer if the two increment statements were put on separate lines. I looked over the second one a few times too many. ?- javaCalls.cpp:449: s/Invaild/Invalid/ ?- javaCalls.cpp:526: s/matched/match/ ; maybe also upper case the initial word of the sentence. ?- javaCalls.cpp: somebody already asked for a comment for the t < os::vm_page_size() check. ?- in SignatureChekker::check_obj(), maybe put the declaration of the "bad" bool into the if-scope as it is never used outside. ?- I would prefer that the tagging mechanism would be described better in JNIHandles.hpp. "Low tag bit in jobject used to distinguish a jweak" seems not sufficient to explain what and why we do the tagging (or how it works). Particularly because some code (shareNativeWrapper.cpp) refers to an explanation in JNIHandles.hpp which is not there. ?- templateInterpreterGenerator_sparc.cpp, line 1523: am I correct that in case of jweaks, this generates a guaranteed unaligned load? E.g. the sequence: brx(Assembler::zero, true, Assembler::pt, store_result); delayed()->ld_ptr(O0, 0, O0); ?// <--- this one I think is a guaranteed unaligned load if O0 contains a jweak. ld_ptr(O0, -JNIHandles::weak_tag_value, O0); On some machines this may cause a SIGBUS afaik - depending on OS configuration/capabilities. I would prefer if the code would not attempt to be clever here. Apparently the same for sharedRuntime_sparc.cpp. ?- I am also not sure why some of the code generation methods have the STATIC_ASSERT(JNIHandles::weak_tag_mask == 1u) and others not. (E.g. ARM32/64 code has it, others apparently not). I would kind of prefer to have it everywhere, but I am also fine with one e.g. in JNIHandles.cpp with a big fat comment. Thanks, ? Thomas From yasuenag at gmail.com Fri Feb 10 14:15:09 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 10 Feb 2017 23:15:09 +0900 Subject: RFR: 8173941: SA does not work if executable is DSO Message-ID: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> Hi all, In modern Linux e.g. Fedora 25, executables are built as DSO for security [1]. java command in OpenJDK which is provided by distribution is also DSO. However, SA does not work with DSO executables. I uploaded a webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8173941/webrev.01/ I discussed about this issue with Andrew in mail thread [2], and this webrev has been reviewed by him. We need one or more reviewer. Could you review it? Currently, OpenJDK 8 in Fedora 25 is built as DSO. So I want to contribute this patch to JDK 8 or later release. I cannot access JPRT. So I need a sponsor. Thanks, Yasumasa [1] https://fedoraproject.org/wiki/Packaging:Guidelines#PIE [2] http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-February/020940.html From aph at redhat.com Fri Feb 10 14:16:28 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 10 Feb 2017 14:16:28 +0000 Subject: RFR: 8173941: SA does not work if executable is DSO In-Reply-To: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> References: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> Message-ID: <9263a5ed-b677-6bb2-faa1-3074b1aa9e8c@redhat.com> On 10/02/17 14:15, Yasumasa Suenaga wrote: > We need one or more reviewer. Could you review it? We don't need another reviewer. We need a sponsor to put it through JPRT. Andrew. From dmitry.samersoff at oracle.com Fri Feb 10 14:41:40 2017 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 10 Feb 2017 17:41:40 +0300 Subject: RFR: 8173941: SA does not work if executable is DSO In-Reply-To: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> References: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> Message-ID: Yasumasa, Changes looks good to me. I'll sponsor the push. -Dmitry On 2017-02-10 17:15, Yasumasa Suenaga wrote: > Hi all, > > In modern Linux e.g. Fedora 25, executables are built as DSO for > security [1]. > java command in OpenJDK which is provided by distribution is also DSO. > However, SA does not work with DSO executables. > > I uploaded a webrev: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8173941/webrev.01/ > > I discussed about this issue with Andrew in mail thread [2], and this > webrev has been reviewed by him. > We need one or more reviewer. Could you review it? > > > Currently, OpenJDK 8 in Fedora 25 is built as DSO. > So I want to contribute this patch to JDK 8 or later release. > > I cannot access JPRT. So I need a sponsor. > > > Thanks, > > Yasumasa > > > [1] https://fedoraproject.org/wiki/Packaging:Guidelines#PIE > [2] > http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-February/020940.html > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From yasuenag at gmail.com Fri Feb 10 14:55:16 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 10 Feb 2017 23:55:16 +0900 Subject: RFR: 8173941: SA does not work if executable is DSO In-Reply-To: References: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> Message-ID: Thanks Dmitry! Should I send a changeset to you for JDK 9? or JDK 10? Yasumasa On 2017/02/10 23:41, Dmitry Samersoff wrote: > Yasumasa, > > Changes looks good to me. > > I'll sponsor the push. > > -Dmitry > > On 2017-02-10 17:15, Yasumasa Suenaga wrote: >> Hi all, >> >> In modern Linux e.g. Fedora 25, executables are built as DSO for >> security [1]. >> java command in OpenJDK which is provided by distribution is also DSO. >> However, SA does not work with DSO executables. >> >> I uploaded a webrev: >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8173941/webrev.01/ >> >> I discussed about this issue with Andrew in mail thread [2], and this >> webrev has been reviewed by him. >> We need one or more reviewer. Could you review it? >> >> >> Currently, OpenJDK 8 in Fedora 25 is built as DSO. >> So I want to contribute this patch to JDK 8 or later release. >> >> I cannot access JPRT. So I need a sponsor. >> >> >> Thanks, >> >> Yasumasa >> >> >> [1] https://fedoraproject.org/wiki/Packaging:Guidelines#PIE >> [2] >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-February/020940.html >> > > From kim.barrett at oracle.com Fri Feb 10 23:05:10 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 10 Feb 2017 18:05:10 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> Message-ID: <2C42A7AD-2F8D-480A-AF7D-22E1A1CF2C28@oracle.com> > On Feb 9, 2017, at 9:34 PM, Coleen Phillimore wrote: > > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/cpu/x86/vm/sharedRuntime_x86_32.cpp.udiff.html > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/cpu/x86/vm/sharedRuntime_x86_64.cpp.udiff.html > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/cpu/x86/vm/templateInterpreterGenerator_x86.cpp.udiff.html > > The code to unpack the oop result is the same at least for the x86 implementations, which unfortunately still have 32/64 bit variants for sharedRuntime_x86.cpp. So in the x86 case, there are 3. I think these should be refactored into a function in macroAssembler_x86.cpp, something like unpack_native_oop_result(). It's not technically unboxing because boxing refers to making primitives into java.lang.Integer, etc objects. > > Sorry that this might be more work and tricky for the other platforms because I didn't diff them, but it would be a lot better. Maintaining separate copies of assembly code is a lot of work too. You can file an RFE/bug to clean this up if there was not a reason for the duplication because of ZBB. I?m working on this. I?m calling the macroAssembler function resolve_jobject. x86 is easy. ARM64 (and ARM32 sharedRuntime) are also easy. ARM32 templateInterpreterGenerator is different, but maybe not for a good enough reason; still investigating that one. Sparc is hard, and I?m going to punt that one. I?ll probably leave the non-Oracle supported ports to their owners to decide whether to do something similar. > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/share/vm/prims/jni.cpp.udiff.html > > Can you change 'l' to obj? never mind, I see why you picked 'l' but it looks like a 1. Don?t blame me; I had nothing to do with that choice. :) > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/share/vm/runtime/javaCalls.hpp.udiff.html > > I see what you've done. These are terrible casts. > > + JNITypes::put_obj((oop)handle, _value, size); // Updates size. > > > I think as a future cleanup, we should make JNITypes::put_obj take (intptr_t) or (jobject,Handle) instead of dealing with the oop class for CheckUnhandledOops. I didn't see any other callers except this one. Yes, that?s the "followup cleanup bug? I was referring to. > Why is value_state uint when it's being stored into uchar? > > + static const uint value_state_primitive = 0; > > > Could they be enums near where _value_state_buffer is declared? I had a reason in an earlier version of the code, but an enum makes sense now, so I?ll make that change. > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/src/share/vm/runtime/jniHandles.hpp.udiff.html > > I don't think it's good to have to include > > +#include "gc/g1/g1SATBCardTableModRefBS.hpp" > > > in jniHandles.hpp header file. > > could you refactor resolve_impl to put the jweak case in the .cpp file? I don't think performance is going to be important for jweak. All this work for an indirection. With the template code, it's hard to see that's all it is. :( Good idea. I?ll make that change. > http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04/test/runtime/jni/ReturnJNIWeak/ReturnJNIWeak.java.html > > Someone else asked this question but how can you assume that GC cleared the weak reference? > > 115 System.gc(); > 116 // Verify weak ref cleared as expected. There are no longer any strong references? > That's all. The change looks good. Thanks. I?ll have a new webrev out soon; need to finish per-review changes and retest first. From kim.barrett at oracle.com Sat Feb 11 00:30:38 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 10 Feb 2017 19:30:38 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1486723790.2702.39.camel@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <1486723790.2702.39.camel@oracle.com> Message-ID: > On Feb 10, 2017, at 5:49 AM, Thomas Schatzl wrote: > > Hi Kim, > > On Mon, 2017-02-06 at 20:03 -0500, Kim Barrett wrote: >> Apologies for taking so long to respond to earlier comments. I've >> been distracted by some other tasks, and it took a while to figure >> out what to do about the problem Mikael reported. >> >> > [...] >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.04 Thanks for looking at this. I?ll have a new webrev containing changes discussed here and elsewhere soon; need to do some testing first. > - ReturnJNIWeak.java: please make sure that the test is not called > with -XX:+ExplicitGCInvokesConcurrent. It will most likely fail in this > case. Yes, I noticed that yesterday. Thanks. > - jniHandles.hpp: it might be useful to explain the external_guard > template parameter to the new guard_value/resolve_impl methods. > > This might be due to my unfamiliarity to the code, so skip this if you > think it is "obvious" what this means. I've added some commentary. The template parameter is just whether these helpers are being called from resolve_external_guard or not. That variant resolves some erroneous cases to NULL, rather than treating them as (possibly unchecked) errors. > From the callers of JNIHandles::resolve() I am actually not sure if the > extra performance due to templatizing the parameter has any impact. I expect making external_guard an ordinary bool argument and letting the compiler constant fold would produce equivalent code. I went with a template parameter because, semantically, my intent is that the value must be a constant, and making it a template parameter enforces that. > - it would be nice if JNIHandles::weak_oops_do(BoolObjectClosure, > OopClosure) would have an assert_at_safepoint_or_JNI_lock_locked() - > but that's probably another issue. Same with the _global_handles array. Separate issue. File an RFE? > - jniHandles.hpp:58. Not sure why the "int" type specifier for > weak_tag_value has an extra space in front of it. ? TBD > - isn't is_global_handle() thread-unsafe too? The assert uses > JNIHandleBlock::chain_contains(), which iterates over the JNIHandles > array unguarded, but it seems possible to add entries concurrently > using e.g. make_global()? > > I.e. one thread adding a global handle, another thread calling e.g. > jni_GetObjectRefType? > > That may be another issue. Same with the accesses to > _weak_global_handle value. Separate issue. File a bug? > - javaCalls.hpp: in line 135 I would prefer if the two increment > statements were put on separate lines. I looked over the second one a > few times too many. Agreed. Done. > - javaCalls.cpp:449: s/Invaild/Invalid/ > > - javaCalls.cpp:526: s/matched/match/ ; maybe also upper case the > initial word of the sentence. Typos fixed. signature is the name of the variable, so I prefer not capitalizing. > - javaCalls.cpp: somebody already asked for a comment for the t < > os::vm_page_size() check. Already added. Still not totally convinced this is appropriate. > - in SignatureChekker::check_obj(), maybe put the declaration of the > "bad" bool into the if-scope as it is never used outside. Done. Also eliminated p variable, which I don?t think helps readability. And cleaned up the comments nearby. > - I would prefer that the tagging mechanism would be described better > in JNIHandles.hpp. "Low tag bit in jobject used to distinguish a jweak" > seems not sufficient to explain what and why we do the tagging (or how > it works). > Particularly because some code (shareNativeWrapper.cpp) refers to an > explanation in JNIHandles.hpp which is not there. Added more description. > - templateInterpreterGenerator_sparc.cpp, line 1523: am I correct that > in case of jweaks, this generates a guaranteed unaligned load? > > E.g. the sequence: > > brx(Assembler::zero, true, Assembler::pt, store_result); > delayed()->ld_ptr(O0, 0, O0); // <--- this one I think is a guaranteed > unaligned load if O0 contains a jweak. > ld_ptr(O0, -JNIHandles::weak_tag_value, O0); The code is correct, there is no unaligned access. In the brx, the second argument is true. That?s the annul flag. That means the instruction in the delay slot will only be executed if the branch is taken, and will be treated as a nop if the branch is not taken. If the annul flag is false then the instruction in the delay slot will be executed whether the branch is taken or not. > On some machines this may cause a SIGBUS afaik - depending on OS > configuration/capabilities. I would prefer if the code would not > attempt to be clever here. > > Apparently the same for sharedRuntime_sparc.cpp. > > - I am also not sure why some of the code generation methods have the > STATIC_ASSERT(JNIHandles::weak_tag_mask == 1u) and others not. (E.g. > ARM32/64 code has it, others apparently not). > I would kind of prefer to have it everywhere, but I am also fine with > one e.g. in JNIHandles.cpp with a big fat comment. In non-ARM code, there?s no dependency on the size of the tag. The ARM/AARCH64 code is using a tbz instruction, test a single bit (specifically bit 0 in this case, see the second argument) and branch if zero. So we?re assuming the tag field is one bit wide and at bit zero, e.g. mask == 1. Hence the static assert. From kim.barrett at oracle.com Sat Feb 11 04:02:17 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 10 Feb 2017 23:02:17 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> Message-ID: <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> > On Feb 8, 2017, at 7:00 PM, Kim Barrett wrote: >> L571: if (t < (size_t)os::vm_page_size()) { >> Again, not your issue, but that size < page size check means >> the oop is bad is screaming for a short comment. > > Added comment. Also removed the unnecessary intermediate variable t, > thereby eliminating some casts. I'm not completly convinced by the > comment, but remaining true to the existing code. Here's the diff: > > @@ -566,9 +568,10 @@ > uint value_state = _value_state[p]; > if (is_value_state_indirect_oop(value_state)) { > intptr_t v = _value[p]; > - if (v != 0 ) { > - size_t t = (size_t)v; > - if (t < (size_t)os::vm_page_size()) { > + if (v != 0) { > + if (v < os::vm_page_size()) { > + // v is a "handle" referring to an oop, cast to integral type. > + // There ought not be any handles in very low memory. > bad = true; > } else if (!resolve_indirect_oop(v, value_state)->is_oop_or_null(true)) { > bad = true; Well, that was dumb. v can be negative, and vm_page_size return (positive) int. Just pretend you never saw that? From david.holmes at oracle.com Mon Feb 13 01:09:51 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 13 Feb 2017 11:09:51 +1000 Subject: RFR: 8173941: SA does not work if executable is DSO In-Reply-To: <9263a5ed-b677-6bb2-faa1-3074b1aa9e8c@redhat.com> References: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> <9263a5ed-b677-6bb2-faa1-3074b1aa9e8c@redhat.com> Message-ID: Hi Yasumasa, Please don't start new email threads on the same topic just to add a cc. I've added to the existing thread and that is now missed on this new one! It makes it hard to track comments, reviews and outstanding issues. serviceability-dev was the correct mailing list for this change, there was no need to also send to hotspot-dev. On 11/02/2017 12:16 AM, Andrew Haley wrote: > On 10/02/17 14:15, Yasumasa Suenaga wrote: >> We need one or more reviewer. Could you review it? > > We don't need another reviewer. We need a sponsor to put it > through JPRT. For hotspot changes we do prefer at least 2 reviews (though only 1 need be a Reviewer). The rules are not clearly documented anywhere that I can see. Based on this: http://openjdk.java.net/guide/changePlanning.html "Changeset pushes before the Feature Complete require at least one Reviewer; pushes after the Feature Complete require at least two Reviewers. In either case, the more the merrier. Some teams may require more Reviewers. Check with members of the Project." we need two Reviewers after FC, but I don't think anyone has been aware of and enforcing that! David ----- > Andrew. > From yasuenag at gmail.com Mon Feb 13 03:02:07 2017 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 13 Feb 2017 12:02:07 +0900 Subject: RFR: 8173941: SA does not work if executable is DSO In-Reply-To: References: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> <9263a5ed-b677-6bb2-faa1-3074b1aa9e8c@redhat.com> Message-ID: Hi David, Sorry for my incorrect mail thread. For this issue, I already got two reviewers: http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-February/020968.html http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-February/020973.html Yasumasa 2017-02-13 10:09 GMT+09:00 David Holmes : > Hi Yasumasa, > > Please don't start new email threads on the same topic just to add a cc. > I've added to the existing thread and that is now missed on this new one! > It makes it hard to track comments, reviews and outstanding issues. > serviceability-dev was the correct mailing list for this change, there was > no need to also send to hotspot-dev. > > On 11/02/2017 12:16 AM, Andrew Haley wrote: > >> On 10/02/17 14:15, Yasumasa Suenaga wrote: >> >>> We need one or more reviewer. Could you review it? >>> >> >> We don't need another reviewer. We need a sponsor to put it >> through JPRT. >> > > For hotspot changes we do prefer at least 2 reviews (though only 1 need be > a Reviewer). The rules are not clearly documented anywhere that I can see. > Based on this: > > http://openjdk.java.net/guide/changePlanning.html > > "Changeset pushes before the Feature Complete require at least one > Reviewer; pushes after the Feature Complete require at least two Reviewers. > In either case, the more the merrier. Some teams may require more > Reviewers. Check with members of the Project." > > we need two Reviewers after FC, but I don't think anyone has been aware of > and enforcing that! > > David > ----- > > > Andrew. >> >> From david.holmes at oracle.com Mon Feb 13 03:29:15 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 13 Feb 2017 13:29:15 +1000 Subject: RFR: 8173941: SA does not work if executable is DSO In-Reply-To: References: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> <9263a5ed-b677-6bb2-faa1-3074b1aa9e8c@redhat.com> Message-ID: <63b324ef-dd57-e815-f270-0a602cc66de1@oracle.com> On 13/02/2017 1:02 PM, Yasumasa Suenaga wrote: > Hi David, > > Sorry for my incorrect mail thread. > > For this issue, I already got two reviewers: > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-February/020968.html > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-February/020973.html Yes. But please see my queries on the original thread. Thanks, David > > Yasumasa > > > 2017-02-13 10:09 GMT+09:00 David Holmes >: > > Hi Yasumasa, > > Please don't start new email threads on the same topic just to add a > cc. I've added to the existing thread and that is now missed on this > new one! It makes it hard to track comments, reviews and outstanding > issues. serviceability-dev was the correct mailing list for this > change, there was no need to also send to hotspot-dev. > > On 11/02/2017 12:16 AM, Andrew Haley wrote: > > On 10/02/17 14:15, Yasumasa Suenaga wrote: > > We need one or more reviewer. Could you review it? > > > We don't need another reviewer. We need a sponsor to put it > through JPRT. > > > For hotspot changes we do prefer at least 2 reviews (though only 1 > need be a Reviewer). The rules are not clearly documented anywhere > that I can see. Based on this: > > http://openjdk.java.net/guide/changePlanning.html > > > "Changeset pushes before the Feature Complete require at least one > Reviewer; pushes after the Feature Complete require at least two > Reviewers. In either case, the more the merrier. Some teams may > require more Reviewers. Check with members of the Project." > > we need two Reviewers after FC, but I don't think anyone has been > aware of and enforcing that! > > David > ----- > > > Andrew. > > From kim.barrett at oracle.com Mon Feb 13 04:25:45 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 12 Feb 2017 23:25:45 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> Message-ID: <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> Here's the updated webrevs, incorporating feedback received so far. full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ For x86 and ARM I added resolve_jobject functions to their respective macroAssembler files, and call those to generate the return object handling code. Merging the two cases for Sparc is harder and I'm going to leave it to someone else. I'm also leaving any such merging for other (non-Oracle) ports to their respective maintainers. Rewrote the error checking in SignatureChekker::check_obj to simplify the control flow and provide more information. Rather than conditionally selecting between ReportJNIFatalError or assert to report problems, instead just always use guarantee. Recall that we only get here if CheckJNICalls or in a debug build. This made SignatureChekker::_thread unused, so removed it. From aph at redhat.com Mon Feb 13 09:35:12 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 13 Feb 2017 09:35:12 +0000 Subject: RFR: 8173941: SA does not work if executable is DSO In-Reply-To: References: <6a8b8a71-d792-d5ed-0bef-369fce313f66@gmail.com> <9263a5ed-b677-6bb2-faa1-3074b1aa9e8c@redhat.com> Message-ID: <53500e7a-f0e2-636c-06b9-e2bf1d7b3bbb@redhat.com> On 13/02/17 01:09, David Holmes wrote: > serviceability-dev was the correct mailing list for this change, there > was no need to also send to hotspot-dev. My bad, sorry. Andrew. From martin.doerr at sap.com Mon Feb 13 14:25:41 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 13 Feb 2017 14:25:41 +0000 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> Message-ID: <0216acc686a44788b0451dce7d94e491@sap.com> Hi Kim, I have factored out resolve_jobject on PPC64 and s390. Complete webrev is here: http://cr.openjdk.java.net/~mdoerr/8166188_s390_ppc64/webrev.06/ Incremental patch (based on webrev.05): http://cr.openjdk.java.net/~mdoerr/8166188_s390_ppc64/webrev.06/8166188_s390_ppc64_incr.patch I prefer masking out the weak tag and only checking it if G1 is used. (The additional check and branch is only needed for G1. Also, see new PPC64 and s390 implementation.) What is preferable on x86? Btw. the indentation looks inconsistent in resolve_jobject. Best regards, Martin -----Original Message----- From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett Sent: Montag, 13. Februar 2017 05:26 To: hotspot-dev developers Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles Here's the updated webrevs, incorporating feedback received so far. full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ For x86 and ARM I added resolve_jobject functions to their respective macroAssembler files, and call those to generate the return object handling code. Merging the two cases for Sparc is harder and I'm going to leave it to someone else. I'm also leaving any such merging for other (non-Oracle) ports to their respective maintainers. Rewrote the error checking in SignatureChekker::check_obj to simplify the control flow and provide more information. Rather than conditionally selecting between ReportJNIFatalError or assert to report problems, instead just always use guarantee. Recall that we only get here if CheckJNICalls or in a debug build. This made SignatureChekker::_thread unused, so removed it. From joseph.provino at oracle.com Mon Feb 13 18:57:02 2017 From: joseph.provino at oracle.com (Joseph Provino) Date: Mon, 13 Feb 2017 13:57:02 -0500 Subject: RFR (XXS): JDK-8138610 add assert to ThreadLocalAllocBuffer::clear_before_allocation to check the storage above top Message-ID: Please review this one line change: CR: JDK-8138610 add assert to ThreadLocalAllocBuffer::clear_before_allocation to check the storage above top Webrev: http://cr.openjdk.java.net/~jprovino/8138610/webrev.00 Tests: Passes JPRT. thanks. joe From coleen.phillimore at oracle.com Mon Feb 13 20:10:43 2017 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 13 Feb 2017 15:10:43 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle Message-ID: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> Summary: Pass THREAD to Handle as argument instead of implicit Thread::current() call. It's a very small change to a number of files. Notably the new JVMCI files have a lot of implicit conversions from oop to handle. I want to get this change in early for jdk10 to stop this usage. I also added a few HandleMarks where I found that the HandlesArea was growing large. See bug for more on the motivation of this change. open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8169881 Tested by running all hotspot jtreg tests with -XX:+CheckUnhandledOops. There weren't any unhandled oops amazingly. Thanks, Coleen From bob.vandette at oracle.com Mon Feb 13 20:30:49 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 13 Feb 2017 15:30:49 -0500 Subject: FR: 8174203: Enable AOT Jtreg tests on Windows x86_64 Message-ID: <46F1628D-B824-42E1-812D-F4A517959CC3@oracle.com> Please review this change for JDK 10, that enables jtreg testing of Hotspot AOT support on Windows. This change also includes some fixes that allow JPRT to be able to run these tests. Thanks to Erik Joelsson for providing the JPRT support. CR: https://bugs.openjdk.java.net/browse/JDK-8174203 WEBREV: http://cr.openjdk.java.net/~bobv/8174203/webrev http://cr.openjdk.java.net/~bobv/8174203/hotspot/webrev Bob. From vladimir.kozlov at oracle.com Mon Feb 13 21:41:45 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 13 Feb 2017 13:41:45 -0800 Subject: FR: 8174203: Enable AOT Jtreg tests on Windows x86_64 In-Reply-To: <46F1628D-B824-42E1-812D-F4A517959CC3@oracle.com> References: <46F1628D-B824-42E1-812D-F4A517959CC3@oracle.com> Message-ID: Hotspot changes are good. Thanks, Vladimir On 2/13/17 12:30 PM, Bob Vandette wrote: > Please review this change for JDK 10, that enables jtreg testing of Hotspot AOT support on Windows. > > This change also includes some fixes that allow JPRT to be able to run these tests. > Thanks to Erik Joelsson for providing the JPRT support. > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8174203 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8174203/webrev > http://cr.openjdk.java.net/~bobv/8174203/hotspot/webrev > > > Bob. > From david.holmes at oracle.com Tue Feb 14 01:44:31 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 Feb 2017 11:44:31 +1000 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> Message-ID: <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> Hi Coleen, "actually S"? :) On 14/02/2017 6:10 AM, Coleen Phillimore wrote: > Summary: Pass THREAD to Handle as argument instead of implicit > Thread::current() call. Well there's a bit more to it than that as you also change parameter types so that we (sometimes) avoid: - oop -> Handle -> oop - Handle -> oop -> Handle across method calls. This leads to some puzzling differences eg: src/share/vm/aot/aotCodeHeap.cpp src/share/vm/classfile/javaClasses.hpp The change in aotCodeHeap.cpp reverses the use of Handle and oop, but it is unclear why Throwable::print and Throwable::print_stack_trace have different signatures in that regard. They used to both take a Handle but now one takes an oop instead. ?? More generally it is unclear why you made the changes you did in places - see below for some specifics. I'm left questioning how to know when to take an oop and when to take a Handle. Of course that question already exists, but these changes didn't make it any clearer for me. I understand why the oop to Handle implicit conversion could be problematic due to the Thread::current() calls, but I don't understand why you couldn't keep (? add?) the Handle to oop implicit conversions ?? --- src/share/vm/c1/c1_Runtime1.cpp Aside: I'm a little surprised that there was not a Handle assignment operator or copy constructor such that given: 863 Handle appendix(THREAD, NULL); 970 appendix = cpce->appendix_if_resolved(pool); that it simply updated the NULL to the desired value, while keeping the THREAD intact. --- src/share/vm/ci/ciInstance.cpp Not at all obvious that replacing Handle with raw oop is correct. I'm not saying it isn't, just that it isn't obvious - and I'll wonder why the Handle was used in the first place. --- src/share/vm/classfile/javaClasses.cpp Why change java_lang_String::as_symbol_or_null to take a Handle instead of an oop if you are simply going to unwrap the Handle to get the oop back. ??? Particular when a caller like src/share/vm/prims/methodHandles.cpp has to create the Handle from oop in the first place. --- src/share/vm/jvmci/jvmciCompilerToVM.hpp It is not at all obvious that JavaArgumentUnboxers are thread-confined! I'm also concerned by API's (unfortunately a number pre-existing) that take a Thread/JavaThread argument but really require it to be the current thread. To that end I'd rather see _thread as _cur_thread and an assertion when it is set. --- src/share/vm/oops/cpCache.cpp Nit: objArrayHandle resolved_references (Thread::current(), ... Please remove space before ( --- src/share/vm/prims/jvm.cpp Nit: need to fix indent on StackWalk::walk call second line. --- src/share/vm/prims/jvmtiGetLoadedClasses.cpp Ditto re _thread -> _cur_thread --- src/share/vm/prims/jvmtiImpl.cpp Nit: space before ( again (on handle constructor) > It's a very small change to a number of files. Notably the new JVMCI > files have a lot of implicit conversions from oop to handle. I want to > get this change in early for jdk10 to stop this usage. I also added a > few HandleMarks where I found that the HandlesArea was growing large. The addition and removal of the HandleMarks seems problematic because there are no obvious rules being applied. It is hard to know where and when a HandleMark is needed or not. Looking at src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to create/destroy the HandleMark on each loop iteration? That would only seem necessary if we know the number of iterations is so high that we might consume all our handle space. Thanks, David ------ > See bug for more on the motivation of this change. > > open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8169881 > > Tested by running all hotspot jtreg tests with -XX:+CheckUnhandledOops. > There weren't any unhandled oops amazingly. > > Thanks, > Coleen From coleen.phillimore at oracle.com Tue Feb 14 03:53:25 2017 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 13 Feb 2017 22:53:25 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> Message-ID: On 2/13/17 8:44 PM, David Holmes wrote: > Hi Coleen, > > "actually S"? :) Well, maybe not but it shouldn't have a lot of merge conflicts for backports. > > On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >> Summary: Pass THREAD to Handle as argument instead of implicit >> Thread::current() call. > > Well there's a bit more to it than that as you also change parameter > types so that we (sometimes) avoid: > > - oop -> Handle -> oop > - Handle -> oop -> Handle Yes, I put that in the bug but not in the RFR. The recommendation is to Handle the oop as far up as possible in the call stack and pass Handle around. > > across method calls. This leads to some puzzling differences eg: > > src/share/vm/aot/aotCodeHeap.cpp > src/share/vm/classfile/javaClasses.hpp > > The change in aotCodeHeap.cpp reverses the use of Handle and oop, but > it is unclear why Throwable::print and Throwable::print_stack_trace > have different signatures in that regard. They used to both take a > Handle but now one takes an oop instead. ?? Yes, you found the exceptions in Throwable. A lot of the java_lang_Throwable (and other javaClasses.hpp classes) pass oop instead of Handle and I changed print to oop to make it consistent, and because they it being called from other functions in Throwable with oop. Throwable::print_stack_trace actually goes to a safepoint so has to be Handle. > > More generally it is unclear why you made the changes you did in > places - see below for some specifics. I'm left questioning how to > know when to take an oop and when to take a Handle. Of course that > question already exists, but these changes didn't make it any clearer > for me. Pass a Handle early and often is still the rule. For all but GC code. There are still some functions that are very small and don't safepoint and take oop. Some of these have migrated to taking Handle, but some haven't. I didn't want to make this change larger for now. > > I understand why the oop to Handle implicit conversion could be > problematic due to the Thread::current() calls, but I don't understand > why you couldn't keep (? add?) the Handle to oop implicit conversions ?? There weren't Handle -> oop implicit conversions. The conversion back to oop uses the operator() which remains. The oop -> Handle conversion has cost, both at runtime and also to the GC because it increases the number of Handles on the Thread->_handle_area. I did this same experiment years ago when Thread::current() was at the top of the profiling call stack but back then I needed to add too many explicit Thread::current() calls. There aren't many in this change, so it's worth doing. Also, metadata handles have the same problem but there are still too many of them to remove the implicit conversion. And {instance}KlassHandle needs to be removed first. They do nothing now but hide the type. > > --- > > src/share/vm/c1/c1_Runtime1.cpp > > Aside: I'm a little surprised that there was not a Handle assignment > operator or copy constructor such that given: > > 863 Handle appendix(THREAD, NULL); > 970 appendix = cpce->appendix_if_resolved(pool); > > that it simply updated the NULL to the desired value, while keeping > the THREAD intact. Handle doesn't save the THREAD so a NULL handle is just oop* NULL. The RHS of the expression has to be converted to Handle and the default assignment operator copies it to appendix. > > --- > > src/share/vm/ci/ciInstance.cpp > > Not at all obvious that replacing Handle with raw oop is correct. I'm > not saying it isn't, just that it isn't obvious - and I'll wonder why > the Handle was used in the first place. It is safe because it's immediately unhandled in all the case statements, and I wanted to avoid a Thread::current call. > > --- > > src/share/vm/classfile/javaClasses.cpp > > Why change java_lang_String::as_symbol_or_null to take a Handle > instead of an oop if you are simply going to unwrap the Handle to get > the oop back. ??? Particular when a caller like > src/share/vm/prims/methodHandles.cpp has to create the Handle from oop > in the first place. I changed it to be the same as java_lang_String::as_symbol(Handle java_string ...) which took a handle. java_lang_String::as_symbol() was the same - it immediately unhandles the argument to pass to the rest of the functions. I changed java_lang_String::as_symbol() to take an oop to match, and there are few scattered changes from that. > > --- > > src/share/vm/jvmci/jvmciCompilerToVM.hpp > > It is not at all obvious that JavaArgumentUnboxers are > thread-confined! I'm also concerned by API's (unfortunately a number > pre-existing) that take a Thread/JavaThread argument but really > require it to be the current thread. To that end I'd rather see > _thread as _cur_thread and an assertion when it is set. Since it's a ResourceObj and not StackObj (setting aside debate about that), I agree with you. I changed it to use Thread::current() where it handles the object, which it explicitly did before. In this case, it is a stack allocated thing but it's not guaranteed to be that. > > --- > > src/share/vm/oops/cpCache.cpp > > Nit: objArrayHandle resolved_references (Thread::current(), ... > > Please remove space before ( fixed. > > --- > > src/share/vm/prims/jvm.cpp > > Nit: need to fix indent on StackWalk::walk call second line. fixed. > > --- > > src/share/vm/prims/jvmtiGetLoadedClasses.cpp > > Ditto re _thread -> _cur_thread > This one is a StackObj (a Closure which should be used that way). So I changed _thread to _cur_thread and added an assert that it == Thread::current() in the constructor. > --- > > src/share/vm/prims/jvmtiImpl.cpp > > Nit: space before ( again (on handle constructor) fixed. > >> It's a very small change to a number of files. Notably the new JVMCI >> files have a lot of implicit conversions from oop to handle. I want to >> get this change in early for jdk10 to stop this usage. I also added a >> few HandleMarks where I found that the HandlesArea was growing large. > > The addition and removal of the HandleMarks seems problematic because > there are no obvious rules being applied. It is hard to know where and > when a HandleMark is needed or not. Looking at > src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to > create/destroy the HandleMark on each loop iteration? That would only > seem necessary if we know the number of iterations is so high that we > might consume all our handle space. Nobody knows when and where to put HandleMarks. It's a dark art. I also was testing with a JVM that restored the assert if HandleArea got over 200 handles and the numbers got very high in these functions. The GC has to walk these HandleAreas and I am trying to reduce work for them. I tried to write about this in the bug. I'd like to see if changing Handle to a proper constructor/destructor like methodHandles, so that only the Handles that are active need to be collected. Then we don't need HandleMark and their associated mystery. There are several HandleMarks in code that obviously doesn't allocate any Handles. I took out several but restored them so that this patch was smaller. To change Handle to a scoped object would require changing passing Handle as a const reference like I did with methodHandle which is too big of a change for early-jdk10. The reason I worked on this RFE was because I was curious to see how many oops/Handles we're passing around and using from within the JVM these days. It's more than I thought. Thank you for reviewing this. I'm going to rerun some tests with the changes above and send another webrev. Coleen > > Thanks, > David > ------ > >> See bug for more on the motivation of this change. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >> >> Tested by running all hotspot jtreg tests with -XX:+CheckUnhandledOops. >> There weren't any unhandled oops amazingly. >> >> Thanks, >> Coleen From david.holmes at oracle.com Tue Feb 14 04:51:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 Feb 2017 14:51:14 +1000 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> Message-ID: Hi Coleen, On 14/02/2017 1:53 PM, Coleen Phillimore wrote: > > > On 2/13/17 8:44 PM, David Holmes wrote: >> Hi Coleen, >> >> "actually S"? :) > > Well, maybe not but it shouldn't have a lot of merge conflicts for > backports. > >> >> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>> Summary: Pass THREAD to Handle as argument instead of implicit >>> Thread::current() call. >> >> Well there's a bit more to it than that as you also change parameter >> types so that we (sometimes) avoid: >> >> - oop -> Handle -> oop >> - Handle -> oop -> Handle > > Yes, I put that in the bug but not in the RFR. The recommendation is to > Handle the oop as far up as possible in the call stack and pass Handle > around. I hear what you are saying but it is hard to actually see that reflected in the code. >> across method calls. This leads to some puzzling differences eg: >> >> src/share/vm/aot/aotCodeHeap.cpp >> src/share/vm/classfile/javaClasses.hpp >> >> The change in aotCodeHeap.cpp reverses the use of Handle and oop, but >> it is unclear why Throwable::print and Throwable::print_stack_trace >> have different signatures in that regard. They used to both take a >> Handle but now one takes an oop instead. ?? > > Yes, you found the exceptions in Throwable. A lot of the > java_lang_Throwable (and other javaClasses.hpp classes) pass oop instead > of Handle and I changed print to oop to make it consistent, and because > they it being called from other functions in Throwable with oop. > Throwable::print_stack_trace actually goes to a safepoint so has to be > Handle. I think in javaClasses.hpp the rule should be oop unless Handle is definitely needed. > >> >> More generally it is unclear why you made the changes you did in >> places - see below for some specifics. I'm left questioning how to >> know when to take an oop and when to take a Handle. Of course that >> question already exists, but these changes didn't make it any clearer >> for me. > > Pass a Handle early and often is still the rule. For all but GC code. > There are still some functions that are very small and don't safepoint > and take oop. Some of these have migrated to taking Handle, but some > haven't. I didn't want to make this change larger for now. > >> >> I understand why the oop to Handle implicit conversion could be >> problematic due to the Thread::current() calls, but I don't understand >> why you couldn't keep (? add?) the Handle to oop implicit conversions ?? > > There weren't Handle -> oop implicit conversions. The conversion back > to oop uses the operator() which remains. Okay then it seems an implicit Handle -> oop would avoid the need to change the caller from foo to foo() if the underlying API switched from oop to Handle. > The oop -> Handle conversion has cost, both at runtime and also to the > GC because it increases the number of Handles on the Thread->_handle_area. Sure. > I did this same experiment years ago when Thread::current() was at the > top of the profiling call stack but back then I needed to add too many > explicit Thread::current() calls. There aren't many in this change, so > it's worth doing. Also, metadata handles have the same problem but > there are still too many of them to remove the implicit conversion. > And {instance}KlassHandle needs to be removed first. They do nothing > now but hide the type. > >> >> --- >> >> src/share/vm/c1/c1_Runtime1.cpp >> >> Aside: I'm a little surprised that there was not a Handle assignment >> operator or copy constructor such that given: >> >> 863 Handle appendix(THREAD, NULL); >> 970 appendix = cpce->appendix_if_resolved(pool); >> >> that it simply updated the NULL to the desired value, while keeping >> the THREAD intact. > > Handle doesn't save the THREAD so a NULL handle is just oop* NULL. The > RHS of the expression has to be converted to Handle and the default > assignment operator copies it to appendix. I had assumed Handle saved the Thread. >> >> --- >> >> src/share/vm/ci/ciInstance.cpp >> >> Not at all obvious that replacing Handle with raw oop is correct. I'm >> not saying it isn't, just that it isn't obvious - and I'll wonder why >> the Handle was used in the first place. > > It is safe because it's immediately unhandled in all the case > statements, and I wanted to avoid a Thread::current call. >> >> --- >> >> src/share/vm/classfile/javaClasses.cpp >> >> Why change java_lang_String::as_symbol_or_null to take a Handle >> instead of an oop if you are simply going to unwrap the Handle to get >> the oop back. ??? Particular when a caller like >> src/share/vm/prims/methodHandles.cpp has to create the Handle from oop >> in the first place. > > I changed it to be the same as java_lang_String::as_symbol(Handle > java_string ...) which took a handle. java_lang_String::as_symbol() was > the same - it immediately unhandles the argument to pass to the rest of > the functions. I changed java_lang_String::as_symbol() to take an oop > to match, and there are few scattered changes from that. I have to disagree with this one. AFAICS there is no reason this: 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { needs to take a Handle in the first place. The java_string is unwrapped to get the oop and the oop is only passed to a few other String methods and then not touched. Looking at the API's in javaClass.hpp the norm should be to take oop (as the bulk of methods do) with Handle only used when actually required. > >> >> --- >> >> src/share/vm/jvmci/jvmciCompilerToVM.hpp >> >> It is not at all obvious that JavaArgumentUnboxers are >> thread-confined! I'm also concerned by API's (unfortunately a number >> pre-existing) that take a Thread/JavaThread argument but really >> require it to be the current thread. To that end I'd rather see >> _thread as _cur_thread and an assertion when it is set. > > Since it's a ResourceObj and not StackObj (setting aside debate about > that), I agree with you. I changed it to use Thread::current() where it > handles the object, which it explicitly did before. In this case, it is > a stack allocated thing but it's not guaranteed to be that. Ok. >> >> --- >> >> src/share/vm/oops/cpCache.cpp >> >> Nit: objArrayHandle resolved_references (Thread::current(), ... >> >> Please remove space before ( > > fixed. > >> >> --- >> >> src/share/vm/prims/jvm.cpp >> >> Nit: need to fix indent on StackWalk::walk call second line. > > fixed. > >> >> --- >> >> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >> >> Ditto re _thread -> _cur_thread >> > > This one is a StackObj (a Closure which should be used that way). So I > changed _thread to _cur_thread and added an assert that it == > Thread::current() in the constructor. Thanks. >> --- >> >> src/share/vm/prims/jvmtiImpl.cpp >> >> Nit: space before ( again (on handle constructor) > > fixed. >> >>> It's a very small change to a number of files. Notably the new JVMCI >>> files have a lot of implicit conversions from oop to handle. I want to >>> get this change in early for jdk10 to stop this usage. I also added a >>> few HandleMarks where I found that the HandlesArea was growing large. >> >> The addition and removal of the HandleMarks seems problematic because >> there are no obvious rules being applied. It is hard to know where and >> when a HandleMark is needed or not. Looking at >> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >> create/destroy the HandleMark on each loop iteration? That would only >> seem necessary if we know the number of iterations is so high that we >> might consume all our handle space. > > Nobody knows when and where to put HandleMarks. It's a dark art. I So a trade-off between too much cleanup overhead if we have too many, and too much GC overhead if we have too few. And no real way to know either way - Ouch! :( > also was testing with a JVM that restored the assert if HandleArea got > over 200 handles and the numbers got very high in these functions. The > GC has to walk these HandleAreas and I am trying to reduce work for them. > > I tried to write about this in the bug. I'd like to see if changing > Handle to a proper constructor/destructor like methodHandles, so that > only the Handles that are active need to be collected. Then we don't > need HandleMark and their associated mystery. There are several > HandleMarks in code that obviously doesn't allocate any Handles. I took > out several but restored them so that this patch was smaller. To > change Handle to a scoped object would require changing passing Handle > as a const reference like I did with methodHandle which is too big of a > change for early-jdk10. Ok. This sounds like a way forward though. In combination with an easy way to detect when handles are required, this would allow very precise management of handle lifetimes. > The reason I worked on this RFE was because I was curious to see how > many oops/Handles we're passing around and using from within the JVM > these days. It's more than I thought. > > Thank you for reviewing this. I'm going to rerun some tests with the > changes above and send another webrev. Ok. Thanks. David ----- > Coleen > >> >> Thanks, >> David >> ------ >> >>> See bug for more on the motivation of this change. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>> >>> Tested by running all hotspot jtreg tests with -XX:+CheckUnhandledOops. >>> There weren't any unhandled oops amazingly. >>> >>> Thanks, >>> Coleen > From erik.joelsson at oracle.com Tue Feb 14 08:15:25 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 14 Feb 2017 09:15:25 +0100 Subject: FR: 8174203: Enable AOT Jtreg tests on Windows x86_64 In-Reply-To: <46F1628D-B824-42E1-812D-F4A517959CC3@oracle.com> References: <46F1628D-B824-42E1-812D-F4A517959CC3@oracle.com> Message-ID: <01ff32d1-21fa-91f1-8dd4-30653c71e285@oracle.com> Looks good to me. /Erik On 2017-02-13 21:30, Bob Vandette wrote: > Please review this change for JDK 10, that enables jtreg testing of Hotspot AOT support on Windows. > > This change also includes some fixes that allow JPRT to be able to run these tests. > Thanks to Erik Joelsson for providing the JPRT support. > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8174203 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8174203/webrev > http://cr.openjdk.java.net/~bobv/8174203/hotspot/webrev > > > Bob. From shafi.s.ahmad at oracle.com Tue Feb 14 09:19:52 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 14 Feb 2017 01:19:52 -0800 (PST) Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field Message-ID: Summary: java.lang.ClassFormatError: Exception "Duplicate field name&signature in class file" should report the name and signature of the field. It's a very small change to single file. In the current implementation name and signature of duplicate field is missing in java.lang.ClassFormatError exception message. Without a field name + signature it is hard to triggering the problem. Webrev link: http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 Testing: jprt and jtreg test. I have verified my changes with the reproduces of https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 code base as I was not able to write reproducer of current issue. With the fix I am getting below exception message. $ java CreateBadClassFile .foreach() call: Exception in thread "main" java.lang.ClassFormatError: Duplicate field name "hasNext" with signature "Ljava.lang.Object;" in class file WidgetCollection$1 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:760) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) . . . Thanks, Coleen From david.holmes at oracle.com Tue Feb 14 13:03:58 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 Feb 2017 23:03:58 +1000 Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: References: Message-ID: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> Hi Shafi, I'm concerned about the use of NEW_RESOURCE_ARRAY_IN_THREAD. If it can't allocate it will abort the VM. That seems like a bad thing to happen. Thanks, David On 14/02/2017 7:19 PM, Shafi Ahmad wrote: > Summary: java.lang.ClassFormatError: Exception "Duplicate field name&signature in class file" should report the name and signature of the field. > > It's a very small change to single file. > In the current implementation name and signature of duplicate field is missing in java.lang.ClassFormatError exception message. > Without a field name + signature it is hard to triggering the problem. > > Webrev link: http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ > bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 > > Testing: jprt and jtreg test. > > I have verified my changes with the reproduces of https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 code base as I was not able to write reproducer of current issue. > With the fix I am getting below exception message. > > $ java CreateBadClassFile > .foreach() call: Exception in thread "main" java.lang.ClassFormatError: Duplicate field name "hasNext" with signature "Ljava.lang.Object;" in class file WidgetCollection$1 > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > . . . > > Thanks, > Coleen > From shafi.s.ahmad at oracle.com Tue Feb 14 13:20:58 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 14 Feb 2017 05:20:58 -0800 (PST) Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> References: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> Message-ID: <9d506110-91a1-4827-99a8-e46e9363eb3d@default> Hi David, Thanks for reviewing it. Initially I started with fixed size of local char array but later I changed my mind and make it dynamic. Let me know if I have to make it local char array like. + unsigned int siglength = sig->utf8_length(); + unsigned int namelength = name->utf8_length(); + unsigned int length = siglength + namelength + 64; + char *buff = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, length); + jio_snprintf(buff, length, + "Duplicate method name \"%s\" with signature \"%s\" in class file", + name->as_C_string(), sig->as_klass_external_name()); + classfile_parse_error("%s %s", buff, CHECK); } to + char buff[fixedsize]; // say fixedsize is 512 + jio_snprintf(buff, 512, + "Duplicate method name \"%s\" with signature \"%s\" in class file", + name->as_C_string(), sig->as_klass_external_name()); + classfile_parse_error("%s %s", buff, CHECK); } Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Tuesday, February 14, 2017 6:34 PM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > Hi Shafi, > > I'm concerned about the use of NEW_RESOURCE_ARRAY_IN_THREAD. If it > can't allocate it will abort the VM. That seems like a bad thing to happen. > > Thanks, > David > > On 14/02/2017 7:19 PM, Shafi Ahmad wrote: > > Summary: java.lang.ClassFormatError: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field. > > > > It's a very small change to single file. > > In the current implementation name and signature of duplicate field is > missing in java.lang.ClassFormatError exception message. > > Without a field name + signature it is hard to triggering the problem. > > > > Webrev link: http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ > > bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 > > > > Testing: jprt and jtreg test. > > > > I have verified my changes with the reproduces of > https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 code > base as I was not able to write reproducer of current issue. > > With the fix I am getting below exception message. > > > > $ java CreateBadClassFile > > .foreach() call: Exception in thread "main" java.lang.ClassFormatError: > Duplicate field name "hasNext" with signature "Ljava.lang.Object;" in class > file WidgetCollection$1 > > at java.lang.ClassLoader.defineClass1(Native Method) > > at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > > . . . > > > > Thanks, > > Coleen > > From kim.barrett at oracle.com Tue Feb 14 15:49:11 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 14 Feb 2017 10:49:11 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <0216acc686a44788b0451dce7d94e491@sap.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> <0216acc686a44788b0451dce7d94e491@sap.com> Message-ID: <53F38C3F-A599-41A7-B1DB-2225AFC4F659@oracle.com> > On Feb 13, 2017, at 9:25 AM, Doerr, Martin wrote: > > Hi Kim, > > I have factored out resolve_jobject on PPC64 and s390. > > Complete webrev is here: > http://cr.openjdk.java.net/~mdoerr/8166188_s390_ppc64/webrev.06/ > > Incremental patch (based on webrev.05): > http://cr.openjdk.java.net/~mdoerr/8166188_s390_ppc64/webrev.06/8166188_s390_ppc64_incr.patch Thanks. I?ll add that to my patch set. > I prefer masking out the weak tag and only checking it if G1 is used. (The additional check and branch is only needed for G1. Also, see new PPC64 and s390 implementation.) I considered that, but chose to not penalize the G1 case (the default case now) to perhaps slightly improve the non-G1 (non-default) case. (Branch prediction may mitigate the extra branch cost.) But it?s unlikely to matter either way. > What is preferable on x86? Btw. the indentation looks inconsistent in resolve_jobject. Thanks for noticing the mis-indentation. Will fix. > > Best regards, > Martin > > > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > Sent: Montag, 13. Februar 2017 05:26 > To: hotspot-dev developers > Cc: s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: Re: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles > > Here's the updated webrevs, incorporating feedback received so far. > > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ > > For x86 and ARM I added resolve_jobject functions to their respective > macroAssembler files, and call those to generate the return object > handling code. Merging the two cases for Sparc is harder and I'm > going to leave it to someone else. I'm also leaving any such merging > for other (non-Oracle) ports to their respective maintainers. > > Rewrote the error checking in SignatureChekker::check_obj to simplify > the control flow and provide more information. Rather than > conditionally selecting between ReportJNIFatalError or assert to > report problems, instead just always use guarantee. Recall that we > only get here if CheckJNICalls or in a debug build. This made > SignatureChekker::_thread unused, so removed it. From mikael.gerdin at oracle.com Tue Feb 14 16:26:56 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 14 Feb 2017 17:26:56 +0100 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> Message-ID: <214376fc-ebae-4f0e-2074-1a3595ce5e1e@oracle.com> Hi Kim, On 2017-02-13 05:25, Kim Barrett wrote: > Here's the updated webrevs, incorporating feedback received so far. > > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ I think your change is ready to go, consider it Reviewed by me. I can't really say that I think this looks good due to the fact that I'm still traumatized by JavaCallArguments and how it uses something I've come to think of as "alternative code" to handle oop parameters but that is something that I hope we can clean up at some point in the future. Thanks for fixing this! /Mikael > > For x86 and ARM I added resolve_jobject functions to their respective > macroAssembler files, and call those to generate the return object > handling code. Merging the two cases for Sparc is harder and I'm > going to leave it to someone else. I'm also leaving any such merging > for other (non-Oracle) ports to their respective maintainers. > > Rewrote the error checking in SignatureChekker::check_obj to simplify > the control flow and provide more information. Rather than > conditionally selecting between ReportJNIFatalError or assert to > report problems, instead just always use guarantee. Recall that we > only get here if CheckJNICalls or in a debug build. This made > SignatureChekker::_thread unused, so removed it. > From erik.helin at oracle.com Tue Feb 14 16:40:55 2017 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 14 Feb 2017 17:40:55 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking Message-ID: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> Hi all, this patch aims to solve the bug [0] by making it safe for a GC to concurrently traverse the oops in a ClassLoaderData. The problem right now is that ClassLoaderData::_handles is a JNIHandleBlock. It is not safe for one thread to iterate over the oops in a JNIHandleBlock while another thread concurrently adds a new oop to the JNIHandleBlock. My proposed solution is to replace JNIHandleBlock with another data structure for ClassLoaderData. ClassLoaderData only needs a place to store oops that a GC can iterate over, it has no use for the rest of the methods in the JNIHandleBlock class. I have implemented a minimal chunked linked list data structure (internal to ClassLoaderData) with only two public methods: - add (only executed by one thread at a time) - oops_do (can be executed by any number of threads, possibly concurrently with a call to `add`) ChunkedHandleList uses load_acquire and release_store to synchronize access to the underlying chunks (arrays). Since ChunkedHandleList utilizes the (rather) minimal requirements of its user (ClassLoaderData), I decided to keep the class internal to ClassLoaderData for now. If other find it useful elsewhere, the we can try to create a more generalized version (this is trickier than it appears at first sight, I tried ;)). I also changed ClassLoaderData::remove_handle to write NULL to the oop* (that is represented by a jobject), instead of using a sentinel oop as done by JNIHandleBlock. The GC closures should be prepared to handle a field in a Java object becoming NULL, so this should be fine. Bug: https://bugs.openjdk.java.net/browse/JDK-8168914 Webrev: http://cr.openjdk.java.net/~ehelin/8168914/00/ Testing: - Tier 1 (aka JPRT), 2, 3, 4, 5. I would appreciate quite a few reviews on this patch, it contains a nice mixture of: - me venturing into the runtime code :) - lock free code - unreproducible bug (even though I'm sure of the root cause) Thanks for everyone participating in the discussions around this bug and potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and Jiangli. Thanks! Erik [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From yumin.qi at gmail.com Tue Feb 14 16:43:36 2017 From: yumin.qi at gmail.com (yumin qi) Date: Tue, 14 Feb 2017 08:43:36 -0800 Subject: TRACESPINNING in taskqueue.hpp Message-ID: Hi, In file taskqueue.hpp, first #undef TRACESPINNING The next whatever in #ifdef TRACESPINNING will not be included in build. Any comment? Thanks Yumin // A class to aid in the termination of a set of parallel tasks using // TaskQueueSet's for work stealing. #undef TRACESPINNING class ParallelTaskTerminator: public StackObj { protected: uint _n_threads; TaskQueueSetSuper* _queue_set; uint _offered_termination; #ifdef TRACESPINNING static uint _total_yields; static uint _total_spins; static uint _total_peeks; #endif From joseph.provino at oracle.com Tue Feb 14 18:14:45 2017 From: joseph.provino at oracle.com (Joseph Provino) Date: Tue, 14 Feb 2017 13:14:45 -0500 Subject: RFR (XXS): JDK-8138610 add assert to ThreadLocalAllocBuffer::clear_before_allocation to check the storage above top In-Reply-To: References: Message-ID: Hi David, I sent it here: hotspot-dev at openjdk.java.net That's right isn't it? joe On 2/13/2017 1:57 PM, Joseph Provino wrote: > > Please review this one line change: > > CR: JDK-8138610 add > assert to ThreadLocalAllocBuffer::clear_before_allocation to check the > storage above top > > Webrev: http://cr.openjdk.java.net/~jprovino/8138610/webrev.00 > > Tests: Passes JPRT. > > thanks. > > joe > From thomas.schatzl at oracle.com Tue Feb 14 18:45:00 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 14 Feb 2017 19:45:00 +0100 Subject: TRACESPINNING in taskqueue.hpp In-Reply-To: References: Message-ID: <1487097900.2687.2.camel@oracle.com> Hi Yumin, On Tue, 2017-02-14 at 08:43 -0800, yumin qi wrote: > Hi, > > ? In file taskqueue.hpp, first #undef??TRACESPINNING > ? The next whatever in #ifdef TRACESPINNING will not be included in > build. > > ? Any comment? ? not sure what you are asking here. Apparently anyone interested in using this code is supposed to change the #undef to #define. Similar to other performance debug code in hotspot. Thanks, ? Thomas From george.triantafillou at oracle.com Tue Feb 14 19:12:02 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 14 Feb 2017 14:12:02 -0500 Subject: RFR (XXS): JDK-8174855 Quarantine failing test jdk/test/sun/management/HotspotRuntimeMBean/GetSafepointSyncTime.java Message-ID: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> Please review this very small fix for JDK-8174855: JBS: https://bugs.openjdk.java.net/browse/JDK-8174855 open webrev: http://cr.openjdk.java.net/~gtriantafill/8174855/webrev/ The test was added to ProblemList.txt. Thanks. -George From yumin.qi at gmail.com Tue Feb 14 21:02:22 2017 From: yumin.qi at gmail.com (yumin qi) Date: Tue, 14 Feb 2017 13:02:22 -0800 Subject: TRACESPINNING in taskqueue.hpp In-Reply-To: <1487097900.2687.2.camel@oracle.com> References: <1487097900.2687.2.camel@oracle.com> Message-ID: Thomas, I don't think it is a good the way to change #undef to #define to include the tracing code since the #ifdef used in several files(in this case, turn on TRACESPINNING in .hpp so it will be available in other .cpp or .hpp files which include taskqueue.hpp) . We can change product.make to have -DTRACESPINNING in gcc flag, but the def will be cancelled here. To have def in makefile turns on the definition everywhere in code but changing #undef to #define in a file only change one file --- if multiple files contains the def, they need changes too. Thanks Yumin On Tue, Feb 14, 2017 at 10:45 AM, Thomas Schatzl wrote: > Hi Yumin, > > On Tue, 2017-02-14 at 08:43 -0800, yumin qi wrote: > > Hi, > > > > In file taskqueue.hpp, first #undef TRACESPINNING > > The next whatever in #ifdef TRACESPINNING will not be included in > > build. > > > > Any comment? > > not sure what you are asking here. > > Apparently anyone interested in using this code is supposed to change > the #undef to #define. Similar to other performance debug code in > hotspot. > > Thanks, > Thomas > > From kim.barrett at oracle.com Tue Feb 14 22:10:01 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 14 Feb 2017 17:10:01 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <214376fc-ebae-4f0e-2074-1a3595ce5e1e@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> <214376fc-ebae-4f0e-2074-1a3595ce5e1e@oracle.com> Message-ID: <7712EEA3-E080-4153-83A4-70455F830C7D@oracle.com> > On Feb 14, 2017, at 11:26 AM, Mikael Gerdin wrote: > > Hi Kim, > > On 2017-02-13 05:25, Kim Barrett wrote: >> Here's the updated webrevs, incorporating feedback received so far. >> >> full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ >> incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ > > I think your change is ready to go, consider it Reviewed by me. Thanks. > > I can't really say that I think this looks good due to the fact that I'm still traumatized by JavaCallArguments and how it uses something I've come to think of as "alternative code" to handle oop parameters but that is something that I hope we can clean up at some point in the future. > > Thanks for fixing this! > /Mikael > >> >> For x86 and ARM I added resolve_jobject functions to their respective >> macroAssembler files, and call those to generate the return object >> handling code. Merging the two cases for Sparc is harder and I'm >> going to leave it to someone else. I'm also leaving any such merging >> for other (non-Oracle) ports to their respective maintainers. >> >> Rewrote the error checking in SignatureChekker::check_obj to simplify >> the control flow and provide more information. Rather than >> conditionally selecting between ReportJNIFatalError or assert to >> report problems, instead just always use guarantee. Recall that we >> only get here if CheckJNICalls or in a debug build. This made >> SignatureChekker::_thread unused, so removed it. From coleen.phillimore at oracle.com Tue Feb 14 22:21:12 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 14 Feb 2017 17:21:12 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> Message-ID: <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> New webrev: open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev see coments below first. On 2/13/17 11:51 PM, David Holmes wrote: > Hi Coleen, > > On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >> >> >> On 2/13/17 8:44 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> "actually S"? :) >> >> Well, maybe not but it shouldn't have a lot of merge conflicts for >> backports. >> >>> >>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>> Thread::current() call. >>> >>> Well there's a bit more to it than that as you also change parameter >>> types so that we (sometimes) avoid: >>> >>> - oop -> Handle -> oop >>> - Handle -> oop -> Handle >> >> Yes, I put that in the bug but not in the RFR. The recommendation is to >> Handle the oop as far up as possible in the call stack and pass Handle >> around. > > I hear what you are saying but it is hard to actually see that > reflected in the code. > >>> across method calls. This leads to some puzzling differences eg: >>> >>> src/share/vm/aot/aotCodeHeap.cpp >>> src/share/vm/classfile/javaClasses.hpp >>> >>> The change in aotCodeHeap.cpp reverses the use of Handle and oop, but >>> it is unclear why Throwable::print and Throwable::print_stack_trace >>> have different signatures in that regard. They used to both take a >>> Handle but now one takes an oop instead. ?? >> >> Yes, you found the exceptions in Throwable. A lot of the >> java_lang_Throwable (and other javaClasses.hpp classes) pass oop instead >> of Handle and I changed print to oop to make it consistent, and because >> they it being called from other functions in Throwable with oop. >> Throwable::print_stack_trace actually goes to a safepoint so has to be >> Handle. > > I think in javaClasses.hpp the rule should be oop unless Handle is > definitely needed. It's easiest to keep it an oop because most of the functions just do an obj_field, etc. oop java_lang_Throwable::message(oop throwable) { return throwable->obj_field(detailMessage_offset); } but it's likely that the oop throwable is a handle up the call stack. Interestingly looking at it's callers is one in javaClasses.cpp that's an oop, one in exceptions.cpp that's a Handle, oop msg = java_lang_Throwable::message(exception()); And then there's the one in universe.cpp that's a naked oop, which can get moved before the call. // get the error object at the slot and set set it to NULL so that the // array isn't keeping it alive anymore. oop exc = preallocated_out_of_memory_errors()->obj_at(next); assert(exc != NULL, "slot has been used already"); preallocated_out_of_memory_errors()->obj_at_put(next, NULL); <= This on G1 can take out a lock for the oop_store and GC and move the default_err // use the message from the default error oop msg = java_lang_Throwable::message(default_err); <= default_err is an oop. assert(msg != NULL, "no message"); java_lang_Throwable::set_message(exc, msg); exc is a naked oop too. I don't think the naked oop detector triggers for obj_at_put/oop_store because G1 wasn't in the code when it was added. Creating a Handle and passing it saves having to visually inspect functions to try to find naked oops. Fixing this to have a local handle though because the callers don't have THREAD. > >> >>> >>> More generally it is unclear why you made the changes you did in >>> places - see below for some specifics. I'm left questioning how to >>> know when to take an oop and when to take a Handle. Of course that >>> question already exists, but these changes didn't make it any clearer >>> for me. >> >> Pass a Handle early and often is still the rule. For all but GC code. >> There are still some functions that are very small and don't safepoint >> and take oop. Some of these have migrated to taking Handle, but some >> haven't. I didn't want to make this change larger for now. >> >>> >>> I understand why the oop to Handle implicit conversion could be >>> problematic due to the Thread::current() calls, but I don't understand >>> why you couldn't keep (? add?) the Handle to oop implicit >>> conversions ?? >> >> There weren't Handle -> oop implicit conversions. The conversion back >> to oop uses the operator() which remains. > > Okay then it seems an implicit Handle -> oop would avoid the need to > change the caller from foo to foo() if the underlying API switched > from oop to Handle. Yes but it's better to just pass the oop along until something really needs to do something with it. > >> The oop -> Handle conversion has cost, both at runtime and also to the >> GC because it increases the number of Handles on the >> Thread->_handle_area. > > Sure. > >> I did this same experiment years ago when Thread::current() was at the >> top of the profiling call stack but back then I needed to add too many >> explicit Thread::current() calls. There aren't many in this change, so >> it's worth doing. Also, metadata handles have the same problem but >> there are still too many of them to remove the implicit conversion. >> And {instance}KlassHandle needs to be removed first. They do nothing >> now but hide the type. >> >>> >>> --- >>> >>> src/share/vm/c1/c1_Runtime1.cpp >>> >>> Aside: I'm a little surprised that there was not a Handle assignment >>> operator or copy constructor such that given: >>> >>> 863 Handle appendix(THREAD, NULL); >>> 970 appendix = cpce->appendix_if_resolved(pool); >>> >>> that it simply updated the NULL to the desired value, while keeping >>> the THREAD intact. >> >> Handle doesn't save the THREAD so a NULL handle is just oop* NULL. The >> RHS of the expression has to be converted to Handle and the default >> assignment operator copies it to appendix. > > I had assumed Handle saved the Thread. No, then it would be two fields. It's only one so passing it is cheap now. > >>> >>> --- >>> >>> src/share/vm/ci/ciInstance.cpp >>> >>> Not at all obvious that replacing Handle with raw oop is correct. I'm >>> not saying it isn't, just that it isn't obvious - and I'll wonder why >>> the Handle was used in the first place. >> >> It is safe because it's immediately unhandled in all the case >> statements, and I wanted to avoid a Thread::current call. >>> >>> --- >>> >>> src/share/vm/classfile/javaClasses.cpp >>> >>> Why change java_lang_String::as_symbol_or_null to take a Handle >>> instead of an oop if you are simply going to unwrap the Handle to get >>> the oop back. ??? Particular when a caller like >>> src/share/vm/prims/methodHandles.cpp has to create the Handle from oop >>> in the first place. >> >> I changed it to be the same as java_lang_String::as_symbol(Handle >> java_string ...) which took a handle. java_lang_String::as_symbol() was >> the same - it immediately unhandles the argument to pass to the rest of >> the functions. I changed java_lang_String::as_symbol() to take an oop >> to match, and there are few scattered changes from that. > > I have to disagree with this one. AFAICS there is no reason this: > > 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { > > needs to take a Handle in the first place. The java_string is > unwrapped to get the oop and the oop is only passed to a few other > String methods and then not touched. Looking at the API's in > javaClass.hpp the norm should be to take oop (as the bulk of methods > do) with Handle only used when actually required. I changed this one to take oop because it was obvious on inspection but note that the "required" is sometimes really hard to see! Chasing naked oop problems isn't something we do very much anymore because we converted many arguments to Handles. Note that the runtime only passes oops much of the time and doesn't usually do anything with them. The exception is javaClasses.cpp though. > >> >>> >>> --- >>> >>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>> >>> It is not at all obvious that JavaArgumentUnboxers are >>> thread-confined! I'm also concerned by API's (unfortunately a number >>> pre-existing) that take a Thread/JavaThread argument but really >>> require it to be the current thread. To that end I'd rather see >>> _thread as _cur_thread and an assertion when it is set. >> >> Since it's a ResourceObj and not StackObj (setting aside debate about >> that), I agree with you. I changed it to use Thread::current() where it >> handles the object, which it explicitly did before. In this case, it is >> a stack allocated thing but it's not guaranteed to be that. > > Ok. > >>> >>> --- >>> >>> src/share/vm/oops/cpCache.cpp >>> >>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>> >>> Please remove space before ( >> >> fixed. >> >>> >>> --- >>> >>> src/share/vm/prims/jvm.cpp >>> >>> Nit: need to fix indent on StackWalk::walk call second line. >> >> fixed. >> >>> >>> --- >>> >>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>> >>> Ditto re _thread -> _cur_thread >>> >> >> This one is a StackObj (a Closure which should be used that way). So I >> changed _thread to _cur_thread and added an assert that it == >> Thread::current() in the constructor. > > Thanks. > >>> --- >>> >>> src/share/vm/prims/jvmtiImpl.cpp >>> >>> Nit: space before ( again (on handle constructor) >> >> fixed. >>> >>>> It's a very small change to a number of files. Notably the new JVMCI >>>> files have a lot of implicit conversions from oop to handle. I want to >>>> get this change in early for jdk10 to stop this usage. I also added a >>>> few HandleMarks where I found that the HandlesArea was growing large. >>> >>> The addition and removal of the HandleMarks seems problematic because >>> there are no obvious rules being applied. It is hard to know where and >>> when a HandleMark is needed or not. Looking at >>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>> create/destroy the HandleMark on each loop iteration? That would only >>> seem necessary if we know the number of iterations is so high that we >>> might consume all our handle space. >> >> Nobody knows when and where to put HandleMarks. It's a dark art. I > > So a trade-off between too much cleanup overhead if we have too many, > and too much GC overhead if we have too few. And no real way to know > either way - Ouch! :( > >> also was testing with a JVM that restored the assert if HandleArea got >> over 200 handles and the numbers got very high in these functions. The >> GC has to walk these HandleAreas and I am trying to reduce work for >> them. >> >> I tried to write about this in the bug. I'd like to see if changing >> Handle to a proper constructor/destructor like methodHandles, so that >> only the Handles that are active need to be collected. Then we don't >> need HandleMark and their associated mystery. There are several >> HandleMarks in code that obviously doesn't allocate any Handles. I took >> out several but restored them so that this patch was smaller. To >> change Handle to a scoped object would require changing passing Handle >> as a const reference like I did with methodHandle which is too big of a >> change for early-jdk10. > > Ok. This sounds like a way forward though. In combination with an easy > way to detect when handles are required, this would allow very precise > management of handle lifetimes. Good, I'm glad you think so too. > >> The reason I worked on this RFE was because I was curious to see how >> many oops/Handles we're passing around and using from within the JVM >> these days. It's more than I thought. >> >> Thank you for reviewing this. I'm going to rerun some tests with the >> changes above and send another webrev. > > Ok. Thanks. Thanks for the discussion and review! Coleen > > David > ----- > >> Coleen >> >>> >>> Thanks, >>> David >>> ------ >>> >>>> See bug for more on the motivation of this change. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>> >>>> Tested by running all hotspot jtreg tests with >>>> -XX:+CheckUnhandledOops. >>>> There weren't any unhandled oops amazingly. >>>> >>>> Thanks, >>>> Coleen >> From david.holmes at oracle.com Tue Feb 14 22:24:33 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 08:24:33 +1000 Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: <9d506110-91a1-4827-99a8-e46e9363eb3d@default> References: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> <9d506110-91a1-4827-99a8-e46e9363eb3d@default> Message-ID: <5bec7006-4301-7f2a-88c9-dd685e447d15@oracle.com> Hi Shafi, On 14/02/2017 11:20 PM, Shafi Ahmad wrote: > Hi David, > > Thanks for reviewing it. > > Initially I started with fixed size of local char array but later I changed my mind and make it dynamic. > Let me know if I have to make it local char array like. > > + unsigned int siglength = sig->utf8_length(); > + unsigned int namelength = name->utf8_length(); > + unsigned int length = siglength + namelength + 64; > + char *buff = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, length); > + jio_snprintf(buff, length, > + "Duplicate method name \"%s\" with signature \"%s\" in class file", > + name->as_C_string(), sig->as_klass_external_name()); > + classfile_parse_error("%s %s", buff, CHECK); > } > > to > > + char buff[fixedsize]; // say fixedsize is 512 > + jio_snprintf(buff, 512, > + "Duplicate method name \"%s\" with signature \"%s\" in class file", > + name->as_C_string(), sig->as_klass_external_name()); > + classfile_parse_error("%s %s", buff, CHECK); > } It could still be dynamic, you just need to use NEW_RESOURCE_ARRAY_IN_THREAD_RETURN_NULL and check for a NULL return, and fallback to not including the additional info. But the underlying Exceptions::fthrow uses a local buffer itself (1024 max), so you could just do the same as you propose above. Though it is very annoying to have to allocate a buffer just to pass it through to classfile_parse_error which passes it through to Exceptions::fthrow which is a varargs method. So another possibility is to just add another overload of classfile_parse_error that takes in the two additional strings you want to print. Thanks, David > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Tuesday, February 14, 2017 6:34 PM >> To: Shafi Ahmad ; hotspot- >> dev at openjdk.java.net >> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field >> name&signature in class file" should report the name and signature of the >> field >> >> Hi Shafi, >> >> I'm concerned about the use of NEW_RESOURCE_ARRAY_IN_THREAD. If it >> can't allocate it will abort the VM. That seems like a bad thing to happen. >> >> Thanks, >> David >> >> On 14/02/2017 7:19 PM, Shafi Ahmad wrote: >>> Summary: java.lang.ClassFormatError: Exception "Duplicate field >> name&signature in class file" should report the name and signature of the >> field. >>> >>> It's a very small change to single file. >>> In the current implementation name and signature of duplicate field is >> missing in java.lang.ClassFormatError exception message. >>> Without a field name + signature it is hard to triggering the problem. >>> >>> Webrev link: http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ >>> bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 >>> >>> Testing: jprt and jtreg test. >>> >>> I have verified my changes with the reproduces of >> https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 code >> base as I was not able to write reproducer of current issue. >>> With the fix I am getting below exception message. >>> >>> $ java CreateBadClassFile >>> .foreach() call: Exception in thread "main" java.lang.ClassFormatError: >> Duplicate field name "hasNext" with signature "Ljava.lang.Object;" in class >> file WidgetCollection$1 >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:760) >>> at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>> . . . >>> >>> Thanks, >>> Coleen >>> From david.holmes at oracle.com Tue Feb 14 22:29:02 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 08:29:02 +1000 Subject: RFR (XXS): JDK-8174855 Quarantine failing test jdk/test/sun/management/HotspotRuntimeMBean/GetSafepointSyncTime.java In-Reply-To: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> References: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> Message-ID: <725d25a7-9d7f-4eb6-a14b-726b2521227a@oracle.com> Hi George, On 15/02/2017 5:12 AM, George Triantafillou wrote: > Please review this very small fix for JDK-8174855: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8174855 > open webrev: http://cr.openjdk.java.net/~gtriantafill/8174855/webrev/ > > > The test was added to ProblemList.txt. Thanks. The bug id in the ProblemList.txt is the bug that needs to be fixed to allow the test to be removed from the ProblemList - not the bug id used to add it to the list. David ----- > -George > From coleen.phillimore at oracle.com Tue Feb 14 22:30:53 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 14 Feb 2017 17:30:53 -0500 Subject: RFR (XXS): JDK-8174855 Quarantine failing test jdk/test/sun/management/HotspotRuntimeMBean/GetSafepointSyncTime.java In-Reply-To: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> References: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> Message-ID: <1933ffa8-9ead-7203-8afd-20d48375d598@oracle.com> This looks good. Coleen On 2/14/17 2:12 PM, George Triantafillou wrote: > Please review this very small fix for JDK-8174855: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8174855 > open webrev: http://cr.openjdk.java.net/~gtriantafill/8174855/webrev/ > > > The test was added to ProblemList.txt. Thanks. > > -George > From david.holmes at oracle.com Tue Feb 14 23:03:33 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 09:03:33 +1000 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> Message-ID: Hi Coleen, On 15/02/2017 8:21 AM, coleen.phillimore at oracle.com wrote: > > New webrev: > > open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev Thanks for bearing with me on this. Changes seem fine - thanks for updating javaClasses to not use Handles unnecessarily. Nits: src/share/vm/jvmci/jvmciCodeInstaller.cpp There are quite a few handle declarations with a space between the variable and constructor ( eg: Handle location (THREAD, objects->obj_at(i)); instead of Handle location(THREAD, objects->obj_at(i)); There are over 40 in that file (I'm guessing you had a semi-automated conversion mechanism). --- Nit: src/share/vm/runtime/handles.hpp -// Handle h1(obj); // allocate new handle // Handle h2(thread, obj); // faster allocation when current thread is known // Handle h3; // declare handle only, no allocation occurs the remaining comments don't quite make sense with the deletion of the first line. Perhaps just: // Handle h1(thread, obj); // allocate new handle in thread // Handle h2; // declare handle only, no allocation occurs Thanks. No need to see updated webrev if you do update. David ----- > see coments below first. > > On 2/13/17 11:51 PM, David Holmes wrote: >> Hi Coleen, >> >> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>> >>> >>> On 2/13/17 8:44 PM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> "actually S"? :) >>> >>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>> backports. >>> >>>> >>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>> Thread::current() call. >>>> >>>> Well there's a bit more to it than that as you also change parameter >>>> types so that we (sometimes) avoid: >>>> >>>> - oop -> Handle -> oop >>>> - Handle -> oop -> Handle >>> >>> Yes, I put that in the bug but not in the RFR. The recommendation is to >>> Handle the oop as far up as possible in the call stack and pass Handle >>> around. >> >> I hear what you are saying but it is hard to actually see that >> reflected in the code. >> >>>> across method calls. This leads to some puzzling differences eg: >>>> >>>> src/share/vm/aot/aotCodeHeap.cpp >>>> src/share/vm/classfile/javaClasses.hpp >>>> >>>> The change in aotCodeHeap.cpp reverses the use of Handle and oop, but >>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>> have different signatures in that regard. They used to both take a >>>> Handle but now one takes an oop instead. ?? >>> >>> Yes, you found the exceptions in Throwable. A lot of the >>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop instead >>> of Handle and I changed print to oop to make it consistent, and because >>> they it being called from other functions in Throwable with oop. >>> Throwable::print_stack_trace actually goes to a safepoint so has to be >>> Handle. >> >> I think in javaClasses.hpp the rule should be oop unless Handle is >> definitely needed. > > It's easiest to keep it an oop because most of the functions just do an > obj_field, etc. > > oop java_lang_Throwable::message(oop throwable) { > return throwable->obj_field(detailMessage_offset); > } > > but it's likely that the oop throwable is a handle up the call stack. > Interestingly looking at it's callers is one in javaClasses.cpp that's > an oop, one in exceptions.cpp that's a Handle, > > oop msg = java_lang_Throwable::message(exception()); > > And then there's the one in universe.cpp that's a naked oop, which can > get moved before the call. > > // get the error object at the slot and set set it to NULL so that the > // array isn't keeping it alive anymore. > oop exc = preallocated_out_of_memory_errors()->obj_at(next); > assert(exc != NULL, "slot has been used already"); > preallocated_out_of_memory_errors()->obj_at_put(next, NULL); <= > This on G1 can take out a lock for the oop_store and GC and move the > default_err > > // use the message from the default error > oop msg = java_lang_Throwable::message(default_err); <= > default_err is an oop. > assert(msg != NULL, "no message"); > java_lang_Throwable::set_message(exc, msg); > > exc is a naked oop too. I don't think the naked oop detector triggers > for obj_at_put/oop_store because G1 wasn't in the code when it was added. > > Creating a Handle and passing it saves having to visually inspect > functions to try to find naked oops. > > Fixing this to have a local handle though because the callers don't have > THREAD. >> >>> >>>> >>>> More generally it is unclear why you made the changes you did in >>>> places - see below for some specifics. I'm left questioning how to >>>> know when to take an oop and when to take a Handle. Of course that >>>> question already exists, but these changes didn't make it any clearer >>>> for me. >>> >>> Pass a Handle early and often is still the rule. For all but GC code. >>> There are still some functions that are very small and don't safepoint >>> and take oop. Some of these have migrated to taking Handle, but some >>> haven't. I didn't want to make this change larger for now. >>> >>>> >>>> I understand why the oop to Handle implicit conversion could be >>>> problematic due to the Thread::current() calls, but I don't understand >>>> why you couldn't keep (? add?) the Handle to oop implicit >>>> conversions ?? >>> >>> There weren't Handle -> oop implicit conversions. The conversion back >>> to oop uses the operator() which remains. >> >> Okay then it seems an implicit Handle -> oop would avoid the need to >> change the caller from foo to foo() if the underlying API switched >> from oop to Handle. > > Yes but it's better to just pass the oop along until something really > needs to do something with it. >> >>> The oop -> Handle conversion has cost, both at runtime and also to the >>> GC because it increases the number of Handles on the >>> Thread->_handle_area. >> >> Sure. >> >>> I did this same experiment years ago when Thread::current() was at the >>> top of the profiling call stack but back then I needed to add too many >>> explicit Thread::current() calls. There aren't many in this change, so >>> it's worth doing. Also, metadata handles have the same problem but >>> there are still too many of them to remove the implicit conversion. >>> And {instance}KlassHandle needs to be removed first. They do nothing >>> now but hide the type. >>> >>>> >>>> --- >>>> >>>> src/share/vm/c1/c1_Runtime1.cpp >>>> >>>> Aside: I'm a little surprised that there was not a Handle assignment >>>> operator or copy constructor such that given: >>>> >>>> 863 Handle appendix(THREAD, NULL); >>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>> >>>> that it simply updated the NULL to the desired value, while keeping >>>> the THREAD intact. >>> >>> Handle doesn't save the THREAD so a NULL handle is just oop* NULL. The >>> RHS of the expression has to be converted to Handle and the default >>> assignment operator copies it to appendix. >> >> I had assumed Handle saved the Thread. > > No, then it would be two fields. It's only one so passing it is cheap now. >> >>>> >>>> --- >>>> >>>> src/share/vm/ci/ciInstance.cpp >>>> >>>> Not at all obvious that replacing Handle with raw oop is correct. I'm >>>> not saying it isn't, just that it isn't obvious - and I'll wonder why >>>> the Handle was used in the first place. >>> >>> It is safe because it's immediately unhandled in all the case >>> statements, and I wanted to avoid a Thread::current call. >>>> >>>> --- >>>> >>>> src/share/vm/classfile/javaClasses.cpp >>>> >>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>> instead of an oop if you are simply going to unwrap the Handle to get >>>> the oop back. ??? Particular when a caller like >>>> src/share/vm/prims/methodHandles.cpp has to create the Handle from oop >>>> in the first place. >>> >>> I changed it to be the same as java_lang_String::as_symbol(Handle >>> java_string ...) which took a handle. java_lang_String::as_symbol() was >>> the same - it immediately unhandles the argument to pass to the rest of >>> the functions. I changed java_lang_String::as_symbol() to take an oop >>> to match, and there are few scattered changes from that. >> >> I have to disagree with this one. AFAICS there is no reason this: >> >> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >> >> needs to take a Handle in the first place. The java_string is >> unwrapped to get the oop and the oop is only passed to a few other >> String methods and then not touched. Looking at the API's in >> javaClass.hpp the norm should be to take oop (as the bulk of methods >> do) with Handle only used when actually required. > > I changed this one to take oop because it was obvious on inspection but > note that the "required" is sometimes really hard to see! Chasing naked > oop problems isn't something we do very much anymore because we > converted many arguments to Handles. Note that the runtime only passes > oops much of the time and doesn't usually do anything with them. The > exception is javaClasses.cpp though. > >> >>> >>>> >>>> --- >>>> >>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>> >>>> It is not at all obvious that JavaArgumentUnboxers are >>>> thread-confined! I'm also concerned by API's (unfortunately a number >>>> pre-existing) that take a Thread/JavaThread argument but really >>>> require it to be the current thread. To that end I'd rather see >>>> _thread as _cur_thread and an assertion when it is set. >>> >>> Since it's a ResourceObj and not StackObj (setting aside debate about >>> that), I agree with you. I changed it to use Thread::current() where it >>> handles the object, which it explicitly did before. In this case, it is >>> a stack allocated thing but it's not guaranteed to be that. >> >> Ok. >> >>>> >>>> --- >>>> >>>> src/share/vm/oops/cpCache.cpp >>>> >>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>> >>>> Please remove space before ( >>> >>> fixed. >>> >>>> >>>> --- >>>> >>>> src/share/vm/prims/jvm.cpp >>>> >>>> Nit: need to fix indent on StackWalk::walk call second line. >>> >>> fixed. >>> >>>> >>>> --- >>>> >>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>> >>>> Ditto re _thread -> _cur_thread >>>> >>> >>> This one is a StackObj (a Closure which should be used that way). So I >>> changed _thread to _cur_thread and added an assert that it == >>> Thread::current() in the constructor. >> >> Thanks. >> >>>> --- >>>> >>>> src/share/vm/prims/jvmtiImpl.cpp >>>> >>>> Nit: space before ( again (on handle constructor) >>> >>> fixed. >>>> >>>>> It's a very small change to a number of files. Notably the new JVMCI >>>>> files have a lot of implicit conversions from oop to handle. I want to >>>>> get this change in early for jdk10 to stop this usage. I also added a >>>>> few HandleMarks where I found that the HandlesArea was growing large. >>>> >>>> The addition and removal of the HandleMarks seems problematic because >>>> there are no obvious rules being applied. It is hard to know where and >>>> when a HandleMark is needed or not. Looking at >>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>> create/destroy the HandleMark on each loop iteration? That would only >>>> seem necessary if we know the number of iterations is so high that we >>>> might consume all our handle space. >>> >>> Nobody knows when and where to put HandleMarks. It's a dark art. I >> >> So a trade-off between too much cleanup overhead if we have too many, >> and too much GC overhead if we have too few. And no real way to know >> either way - Ouch! :( >> >>> also was testing with a JVM that restored the assert if HandleArea got >>> over 200 handles and the numbers got very high in these functions. The >>> GC has to walk these HandleAreas and I am trying to reduce work for >>> them. >>> >>> I tried to write about this in the bug. I'd like to see if changing >>> Handle to a proper constructor/destructor like methodHandles, so that >>> only the Handles that are active need to be collected. Then we don't >>> need HandleMark and their associated mystery. There are several >>> HandleMarks in code that obviously doesn't allocate any Handles. I took >>> out several but restored them so that this patch was smaller. To >>> change Handle to a scoped object would require changing passing Handle >>> as a const reference like I did with methodHandle which is too big of a >>> change for early-jdk10. >> >> Ok. This sounds like a way forward though. In combination with an easy >> way to detect when handles are required, this would allow very precise >> management of handle lifetimes. > > Good, I'm glad you think so too. >> >>> The reason I worked on this RFE was because I was curious to see how >>> many oops/Handles we're passing around and using from within the JVM >>> these days. It's more than I thought. >>> >>> Thank you for reviewing this. I'm going to rerun some tests with the >>> changes above and send another webrev. >> >> Ok. Thanks. > > Thanks for the discussion and review! > > Coleen >> >> David >> ----- >> >>> Coleen >>> >>>> >>>> Thanks, >>>> David >>>> ------ >>>> >>>>> See bug for more on the motivation of this change. >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>> >>>>> Tested by running all hotspot jtreg tests with >>>>> -XX:+CheckUnhandledOops. >>>>> There weren't any unhandled oops amazingly. >>>>> >>>>> Thanks, >>>>> Coleen >>> > From coleen.phillimore at oracle.com Tue Feb 14 23:16:50 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 14 Feb 2017 18:16:50 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> Message-ID: <539bdcbd-ee20-5ebd-59a1-f573abb583a6@oracle.com> On 2/14/17 6:03 PM, David Holmes wrote: > Hi Coleen, > > On 15/02/2017 8:21 AM, coleen.phillimore at oracle.com wrote: >> >> New webrev: >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev > > Thanks for bearing with me on this. Changes seem fine - thanks for > updating javaClasses to not use Handles unnecessarily. Thanks for being brave enough to look at this change! > > Nits: src/share/vm/jvmci/jvmciCodeInstaller.cpp > > There are quite a few handle declarations with a space between the > variable and constructor ( eg: > > Handle location (THREAD, objects->obj_at(i)); > > instead of > > Handle location(THREAD, objects->obj_at(i)); > > There are over 40 in that file (I'm guessing you had a semi-automated > conversion mechanism). > Nah, I just got tired of typing. I'll fix them. > --- > > Nit: src/share/vm/runtime/handles.hpp > > -// Handle h1(obj); // allocate new handle > // Handle h2(thread, obj); // faster allocation when current > thread is known > // Handle h3; // declare handle only, no > allocation occurs > > the remaining comments don't quite make sense with the deletion of the > first line. Perhaps just: > > // Handle h1(thread, obj); // allocate new handle in thread > // Handle h2; // declare handle only, no > allocation occurs > ok, I'll change as suggested. > Thanks. No need to see updated webrev if you do update. > Thanks! Coleen > David > ----- > >> see coments below first. >> >> On 2/13/17 11:51 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>>> >>>> >>>> On 2/13/17 8:44 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> "actually S"? :) >>>> >>>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>>> backports. >>>> >>>>> >>>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>>> Thread::current() call. >>>>> >>>>> Well there's a bit more to it than that as you also change parameter >>>>> types so that we (sometimes) avoid: >>>>> >>>>> - oop -> Handle -> oop >>>>> - Handle -> oop -> Handle >>>> >>>> Yes, I put that in the bug but not in the RFR. The recommendation >>>> is to >>>> Handle the oop as far up as possible in the call stack and pass Handle >>>> around. >>> >>> I hear what you are saying but it is hard to actually see that >>> reflected in the code. >>> >>>>> across method calls. This leads to some puzzling differences eg: >>>>> >>>>> src/share/vm/aot/aotCodeHeap.cpp >>>>> src/share/vm/classfile/javaClasses.hpp >>>>> >>>>> The change in aotCodeHeap.cpp reverses the use of Handle and oop, but >>>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>>> have different signatures in that regard. They used to both take a >>>>> Handle but now one takes an oop instead. ?? >>>> >>>> Yes, you found the exceptions in Throwable. A lot of the >>>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop >>>> instead >>>> of Handle and I changed print to oop to make it consistent, and >>>> because >>>> they it being called from other functions in Throwable with oop. >>>> Throwable::print_stack_trace actually goes to a safepoint so has to be >>>> Handle. >>> >>> I think in javaClasses.hpp the rule should be oop unless Handle is >>> definitely needed. >> >> It's easiest to keep it an oop because most of the functions just do an >> obj_field, etc. >> >> oop java_lang_Throwable::message(oop throwable) { >> return throwable->obj_field(detailMessage_offset); >> } >> >> but it's likely that the oop throwable is a handle up the call stack. >> Interestingly looking at it's callers is one in javaClasses.cpp that's >> an oop, one in exceptions.cpp that's a Handle, >> >> oop msg = java_lang_Throwable::message(exception()); >> >> And then there's the one in universe.cpp that's a naked oop, which can >> get moved before the call. >> >> // get the error object at the slot and set set it to NULL so >> that the >> // array isn't keeping it alive anymore. >> oop exc = preallocated_out_of_memory_errors()->obj_at(next); >> assert(exc != NULL, "slot has been used already"); >> preallocated_out_of_memory_errors()->obj_at_put(next, NULL); <= >> This on G1 can take out a lock for the oop_store and GC and move the >> default_err >> >> // use the message from the default error >> oop msg = java_lang_Throwable::message(default_err); <= >> default_err is an oop. >> assert(msg != NULL, "no message"); >> java_lang_Throwable::set_message(exc, msg); >> >> exc is a naked oop too. I don't think the naked oop detector triggers >> for obj_at_put/oop_store because G1 wasn't in the code when it was >> added. >> >> Creating a Handle and passing it saves having to visually inspect >> functions to try to find naked oops. >> >> Fixing this to have a local handle though because the callers don't have >> THREAD. >>> >>>> >>>>> >>>>> More generally it is unclear why you made the changes you did in >>>>> places - see below for some specifics. I'm left questioning how to >>>>> know when to take an oop and when to take a Handle. Of course that >>>>> question already exists, but these changes didn't make it any clearer >>>>> for me. >>>> >>>> Pass a Handle early and often is still the rule. For all but GC code. >>>> There are still some functions that are very small and don't safepoint >>>> and take oop. Some of these have migrated to taking Handle, but some >>>> haven't. I didn't want to make this change larger for now. >>>> >>>>> >>>>> I understand why the oop to Handle implicit conversion could be >>>>> problematic due to the Thread::current() calls, but I don't >>>>> understand >>>>> why you couldn't keep (? add?) the Handle to oop implicit >>>>> conversions ?? >>>> >>>> There weren't Handle -> oop implicit conversions. The conversion back >>>> to oop uses the operator() which remains. >>> >>> Okay then it seems an implicit Handle -> oop would avoid the need to >>> change the caller from foo to foo() if the underlying API switched >>> from oop to Handle. >> >> Yes but it's better to just pass the oop along until something really >> needs to do something with it. >>> >>>> The oop -> Handle conversion has cost, both at runtime and also to the >>>> GC because it increases the number of Handles on the >>>> Thread->_handle_area. >>> >>> Sure. >>> >>>> I did this same experiment years ago when Thread::current() was at the >>>> top of the profiling call stack but back then I needed to add too many >>>> explicit Thread::current() calls. There aren't many in this >>>> change, so >>>> it's worth doing. Also, metadata handles have the same problem but >>>> there are still too many of them to remove the implicit conversion. >>>> And {instance}KlassHandle needs to be removed first. They do nothing >>>> now but hide the type. >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/c1/c1_Runtime1.cpp >>>>> >>>>> Aside: I'm a little surprised that there was not a Handle assignment >>>>> operator or copy constructor such that given: >>>>> >>>>> 863 Handle appendix(THREAD, NULL); >>>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>>> >>>>> that it simply updated the NULL to the desired value, while keeping >>>>> the THREAD intact. >>>> >>>> Handle doesn't save the THREAD so a NULL handle is just oop* NULL. The >>>> RHS of the expression has to be converted to Handle and the default >>>> assignment operator copies it to appendix. >>> >>> I had assumed Handle saved the Thread. >> >> No, then it would be two fields. It's only one so passing it is >> cheap now. >>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/ci/ciInstance.cpp >>>>> >>>>> Not at all obvious that replacing Handle with raw oop is correct. I'm >>>>> not saying it isn't, just that it isn't obvious - and I'll wonder why >>>>> the Handle was used in the first place. >>>> >>>> It is safe because it's immediately unhandled in all the case >>>> statements, and I wanted to avoid a Thread::current call. >>>>> >>>>> --- >>>>> >>>>> src/share/vm/classfile/javaClasses.cpp >>>>> >>>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>>> instead of an oop if you are simply going to unwrap the Handle to get >>>>> the oop back. ??? Particular when a caller like >>>>> src/share/vm/prims/methodHandles.cpp has to create the Handle from >>>>> oop >>>>> in the first place. >>>> >>>> I changed it to be the same as java_lang_String::as_symbol(Handle >>>> java_string ...) which took a handle. java_lang_String::as_symbol() >>>> was >>>> the same - it immediately unhandles the argument to pass to the >>>> rest of >>>> the functions. I changed java_lang_String::as_symbol() to take an oop >>>> to match, and there are few scattered changes from that. >>> >>> I have to disagree with this one. AFAICS there is no reason this: >>> >>> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >>> >>> needs to take a Handle in the first place. The java_string is >>> unwrapped to get the oop and the oop is only passed to a few other >>> String methods and then not touched. Looking at the API's in >>> javaClass.hpp the norm should be to take oop (as the bulk of methods >>> do) with Handle only used when actually required. >> >> I changed this one to take oop because it was obvious on inspection but >> note that the "required" is sometimes really hard to see! Chasing naked >> oop problems isn't something we do very much anymore because we >> converted many arguments to Handles. Note that the runtime only passes >> oops much of the time and doesn't usually do anything with them. The >> exception is javaClasses.cpp though. >> >>> >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>>> >>>>> It is not at all obvious that JavaArgumentUnboxers are >>>>> thread-confined! I'm also concerned by API's (unfortunately a number >>>>> pre-existing) that take a Thread/JavaThread argument but really >>>>> require it to be the current thread. To that end I'd rather see >>>>> _thread as _cur_thread and an assertion when it is set. >>>> >>>> Since it's a ResourceObj and not StackObj (setting aside debate about >>>> that), I agree with you. I changed it to use Thread::current() >>>> where it >>>> handles the object, which it explicitly did before. In this case, >>>> it is >>>> a stack allocated thing but it's not guaranteed to be that. >>> >>> Ok. >>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/oops/cpCache.cpp >>>>> >>>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>>> >>>>> Please remove space before ( >>>> >>>> fixed. >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jvm.cpp >>>>> >>>>> Nit: need to fix indent on StackWalk::walk call second line. >>>> >>>> fixed. >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>>> >>>>> Ditto re _thread -> _cur_thread >>>>> >>>> >>>> This one is a StackObj (a Closure which should be used that way). So I >>>> changed _thread to _cur_thread and added an assert that it == >>>> Thread::current() in the constructor. >>> >>> Thanks. >>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jvmtiImpl.cpp >>>>> >>>>> Nit: space before ( again (on handle constructor) >>>> >>>> fixed. >>>>> >>>>>> It's a very small change to a number of files. Notably the new >>>>>> JVMCI >>>>>> files have a lot of implicit conversions from oop to handle. I >>>>>> want to >>>>>> get this change in early for jdk10 to stop this usage. I also >>>>>> added a >>>>>> few HandleMarks where I found that the HandlesArea was growing >>>>>> large. >>>>> >>>>> The addition and removal of the HandleMarks seems problematic because >>>>> there are no obvious rules being applied. It is hard to know where >>>>> and >>>>> when a HandleMark is needed or not. Looking at >>>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>>> create/destroy the HandleMark on each loop iteration? That would only >>>>> seem necessary if we know the number of iterations is so high that we >>>>> might consume all our handle space. >>>> >>>> Nobody knows when and where to put HandleMarks. It's a dark art. I >>> >>> So a trade-off between too much cleanup overhead if we have too many, >>> and too much GC overhead if we have too few. And no real way to know >>> either way - Ouch! :( >>> >>>> also was testing with a JVM that restored the assert if HandleArea got >>>> over 200 handles and the numbers got very high in these functions. >>>> The >>>> GC has to walk these HandleAreas and I am trying to reduce work for >>>> them. >>>> >>>> I tried to write about this in the bug. I'd like to see if changing >>>> Handle to a proper constructor/destructor like methodHandles, so that >>>> only the Handles that are active need to be collected. Then we don't >>>> need HandleMark and their associated mystery. There are several >>>> HandleMarks in code that obviously doesn't allocate any Handles. I >>>> took >>>> out several but restored them so that this patch was smaller. To >>>> change Handle to a scoped object would require changing passing Handle >>>> as a const reference like I did with methodHandle which is too big >>>> of a >>>> change for early-jdk10. >>> >>> Ok. This sounds like a way forward though. In combination with an easy >>> way to detect when handles are required, this would allow very precise >>> management of handle lifetimes. >> >> Good, I'm glad you think so too. >>> >>>> The reason I worked on this RFE was because I was curious to see how >>>> many oops/Handles we're passing around and using from within the JVM >>>> these days. It's more than I thought. >>>> >>>> Thank you for reviewing this. I'm going to rerun some tests with the >>>> changes above and send another webrev. >>> >>> Ok. Thanks. >> >> Thanks for the discussion and review! >> >> Coleen >>> >>> David >>> ----- >>> >>>> Coleen >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> ------ >>>>> >>>>>> See bug for more on the motivation of this change. >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>>> >>>>>> Tested by running all hotspot jtreg tests with >>>>>> -XX:+CheckUnhandledOops. >>>>>> There weren't any unhandled oops amazingly. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>> >> From coleen.phillimore at oracle.com Tue Feb 14 23:31:09 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 14 Feb 2017 18:31:09 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <214376fc-ebae-4f0e-2074-1a3595ce5e1e@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> <214376fc-ebae-4f0e-2074-1a3595ce5e1e@oracle.com> Message-ID: <154b492a-98b8-de1f-5166-d47a303bd462@oracle.com> On 2/14/17 11:26 AM, Mikael Gerdin wrote: > Hi Kim, > > On 2017-02-13 05:25, Kim Barrett wrote: >> Here's the updated webrevs, incorporating feedback received so far. >> >> full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ >> incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ > > I think your change is ready to go, consider it Reviewed by me. > > I can't really say that I think this looks good due to the fact that > I'm still traumatized by JavaCallArguments and how it uses something > I've come to think of as "alternative code" to handle oop parameters > but that is something that I hope we can clean up at some point in the > future. I agree. Also, I'm traumatized by the external_guard template parameter in jniHandles.hpp. Kim, thank you for making the changes I suggested, especially in the x86 code. It looks good to me. Coleen > > Thanks for fixing this! > /Mikael > >> >> For x86 and ARM I added resolve_jobject functions to their respective >> macroAssembler files, and call those to generate the return object >> handling code. Merging the two cases for Sparc is harder and I'm >> going to leave it to someone else. I'm also leaving any such merging >> for other (non-Oracle) ports to their respective maintainers. >> >> Rewrote the error checking in SignatureChekker::check_obj to simplify >> the control flow and provide more information. Rather than >> conditionally selecting between ReportJNIFatalError or assert to >> report problems, instead just always use guarantee. Recall that we >> only get here if CheckJNICalls or in a debug build. This made >> SignatureChekker::_thread unused, so removed it. >> From kim.barrett at oracle.com Wed Feb 15 00:56:01 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 14 Feb 2017 19:56:01 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <154b492a-98b8-de1f-5166-d47a303bd462@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> <214376fc-ebae-4f0e-2074-1a3595ce5e1e@oracle.com> <154b492a-98b8-de1f-5166-d47a303bd462@oracle.com> Message-ID: > On Feb 14, 2017, at 6:31 PM, coleen.phillimore at oracle.com wrote: > > > > On 2/14/17 11:26 AM, Mikael Gerdin wrote: >> Hi Kim, >> >> On 2017-02-13 05:25, Kim Barrett wrote: >>> Here's the updated webrevs, incorporating feedback received so far. >>> >>> full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ >>> incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ >> >> I think your change is ready to go, consider it Reviewed by me. >> >> I can't really say that I think this looks good due to the fact that I'm still traumatized by JavaCallArguments and how it uses something I've come to think of as "alternative code" to handle oop parameters but that is something that I hope we can clean up at some point in the future. > > I agree. Also, I'm traumatized by the external_guard template parameter in jniHandles.hpp. > > Kim, thank you for making the changes I suggested, especially in the x86 code. > > It looks good to me. Thanks. > > Coleen > >> >> Thanks for fixing this! >> /Mikael >> >>> >>> For x86 and ARM I added resolve_jobject functions to their respective >>> macroAssembler files, and call those to generate the return object >>> handling code. Merging the two cases for Sparc is harder and I'm >>> going to leave it to someone else. I'm also leaving any such merging >>> for other (non-Oracle) ports to their respective maintainers. >>> >>> Rewrote the error checking in SignatureChekker::check_obj to simplify >>> the control flow and provide more information. Rather than >>> conditionally selecting between ReportJNIFatalError or assert to >>> report problems, instead just always use guarantee. Recall that we >>> only get here if CheckJNICalls or in a debug build. This made >>> SignatureChekker::_thread unused, so removed it. From kim.barrett at oracle.com Wed Feb 15 01:18:15 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 14 Feb 2017 20:18:15 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> Message-ID: Not a review yet (though I will try), but saw this plus your question to me earlier today... > On Feb 14, 2017, at 5:21 PM, coleen.phillimore at oracle.com wrote: > And then there's the one in universe.cpp that's a naked oop, which can get moved before the call. > > // get the error object at the slot and set set it to NULL so that the > // array isn't keeping it alive anymore. > oop exc = preallocated_out_of_memory_errors()->obj_at(next); > assert(exc != NULL, "slot has been used already"); > preallocated_out_of_memory_errors()->obj_at_put(next, NULL); <= This on G1 can take out a lock for the oop_store and GC and move the default_err I don?t think there?s a problem here. Yes, it takes locks, but always without safepoint checks. From george.triantafillou at oracle.com Wed Feb 15 01:40:00 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 14 Feb 2017 20:40:00 -0500 Subject: RFR (XXS): JDK-8174855 Quarantine failing test jdk/test/sun/management/HotspotRuntimeMBean/GetSafepointSyncTime.java In-Reply-To: <725d25a7-9d7f-4eb6-a14b-726b2521227a@oracle.com> References: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> <725d25a7-9d7f-4eb6-a14b-726b2521227a@oracle.com> Message-ID: Hi David, Thanks for the review. Updated webrev: http://cr.openjdk.java.net/~gtriantafill/8174855/webrev.01/ -George On 2/14/2017 5:29 PM, David Holmes wrote: > Hi George, > > On 15/02/2017 5:12 AM, George Triantafillou wrote: >> Please review this very small fix for JDK-8174855: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8174855 >> open webrev: http://cr.openjdk.java.net/~gtriantafill/8174855/webrev/ >> >> >> The test was added to ProblemList.txt. Thanks. > > The bug id in the ProblemList.txt is the bug that needs to be fixed to > allow the test to be removed from the ProblemList - not the bug id > used to add it to the list. > > David > ----- > >> -George >> From david.holmes at oracle.com Wed Feb 15 01:48:56 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 11:48:56 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> Message-ID: <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> Hi Erik, On 15/02/2017 2:40 AM, Erik Helin wrote: > Hi all, > > this patch aims to solve the bug [0] by making it safe for a GC to > concurrently traverse the oops in a ClassLoaderData. > > The problem right now is that ClassLoaderData::_handles is a > JNIHandleBlock. It is not safe for one thread to iterate over the oops > in a JNIHandleBlock while another thread concurrently adds a new oop to > the JNIHandleBlock. > > My proposed solution is to replace JNIHandleBlock with another data > structure for ClassLoaderData. ClassLoaderData only needs a place to > store oops that a GC can iterate over, it has no use for the rest of the > methods in the JNIHandleBlock class. I have implemented a minimal > chunked linked list data structure (internal to ClassLoaderData) with > only two public methods: > - add (only executed by one thread at a time) > - oops_do (can be executed by any number of threads, possibly > concurrently with a call to `add`) > > ChunkedHandleList uses load_acquire and release_store to synchronize > access to the underlying chunks (arrays). Since ChunkedHandleList > utilizes the (rather) minimal requirements of its user > (ClassLoaderData), I decided to keep the class internal to > ClassLoaderData for now. If other find it useful elsewhere, the we can > try to create a more generalized version (this is trickier than it > appears at first sight, I tried ;)). > > I also changed ClassLoaderData::remove_handle to write NULL to the oop* > (that is represented by a jobject), instead of using a sentinel oop as > done by JNIHandleBlock. The GC closures should be prepared to handle a > field in a Java object becoming NULL, so this should be fine. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8168914 > > Webrev: > http://cr.openjdk.java.net/~ehelin/8168914/00/ oops_do can miss an oop that is in the process of being added - is that a problem? 140 if (c->data[i] != NULL) { 141 f->do_oop(&c->data[i]); 142 } Given a slot can be nulled out concurrently at any time, is it worth does this NULL check? The OopClosure will have to do its own NULL check anyway. 144 c = (Chunk*) OrderAccess::load_ptr_acquire((volatile intptr_t*)&c->next); This doesn't need to be a load-acquire. You already loaded 'c' via load-acquire of _head (or chained therefrom) and its next field is set prior to the setting of the _head that you read. 624 jobject ClassLoaderData::add_handle(Handle h) { 625 MutexLockerEx ml(metaspace_lock(), Mutex::_no_safepoint_check_flag); 626 return (jobject) _handles.add(h()); 627 } 628 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { 630 assert(_handles.contains((oop*) h), "Got unexpected handle " PTR_FORMAT, p2i((oop*) h)); 631 *((oop*) h) = NULL; 632 } I'm a bit unclear on the typing here. Previously we use a JNI Handle which was a jobject. But now we've simply stored an oop into a slot in an array. We pass the address of that slot back as a jobject, but is it really? I would also expect contains to take an oop not an oop* - it seems to be asking "is this an address of a slot in our ChunkedHandleList" rather than asking "is this oop in our ChunkedHandleList". ?? A few minor comments: Copyrights need updating. src/share/vm/classfile/classLoaderData.hpp 177 // Only on thread ... Typo: on -> one --- src/share/vm/classfile/moduleEntry.cpp 90 // Create a JNI handle for the shared ProtectionDomain and save it atomically. 91 // If someone beats us setting the _pd cache, the created JNI handle is destroyed. These are no longer JNI Handles. Thanks, David ----- > Testing: > - Tier 1 (aka JPRT), 2, 3, 4, 5. > > I would appreciate quite a few reviews on this patch, it contains a nice > mixture of: > - me venturing into the runtime code :) > - lock free code > - unreproducible bug (even though I'm sure of the root cause) > > Thanks for everyone participating in the discussions around this bug and > potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and Jiangli. > > Thanks! > Erik > > [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From david.holmes at oracle.com Wed Feb 15 01:59:44 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 11:59:44 +1000 Subject: TRACESPINNING in taskqueue.hpp In-Reply-To: <1487097900.2687.2.camel@oracle.com> References: <1487097900.2687.2.camel@oracle.com> Message-ID: On 15/02/2017 4:45 AM, Thomas Schatzl wrote: > Hi Yumin, > > On Tue, 2017-02-14 at 08:43 -0800, yumin qi wrote: >> Hi, >> >> In file taskqueue.hpp, first #undef TRACESPINNING >> The next whatever in #ifdef TRACESPINNING will not be included in >> build. >> >> Any comment? > > not sure what you are asking here. > > Apparently anyone interested in using this code is supposed to change > the #undef to #define. Similar to other performance debug code in > hotspot. Right ... except there's really no reason to have the undef unless TRACESPINNING is normally set by the build - which it isn't. It would be marginally more developer friendly to just be able to add -DTRACESPINNING to the make args than having to edit the file first. Cheers, David > Thanks, > Thomas > From david.holmes at oracle.com Wed Feb 15 02:09:15 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 12:09:15 +1000 Subject: RFR (XXS): JDK-8174855 Quarantine failing test jdk/test/sun/management/HotspotRuntimeMBean/GetSafepointSyncTime.java In-Reply-To: References: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> <725d25a7-9d7f-4eb6-a14b-726b2521227a@oracle.com> Message-ID: <1490077f-7f1c-a6b2-4a9e-e95fe16ae643@oracle.com> On 15/02/2017 11:40 AM, George Triantafillou wrote: > Hi David, > > Thanks for the review. Updated webrev: > > http://cr.openjdk.java.net/~gtriantafill/8174855/webrev.01/ > Looks good! :) Thanks, David > -George > > On 2/14/2017 5:29 PM, David Holmes wrote: >> Hi George, >> >> On 15/02/2017 5:12 AM, George Triantafillou wrote: >>> Please review this very small fix for JDK-8174855: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8174855 >>> open webrev: http://cr.openjdk.java.net/~gtriantafill/8174855/webrev/ >>> >>> >>> The test was added to ProblemList.txt. Thanks. >> >> The bug id in the ProblemList.txt is the bug that needs to be fixed to >> allow the test to be removed from the ProblemList - not the bug id >> used to add it to the list. >> >> David >> ----- >> >>> -George >>> > From david.holmes at oracle.com Wed Feb 15 02:33:43 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 12:33:43 +1000 Subject: TRACESPINNING in taskqueue.hpp In-Reply-To: References: <1487097900.2687.2.camel@oracle.com> Message-ID: <177fa738-215b-5871-6c0f-67339d5dab40@oracle.com> Correction ... On 15/02/2017 11:59 AM, David Holmes wrote: > On 15/02/2017 4:45 AM, Thomas Schatzl wrote: >> Hi Yumin, >> >> On Tue, 2017-02-14 at 08:43 -0800, yumin qi wrote: >>> Hi, >>> >>> In file taskqueue.hpp, first #undef TRACESPINNING >>> The next whatever in #ifdef TRACESPINNING will not be included in >>> build. >>> >>> Any comment? >> >> not sure what you are asking here. >> >> Apparently anyone interested in using this code is supposed to change >> the #undef to #define. Similar to other performance debug code in >> hotspot. I misread the suggestion ... you may need to delete the undef not change it to a define. As Yumin notes this flag is used in multiple files and it isn't clear they all include this header. David > Right ... except there's really no reason to have the undef unless > TRACESPINNING is normally set by the build - which it isn't. It would be > marginally more developer friendly to just be able to add > -DTRACESPINNING to the make args than having to edit the file first. > > Cheers, > David > >> Thanks, >> Thomas >> From shafi.s.ahmad at oracle.com Wed Feb 15 06:02:40 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 14 Feb 2017 22:02:40 -0800 (PST) Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: <5bec7006-4301-7f2a-88c9-dd685e447d15@oracle.com> References: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> <9d506110-91a1-4827-99a8-e46e9363eb3d@default> <5bec7006-4301-7f2a-88c9-dd685e447d15@oracle.com> Message-ID: Hi David, Please find updated webrev at http://cr.openjdk.java.net/~shshahma/8171194/webrev.01/ Added overloaded method classfile_parse_error. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 15, 2017 3:55 AM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > Hi Shafi, > > On 14/02/2017 11:20 PM, Shafi Ahmad wrote: > > Hi David, > > > > Thanks for reviewing it. > > > > Initially I started with fixed size of local char array but later I changed my > mind and make it dynamic. > > Let me know if I have to make it local char array like. > > > > + unsigned int siglength = sig->utf8_length(); > > + unsigned int namelength = name->utf8_length(); > > + unsigned int length = siglength + namelength + 64; > > + char *buff = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, > length); > > + jio_snprintf(buff, length, > > + "Duplicate method name \"%s\" with signature \"%s\" in class > file", > > + name->as_C_string(), sig->as_klass_external_name()); > > + classfile_parse_error("%s %s", buff, CHECK); > > } > > > > to > > > > + char buff[fixedsize]; // say fixedsize is 512 > > + jio_snprintf(buff, 512, > > + "Duplicate method name \"%s\" with signature \"%s\" in class > file", > > + name->as_C_string(), sig->as_klass_external_name()); > > + classfile_parse_error("%s %s", buff, CHECK); > > } > > It could still be dynamic, you just need to use > NEW_RESOURCE_ARRAY_IN_THREAD_RETURN_NULL and check for a NULL > return, and fallback to not including the additional info. But the underlying > Exceptions::fthrow uses a local buffer itself (1024 max), so you could just do > the same as you propose above. > > Though it is very annoying to have to allocate a buffer just to pass it through > to classfile_parse_error which passes it through to Exceptions::fthrow which > is a varargs method. So another possibility is to just add another overload of > classfile_parse_error that takes in the two additional strings you want to > print. > > Thanks, > David > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Tuesday, February 14, 2017 6:34 PM > >> To: Shafi Ahmad ; hotspot- > >> dev at openjdk.java.net > >> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > >> name&signature in class file" should report the name and signature of > >> the field > >> > >> Hi Shafi, > >> > >> I'm concerned about the use of NEW_RESOURCE_ARRAY_IN_THREAD. If > it > >> can't allocate it will abort the VM. That seems like a bad thing to happen. > >> > >> Thanks, > >> David > >> > >> On 14/02/2017 7:19 PM, Shafi Ahmad wrote: > >>> Summary: java.lang.ClassFormatError: Exception "Duplicate field > >> name&signature in class file" should report the name and signature of > >> the field. > >>> > >>> It's a very small change to single file. > >>> In the current implementation name and signature of duplicate field > >>> is > >> missing in java.lang.ClassFormatError exception message. > >>> Without a field name + signature it is hard to triggering the problem. > >>> > >>> Webrev link: > http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ > >>> bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 > >>> > >>> Testing: jprt and jtreg test. > >>> > >>> I have verified my changes with the reproduces of > >> https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 code > >> base as I was not able to write reproducer of current issue. > >>> With the fix I am getting below exception message. > >>> > >>> $ java CreateBadClassFile > >>> .foreach() call: Exception in thread "main" java.lang.ClassFormatError: > >> Duplicate field name "hasNext" with signature "Ljava.lang.Object;" in > >> class file WidgetCollection$1 > >>> at java.lang.ClassLoader.defineClass1(Native Method) > >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > >>> at > >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:14 > >> 2) > >>> . . . > >>> > >>> Thanks, > >>> Coleen > >>> From david.holmes at oracle.com Wed Feb 15 06:23:50 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 16:23:50 +1000 Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: References: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> <9d506110-91a1-4827-99a8-e46e9363eb3d@default> <5bec7006-4301-7f2a-88c9-dd685e447d15@oracle.com> Message-ID: Looks good to me! Nice and neat. Thanks, David On 15/02/2017 4:02 PM, Shafi Ahmad wrote: > Hi David, > > Please find updated webrev at http://cr.openjdk.java.net/~shshahma/8171194/webrev.01/ > Added overloaded method classfile_parse_error. > > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, February 15, 2017 3:55 AM >> To: Shafi Ahmad ; hotspot- >> dev at openjdk.java.net >> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field >> name&signature in class file" should report the name and signature of the >> field >> >> Hi Shafi, >> >> On 14/02/2017 11:20 PM, Shafi Ahmad wrote: >>> Hi David, >>> >>> Thanks for reviewing it. >>> >>> Initially I started with fixed size of local char array but later I changed my >> mind and make it dynamic. >>> Let me know if I have to make it local char array like. >>> >>> + unsigned int siglength = sig->utf8_length(); >>> + unsigned int namelength = name->utf8_length(); >>> + unsigned int length = siglength + namelength + 64; >>> + char *buff = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, >> length); >>> + jio_snprintf(buff, length, >>> + "Duplicate method name \"%s\" with signature \"%s\" in class >> file", >>> + name->as_C_string(), sig->as_klass_external_name()); >>> + classfile_parse_error("%s %s", buff, CHECK); >>> } >>> >>> to >>> >>> + char buff[fixedsize]; // say fixedsize is 512 >>> + jio_snprintf(buff, 512, >>> + "Duplicate method name \"%s\" with signature \"%s\" in class >> file", >>> + name->as_C_string(), sig->as_klass_external_name()); >>> + classfile_parse_error("%s %s", buff, CHECK); >>> } >> >> It could still be dynamic, you just need to use >> NEW_RESOURCE_ARRAY_IN_THREAD_RETURN_NULL and check for a NULL >> return, and fallback to not including the additional info. But the underlying >> Exceptions::fthrow uses a local buffer itself (1024 max), so you could just do >> the same as you propose above. >> >> Though it is very annoying to have to allocate a buffer just to pass it through >> to classfile_parse_error which passes it through to Exceptions::fthrow which >> is a varargs method. So another possibility is to just add another overload of >> classfile_parse_error that takes in the two additional strings you want to >> print. >> >> Thanks, >> David >> >>> Regards, >>> Shafi >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Tuesday, February 14, 2017 6:34 PM >>>> To: Shafi Ahmad ; hotspot- >>>> dev at openjdk.java.net >>>> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field >>>> name&signature in class file" should report the name and signature of >>>> the field >>>> >>>> Hi Shafi, >>>> >>>> I'm concerned about the use of NEW_RESOURCE_ARRAY_IN_THREAD. If >> it >>>> can't allocate it will abort the VM. That seems like a bad thing to happen. >>>> >>>> Thanks, >>>> David >>>> >>>> On 14/02/2017 7:19 PM, Shafi Ahmad wrote: >>>>> Summary: java.lang.ClassFormatError: Exception "Duplicate field >>>> name&signature in class file" should report the name and signature of >>>> the field. >>>>> >>>>> It's a very small change to single file. >>>>> In the current implementation name and signature of duplicate field >>>>> is >>>> missing in java.lang.ClassFormatError exception message. >>>>> Without a field name + signature it is hard to triggering the problem. >>>>> >>>>> Webrev link: >> http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ >>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 >>>>> >>>>> Testing: jprt and jtreg test. >>>>> >>>>> I have verified my changes with the reproduces of >>>> https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 code >>>> base as I was not able to write reproducer of current issue. >>>>> With the fix I am getting below exception message. >>>>> >>>>> $ java CreateBadClassFile >>>>> .foreach() call: Exception in thread "main" java.lang.ClassFormatError: >>>> Duplicate field name "hasNext" with signature "Ljava.lang.Object;" in >>>> class file WidgetCollection$1 >>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:760) >>>>> at >>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:14 >>>> 2) >>>>> . . . >>>>> >>>>> Thanks, >>>>> Coleen >>>>> From shafi.s.ahmad at oracle.com Wed Feb 15 06:29:36 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 14 Feb 2017 22:29:36 -0800 (PST) Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: References: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> <9d506110-91a1-4827-99a8-e46e9363eb3d@default> <5bec7006-4301-7f2a-88c9-dd685e447d15@oracle.com> Message-ID: Thank you David for reviewing it. All, May I get second reviewer for this trivial change. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 15, 2017 11:54 AM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > Looks good to me! Nice and neat. > > Thanks, > David > > On 15/02/2017 4:02 PM, Shafi Ahmad wrote: > > Hi David, > > > > Please find updated webrev at > > http://cr.openjdk.java.net/~shshahma/8171194/webrev.01/ > > Added overloaded method classfile_parse_error. > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Wednesday, February 15, 2017 3:55 AM > >> To: Shafi Ahmad ; hotspot- > >> dev at openjdk.java.net > >> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > >> name&signature in class file" should report the name and signature of > >> the field > >> > >> Hi Shafi, > >> > >> On 14/02/2017 11:20 PM, Shafi Ahmad wrote: > >>> Hi David, > >>> > >>> Thanks for reviewing it. > >>> > >>> Initially I started with fixed size of local char array but later I > >>> changed my > >> mind and make it dynamic. > >>> Let me know if I have to make it local char array like. > >>> > >>> + unsigned int siglength = sig->utf8_length(); > >>> + unsigned int namelength = name->utf8_length(); > >>> + unsigned int length = siglength + namelength + 64; > >>> + char *buff = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, > >> length); > >>> + jio_snprintf(buff, length, > >>> + "Duplicate method name \"%s\" with signature > >>> + \"%s\" in class > >> file", > >>> + name->as_C_string(), sig->as_klass_external_name()); > >>> + classfile_parse_error("%s %s", buff, CHECK); > >>> } > >>> > >>> to > >>> > >>> + char buff[fixedsize]; // say fixedsize is 512 > >>> + jio_snprintf(buff, 512, > >>> + "Duplicate method name \"%s\" with signature > >>> + \"%s\" in class > >> file", > >>> + name->as_C_string(), sig->as_klass_external_name()); > >>> + classfile_parse_error("%s %s", buff, CHECK); > >>> } > >> > >> It could still be dynamic, you just need to use > >> NEW_RESOURCE_ARRAY_IN_THREAD_RETURN_NULL and check for a > NULL return, > >> and fallback to not including the additional info. But the underlying > >> Exceptions::fthrow uses a local buffer itself (1024 max), so you > >> could just do the same as you propose above. > >> > >> Though it is very annoying to have to allocate a buffer just to pass > >> it through to classfile_parse_error which passes it through to > >> Exceptions::fthrow which is a varargs method. So another possibility > >> is to just add another overload of classfile_parse_error that takes > >> in the two additional strings you want to print. > >> > >> Thanks, > >> David > >> > >>> Regards, > >>> Shafi > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Tuesday, February 14, 2017 6:34 PM > >>>> To: Shafi Ahmad ; hotspot- > >>>> dev at openjdk.java.net > >>>> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > >>>> name&signature in class file" should report the name and signature > >>>> of the field > >>>> > >>>> Hi Shafi, > >>>> > >>>> I'm concerned about the use of NEW_RESOURCE_ARRAY_IN_THREAD. > If > >> it > >>>> can't allocate it will abort the VM. That seems like a bad thing to > happen. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>> On 14/02/2017 7:19 PM, Shafi Ahmad wrote: > >>>>> Summary: java.lang.ClassFormatError: Exception "Duplicate field > >>>> name&signature in class file" should report the name and signature > >>>> of the field. > >>>>> > >>>>> It's a very small change to single file. > >>>>> In the current implementation name and signature of duplicate > >>>>> field is > >>>> missing in java.lang.ClassFormatError exception message. > >>>>> Without a field name + signature it is hard to triggering the problem. > >>>>> > >>>>> Webrev link: > >> http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ > >>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 > >>>>> > >>>>> Testing: jprt and jtreg test. > >>>>> > >>>>> I have verified my changes with the reproduces of > >>>> https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 > >>>> code base as I was not able to write reproducer of current issue. > >>>>> With the fix I am getting below exception message. > >>>>> > >>>>> $ java CreateBadClassFile > >>>>> .foreach() call: Exception in thread "main" java.lang.ClassFormatError: > >>>> Duplicate field name "hasNext" with signature "Ljava.lang.Object;" > >>>> in class file WidgetCollection$1 > >>>>> at java.lang.ClassLoader.defineClass1(Native Method) > >>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > >>>>> at > >>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java: > >>>> 14 > >>>> 2) > >>>>> . . . > >>>>> > >>>>> Thanks, > >>>>> Coleen > >>>>> From serguei.spitsyn at oracle.com Wed Feb 15 07:18:30 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Feb 2017 23:18:30 -0800 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> Message-ID: Hi Coleen, Thank you for the review discussion with David! It helps to understand the webrev details. It looks good in general. Some comments/questions below. http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCodeInstaller.cpp.udiff.html for (jint i = 0; i < values->length(); i++) { ScopeValue* cur_second = NULL; - Handle object = values->obj_at(i); - BasicType type = JVMCIRuntime::kindToBasicType(slotKinds->obj_at(i), CHECK); + Handle object (THREAD, values->obj_at(i)); + Handle slot_kind (THREAD, slotKinds->obj_at(i)); + BasicType type = JVMCIRuntime::kindToBasicType(slot_kind, CHECK); Do we need a HandleMark in the loop? http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCompilerToVM.cpp.udiff.html Handle result; if (!klass.is_null()) { - result = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); + oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); + result = Handle(THREAD, result_oop); } else { result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); } return JNIHandles::make_local(THREAD, result()); Not clear, why does the result need to be a handle. http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.udiff.html + const char* javapkg = "java/"; . . . - strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { + strncmp(class_name->as_C_string(), javapkg, strlen(javapkg)) == 0) { Why this change is needed? It is a reverse of the recent Rachel's fix. http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html 701 void InstanceKlass::initialize_impl(instanceKlassHandle this_k, TRAPS) { 702 HandleMark hm(THREAD); . . . 712 // refer to the JVM book page 47 for description of steps 713 // Step 1 714 { 715 HandleMark hm(THREAD); It is not clear what is the reason for one more HandleMark at 715. Thanks, Serguei On 2/14/17 14:21, coleen.phillimore at oracle.com wrote: > > New webrev: > > open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev > > see coments below first. > > On 2/13/17 11:51 PM, David Holmes wrote: >> Hi Coleen, >> >> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>> >>> >>> On 2/13/17 8:44 PM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> "actually S"? :) >>> >>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>> backports. >>> >>>> >>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>> Thread::current() call. >>>> >>>> Well there's a bit more to it than that as you also change parameter >>>> types so that we (sometimes) avoid: >>>> >>>> - oop -> Handle -> oop >>>> - Handle -> oop -> Handle >>> >>> Yes, I put that in the bug but not in the RFR. The recommendation >>> is to >>> Handle the oop as far up as possible in the call stack and pass Handle >>> around. >> >> I hear what you are saying but it is hard to actually see that >> reflected in the code. >> >>>> across method calls. This leads to some puzzling differences eg: >>>> >>>> src/share/vm/aot/aotCodeHeap.cpp >>>> src/share/vm/classfile/javaClasses.hpp >>>> >>>> The change in aotCodeHeap.cpp reverses the use of Handle and oop, but >>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>> have different signatures in that regard. They used to both take a >>>> Handle but now one takes an oop instead. ?? >>> >>> Yes, you found the exceptions in Throwable. A lot of the >>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop >>> instead >>> of Handle and I changed print to oop to make it consistent, and because >>> they it being called from other functions in Throwable with oop. >>> Throwable::print_stack_trace actually goes to a safepoint so has to be >>> Handle. >> >> I think in javaClasses.hpp the rule should be oop unless Handle is >> definitely needed. > > It's easiest to keep it an oop because most of the functions just do > an obj_field, etc. > > oop java_lang_Throwable::message(oop throwable) { > return throwable->obj_field(detailMessage_offset); > } > > but it's likely that the oop throwable is a handle up the call > stack. Interestingly looking at it's callers is one in > javaClasses.cpp that's an oop, one in exceptions.cpp that's a Handle, > > oop msg = java_lang_Throwable::message(exception()); > > And then there's the one in universe.cpp that's a naked oop, which can > get moved before the call. > > // get the error object at the slot and set set it to NULL so that > the > // array isn't keeping it alive anymore. > oop exc = preallocated_out_of_memory_errors()->obj_at(next); > assert(exc != NULL, "slot has been used already"); > preallocated_out_of_memory_errors()->obj_at_put(next, NULL); > <= This on G1 can take out a lock for the oop_store and GC and move > the default_err > > // use the message from the default error > oop msg = java_lang_Throwable::message(default_err); <= > default_err is an oop. > assert(msg != NULL, "no message"); > java_lang_Throwable::set_message(exc, msg); > > exc is a naked oop too. I don't think the naked oop detector triggers > for obj_at_put/oop_store because G1 wasn't in the code when it was added. > > Creating a Handle and passing it saves having to visually inspect > functions to try to find naked oops. > > Fixing this to have a local handle though because the callers don't > have THREAD. >> >>> >>>> >>>> More generally it is unclear why you made the changes you did in >>>> places - see below for some specifics. I'm left questioning how to >>>> know when to take an oop and when to take a Handle. Of course that >>>> question already exists, but these changes didn't make it any clearer >>>> for me. >>> >>> Pass a Handle early and often is still the rule. For all but GC code. >>> There are still some functions that are very small and don't safepoint >>> and take oop. Some of these have migrated to taking Handle, but some >>> haven't. I didn't want to make this change larger for now. >>> >>>> >>>> I understand why the oop to Handle implicit conversion could be >>>> problematic due to the Thread::current() calls, but I don't understand >>>> why you couldn't keep (? add?) the Handle to oop implicit >>>> conversions ?? >>> >>> There weren't Handle -> oop implicit conversions. The conversion back >>> to oop uses the operator() which remains. >> >> Okay then it seems an implicit Handle -> oop would avoid the need to >> change the caller from foo to foo() if the underlying API switched >> from oop to Handle. > > Yes but it's better to just pass the oop along until something really > needs to do something with it. >> >>> The oop -> Handle conversion has cost, both at runtime and also to the >>> GC because it increases the number of Handles on the >>> Thread->_handle_area. >> >> Sure. >> >>> I did this same experiment years ago when Thread::current() was at the >>> top of the profiling call stack but back then I needed to add too many >>> explicit Thread::current() calls. There aren't many in this change, so >>> it's worth doing. Also, metadata handles have the same problem but >>> there are still too many of them to remove the implicit conversion. >>> And {instance}KlassHandle needs to be removed first. They do nothing >>> now but hide the type. >>> >>>> >>>> --- >>>> >>>> src/share/vm/c1/c1_Runtime1.cpp >>>> >>>> Aside: I'm a little surprised that there was not a Handle assignment >>>> operator or copy constructor such that given: >>>> >>>> 863 Handle appendix(THREAD, NULL); >>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>> >>>> that it simply updated the NULL to the desired value, while keeping >>>> the THREAD intact. >>> >>> Handle doesn't save the THREAD so a NULL handle is just oop* NULL. The >>> RHS of the expression has to be converted to Handle and the default >>> assignment operator copies it to appendix. >> >> I had assumed Handle saved the Thread. > > No, then it would be two fields. It's only one so passing it is cheap > now. >> >>>> >>>> --- >>>> >>>> src/share/vm/ci/ciInstance.cpp >>>> >>>> Not at all obvious that replacing Handle with raw oop is correct. I'm >>>> not saying it isn't, just that it isn't obvious - and I'll wonder why >>>> the Handle was used in the first place. >>> >>> It is safe because it's immediately unhandled in all the case >>> statements, and I wanted to avoid a Thread::current call. >>>> >>>> --- >>>> >>>> src/share/vm/classfile/javaClasses.cpp >>>> >>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>> instead of an oop if you are simply going to unwrap the Handle to get >>>> the oop back. ??? Particular when a caller like >>>> src/share/vm/prims/methodHandles.cpp has to create the Handle from oop >>>> in the first place. >>> >>> I changed it to be the same as java_lang_String::as_symbol(Handle >>> java_string ...) which took a handle. java_lang_String::as_symbol() was >>> the same - it immediately unhandles the argument to pass to the rest of >>> the functions. I changed java_lang_String::as_symbol() to take an oop >>> to match, and there are few scattered changes from that. >> >> I have to disagree with this one. AFAICS there is no reason this: >> >> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >> >> needs to take a Handle in the first place. The java_string is >> unwrapped to get the oop and the oop is only passed to a few other >> String methods and then not touched. Looking at the API's in >> javaClass.hpp the norm should be to take oop (as the bulk of methods >> do) with Handle only used when actually required. > > I changed this one to take oop because it was obvious on inspection > but note that the "required" is sometimes really hard to see! Chasing > naked oop problems isn't something we do very much anymore because we > converted many arguments to Handles. Note that the runtime only > passes oops much of the time and doesn't usually do anything with > them. The exception is javaClasses.cpp though. > >> >>> >>>> >>>> --- >>>> >>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>> >>>> It is not at all obvious that JavaArgumentUnboxers are >>>> thread-confined! I'm also concerned by API's (unfortunately a number >>>> pre-existing) that take a Thread/JavaThread argument but really >>>> require it to be the current thread. To that end I'd rather see >>>> _thread as _cur_thread and an assertion when it is set. >>> >>> Since it's a ResourceObj and not StackObj (setting aside debate about >>> that), I agree with you. I changed it to use Thread::current() >>> where it >>> handles the object, which it explicitly did before. In this case, >>> it is >>> a stack allocated thing but it's not guaranteed to be that. >> >> Ok. >> >>>> >>>> --- >>>> >>>> src/share/vm/oops/cpCache.cpp >>>> >>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>> >>>> Please remove space before ( >>> >>> fixed. >>> >>>> >>>> --- >>>> >>>> src/share/vm/prims/jvm.cpp >>>> >>>> Nit: need to fix indent on StackWalk::walk call second line. >>> >>> fixed. >>> >>>> >>>> --- >>>> >>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>> >>>> Ditto re _thread -> _cur_thread >>>> >>> >>> This one is a StackObj (a Closure which should be used that way). So I >>> changed _thread to _cur_thread and added an assert that it == >>> Thread::current() in the constructor. >> >> Thanks. >> >>>> --- >>>> >>>> src/share/vm/prims/jvmtiImpl.cpp >>>> >>>> Nit: space before ( again (on handle constructor) >>> >>> fixed. >>>> >>>>> It's a very small change to a number of files. Notably the new JVMCI >>>>> files have a lot of implicit conversions from oop to handle. I >>>>> want to >>>>> get this change in early for jdk10 to stop this usage. I also >>>>> added a >>>>> few HandleMarks where I found that the HandlesArea was growing large. >>>> >>>> The addition and removal of the HandleMarks seems problematic because >>>> there are no obvious rules being applied. It is hard to know where and >>>> when a HandleMark is needed or not. Looking at >>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>> create/destroy the HandleMark on each loop iteration? That would only >>>> seem necessary if we know the number of iterations is so high that we >>>> might consume all our handle space. >>> >>> Nobody knows when and where to put HandleMarks. It's a dark art. I >> >> So a trade-off between too much cleanup overhead if we have too many, >> and too much GC overhead if we have too few. And no real way to know >> either way - Ouch! :( >> >>> also was testing with a JVM that restored the assert if HandleArea got >>> over 200 handles and the numbers got very high in these functions. The >>> GC has to walk these HandleAreas and I am trying to reduce work for >>> them. >>> >>> I tried to write about this in the bug. I'd like to see if changing >>> Handle to a proper constructor/destructor like methodHandles, so that >>> only the Handles that are active need to be collected. Then we don't >>> need HandleMark and their associated mystery. There are several >>> HandleMarks in code that obviously doesn't allocate any Handles. I >>> took >>> out several but restored them so that this patch was smaller. To >>> change Handle to a scoped object would require changing passing Handle >>> as a const reference like I did with methodHandle which is too big of a >>> change for early-jdk10. >> >> Ok. This sounds like a way forward though. In combination with an >> easy way to detect when handles are required, this would allow very >> precise management of handle lifetimes. > > Good, I'm glad you think so too. >> >>> The reason I worked on this RFE was because I was curious to see how >>> many oops/Handles we're passing around and using from within the JVM >>> these days. It's more than I thought. >>> >>> Thank you for reviewing this. I'm going to rerun some tests with the >>> changes above and send another webrev. >> >> Ok. Thanks. > > Thanks for the discussion and review! > > Coleen >> >> David >> ----- >> >>> Coleen >>> >>>> >>>> Thanks, >>>> David >>>> ------ >>>> >>>>> See bug for more on the motivation of this change. >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>> >>>>> Tested by running all hotspot jtreg tests with >>>>> -XX:+CheckUnhandledOops. >>>>> There weren't any unhandled oops amazingly. >>>>> >>>>> Thanks, >>>>> Coleen >>> > From thomas.schatzl at oracle.com Wed Feb 15 09:12:56 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 15 Feb 2017 10:12:56 +0100 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> Message-ID: <1487149976.3359.13.camel@oracle.com> Hi, On Sun, 2017-02-12 at 23:25 -0500, Kim Barrett wrote: > Here's the updated webrevs, incorporating feedback received so far. > > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ > ? Ship it. Thanks, ? Thomas From shafi.s.ahmad at oracle.com Wed Feb 15 11:35:18 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 15 Feb 2017 03:35:18 -0800 (PST) Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed Message-ID: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> Hi All, Summary: Adding return value check and update index variable. It's a very small change to single file. Webrev link: http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 Testing: jprt and jtreg test. Regards, Shafi From david.holmes at oracle.com Wed Feb 15 12:05:37 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Feb 2017 22:05:37 +1000 Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> Message-ID: <02e6e70f-7ac8-d41b-e51b-c8ecbf40241b@oracle.com> Hi Shafi, Looks good to me. Thanks, David On 15/02/2017 9:35 PM, Shafi Ahmad wrote: > Hi All, > > Summary: Adding return value check and update index variable. It's a very small change to single file. > > Webrev link: http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ > bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 > > Testing: jprt and jtreg test. > > Regards, > Shafi > From shafi.s.ahmad at oracle.com Wed Feb 15 12:06:50 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 15 Feb 2017 04:06:50 -0800 (PST) Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <02e6e70f-7ac8-d41b-e51b-c8ecbf40241b@oracle.com> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> <02e6e70f-7ac8-d41b-e51b-c8ecbf40241b@oracle.com> Message-ID: Hi David, Thank you for the review. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 15, 2017 5:36 PM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of > JDK_Version::to_string OR proper return statement is missing OR proper > comment is needed > > Hi Shafi, > > Looks good to me. > > Thanks, > David > > On 15/02/2017 9:35 PM, Shafi Ahmad wrote: > > Hi All, > > > > Summary: Adding return value check and update index variable. It's a very > small change to single file. > > > > Webrev link: http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ > > bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 > > > > Testing: jprt and jtreg test. > > > > Regards, > > Shafi > > From george.triantafillou at oracle.com Wed Feb 15 14:29:02 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 15 Feb 2017 09:29:02 -0500 Subject: RFR (XXS): JDK-8174855 Quarantine failing test jdk/test/sun/management/HotspotRuntimeMBean/GetSafepointSyncTime.java In-Reply-To: <1490077f-7f1c-a6b2-4a9e-e95fe16ae643@oracle.com> References: <1b5865c0-6f4d-663e-4852-b18dc376be0b@oracle.com> <725d25a7-9d7f-4eb6-a14b-726b2521227a@oracle.com> <1490077f-7f1c-a6b2-4a9e-e95fe16ae643@oracle.com> Message-ID: Thanks David. -George On 2/14/2017 9:09 PM, David Holmes wrote: > On 15/02/2017 11:40 AM, George Triantafillou wrote: >> Hi David, >> >> Thanks for the review. Updated webrev: >> >> http://cr.openjdk.java.net/~gtriantafill/8174855/webrev.01/ >> > > Looks good! :) > > Thanks, > David > >> -George >> >> On 2/14/2017 5:29 PM, David Holmes wrote: >>> Hi George, >>> >>> On 15/02/2017 5:12 AM, George Triantafillou wrote: >>>> Please review this very small fix for JDK-8174855: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8174855 >>>> open webrev: http://cr.openjdk.java.net/~gtriantafill/8174855/webrev/ >>>> >>>> >>>> The test was added to ProblemList.txt. Thanks. >>> >>> The bug id in the ProblemList.txt is the bug that needs to be fixed to >>> allow the test to be removed from the ProblemList - not the bug id >>> used to add it to the list. >>> >>> David >>> ----- >>> >>>> -George >>>> >> From erik.helin at oracle.com Wed Feb 15 15:07:04 2017 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 15 Feb 2017 16:07:04 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> Message-ID: <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> On 02/15/2017 02:48 AM, David Holmes wrote: > Hi Erik, Hi David, thanks for having a look! Please see new patches at: - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ > oops_do can miss an oop that is in the process of being added - is that > a problem? Nope, that is not an issue. Even if `add_handle` and `oops_do` where synchronized with each other (for example by using lock), you can still have the scenario where a GC thread first runs (and finishes) `oops_do` and then `add_handle` is called (a linearized execution). This scenario is something the concurrent marking algorithms already should be prepared to handle. > 140 if (c->data[i] != NULL) { > 141 f->do_oop(&c->data[i]); > 142 } > > Given a slot can be nulled out concurrently at any time, is it worth > does this NULL check? The OopClosure will have to do its own NULL check > anyway. Sure, this was mostly to be a bit consistent with JNIHandleBlock (it does the same NULL check). Think of it as an optimization, there is no reason to call `f->do_oop` if `c->data[i] == NULL`. Do you think it reads better if the NULL check is removed? I don't have a strong opinion on this one. > 144 c = (Chunk*) OrderAccess::load_ptr_acquire((volatile > intptr_t*)&c->next); > > This doesn't need to be a load-acquire. You already loaded 'c' via > load-acquire of _head (or chained therefrom) and its next field is set > prior to the setting of the _head that you read. Agree. I just discussed this with Erik ? as well, and we all agree on this. I also removed the `volatile` specifier for `next`. > 624 jobject ClassLoaderData::add_handle(Handle h) { > 625 MutexLockerEx ml(metaspace_lock(), Mutex::_no_safepoint_check_flag); > 626 return (jobject) _handles.add(h()); > 627 } > 628 > 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { > 630 assert(_handles.contains((oop*) h), "Got unexpected handle " > PTR_FORMAT, p2i((oop*) h)); > 631 *((oop*) h) = NULL; > 632 } > > I'm a bit unclear on the typing here. Previously we use a JNI Handle > which was a jobject. But now we've simply stored an oop into a slot in > an array. We pass the address of that slot back as a jobject, but is it > really? So, this is a bit confusing, and I discussed this with Coleen a few days ago. jobject was used originally used because that is what `JNIHandleBlock::allocate_handle` returns. Using something other than jobject means updating `ModuleEntry` and in particular users `ModuleEntry::_pd`. This is not something we are keen on doing right now for JDK 9, and Coleen is planning to clean up how the runtime handles oops for 10 (or at least improve the situation). I don't know what jobject is meant to represent, in the code it is just an empty class (an opaque type). ClassLoaderData::add_handle could just as well return an oop*, but AFAIU we try to avoid oops and oop* in the runtime code (because that might mean that some code forgot to tell the GC about the oop). Maybe I'm wrong here? > I would also expect contains to take an oop not an oop* - it > seems to be asking "is this an address of a slot in our > ChunkedHandleList" rather than asking "is this oop in our > ChunkedHandleList". ?? I actually wanted the assert to check whether the passed jobject (which really is an oop*) actually originated from a call to ClassLoaderData::add_handle. I don't want any code to pass a jobject to ClassLoaderData::remove_handle_unsafe that doesn't come from a call to ClassLoaderData::add_handle. > A few minor comments: > > Copyrights need updating. Aaah, it is that time of the year again. Fixed. > src/share/vm/classfile/classLoaderData.hpp > > 177 // Only on thread ... > > Typo: on -> one Thanks, fixed. > src/share/vm/classfile/moduleEntry.cpp > > 90 // Create a JNI handle for the shared ProtectionDomain and save it > atomically. > 91 // If someone beats us setting the _pd cache, the created JNI > handle is destroyed. > > These are no longer JNI Handles. Thanks, I updated the comment. Thanks, Erik > Thanks, > David > ----- > >> Testing: >> - Tier 1 (aka JPRT), 2, 3, 4, 5. >> >> I would appreciate quite a few reviews on this patch, it contains a nice >> mixture of: >> - me venturing into the runtime code :) >> - lock free code >> - unreproducible bug (even though I'm sure of the root cause) >> >> Thanks for everyone participating in the discussions around this bug and >> potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and >> Jiangli. >> >> Thanks! >> Erik >> >> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From erik.osterlund at oracle.com Wed Feb 15 15:22:44 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 15 Feb 2017 16:22:44 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> Message-ID: <58A47244.8010700@oracle.com> Hi Erik, I looked at the memory ordering aspect and I think it looks good as I discussed with you, now that the unnecessary synchronization on the immutable next value has been removed. Thanks, /Erik On 2017-02-15 16:07, Erik Helin wrote: > On 02/15/2017 02:48 AM, David Holmes wrote: >> Hi Erik, > > Hi David, > > thanks for having a look! Please see new patches at: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ > >> oops_do can miss an oop that is in the process of being added - is that >> a problem? > > Nope, that is not an issue. Even if `add_handle` and `oops_do` where > synchronized with each other (for example by using lock), you can > still have the scenario where a GC thread first runs (and finishes) > `oops_do` and then `add_handle` is called (a linearized execution). > > This scenario is something the concurrent marking algorithms already > should be prepared to handle. > >> 140 if (c->data[i] != NULL) { >> 141 f->do_oop(&c->data[i]); >> 142 } >> >> Given a slot can be nulled out concurrently at any time, is it worth >> does this NULL check? The OopClosure will have to do its own NULL check >> anyway. > > Sure, this was mostly to be a bit consistent with JNIHandleBlock (it > does the same NULL check). Think of it as an optimization, there is no > reason to call `f->do_oop` if `c->data[i] == NULL`. Do you think it > reads better if the NULL check is removed? I don't have a strong > opinion on this one. > >> 144 c = (Chunk*) OrderAccess::load_ptr_acquire((volatile >> intptr_t*)&c->next); >> >> This doesn't need to be a load-acquire. You already loaded 'c' via >> load-acquire of _head (or chained therefrom) and its next field is set >> prior to the setting of the _head that you read. > > Agree. I just discussed this with Erik ? as well, and we all agree on > this. I also removed the `volatile` specifier for `next`. > >> 624 jobject ClassLoaderData::add_handle(Handle h) { >> 625 MutexLockerEx ml(metaspace_lock(), >> Mutex::_no_safepoint_check_flag); >> 626 return (jobject) _handles.add(h()); >> 627 } >> 628 >> 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { >> 630 assert(_handles.contains((oop*) h), "Got unexpected handle " >> PTR_FORMAT, p2i((oop*) h)); >> 631 *((oop*) h) = NULL; >> 632 } >> >> I'm a bit unclear on the typing here. Previously we use a JNI Handle >> which was a jobject. But now we've simply stored an oop into a slot in >> an array. We pass the address of that slot back as a jobject, but is it >> really? > > So, this is a bit confusing, and I discussed this with Coleen a few > days ago. jobject was used originally used because that is what > `JNIHandleBlock::allocate_handle` returns. Using something other than > jobject means updating `ModuleEntry` and in particular users > `ModuleEntry::_pd`. This is not something we are keen on doing right > now for JDK 9, and Coleen is planning to clean up how the runtime > handles oops for 10 (or at least improve the situation). > > I don't know what jobject is meant to represent, in the code it is > just an empty class (an opaque type). > > ClassLoaderData::add_handle could just as well return an oop*, but > AFAIU we try to avoid oops and oop* in the runtime code (because that > might mean that some code forgot to tell the GC about the oop). Maybe > I'm wrong here? > > > I would also expect contains to take an oop not an oop* - it > > seems to be asking "is this an address of a slot in our > > ChunkedHandleList" rather than asking "is this oop in our > > ChunkedHandleList". ?? > > I actually wanted the assert to check whether the passed jobject > (which really is an oop*) actually originated from a call to > ClassLoaderData::add_handle. I don't want any code to pass a jobject > to ClassLoaderData::remove_handle_unsafe that doesn't come from a call > to ClassLoaderData::add_handle. > >> A few minor comments: >> >> Copyrights need updating. > > Aaah, it is that time of the year again. Fixed. > >> src/share/vm/classfile/classLoaderData.hpp >> >> 177 // Only on thread ... >> >> Typo: on -> one > > Thanks, fixed. > >> src/share/vm/classfile/moduleEntry.cpp >> >> 90 // Create a JNI handle for the shared ProtectionDomain and save it >> atomically. >> 91 // If someone beats us setting the _pd cache, the created JNI >> handle is destroyed. >> >> These are no longer JNI Handles. > > Thanks, I updated the comment. > > Thanks, > Erik > >> Thanks, >> David >> ----- >> >>> Testing: >>> - Tier 1 (aka JPRT), 2, 3, 4, 5. >>> >>> I would appreciate quite a few reviews on this patch, it contains a >>> nice >>> mixture of: >>> - me venturing into the runtime code :) >>> - lock free code >>> - unreproducible bug (even though I'm sure of the root cause) >>> >>> Thanks for everyone participating in the discussions around this bug >>> and >>> potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and >>> Jiangli. >>> >>> Thanks! >>> Erik >>> >>> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From thomas.schatzl at oracle.com Wed Feb 15 15:28:48 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 15 Feb 2017 16:28:48 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> Message-ID: <1487172528.3359.19.camel@oracle.com> Hi, On Wed, 2017-02-15 at 16:07 +0100, Erik Helin wrote: > On 02/15/2017 02:48 AM, David Holmes wrote: > > > > Hi Erik, > Hi David, > > thanks for having a look! Please see new patches at: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ > http://cr.openjdk.java.net/~ehelin/8168914/01/src/share/vm/classfile/cl assLoaderData.hpp.frames.html ?177?????// Only one thread can add a time, guarded by the Metaspace_lock. -> Only one thread can add elements at a time, ... Also, I would somewhat prefer if the method actually asserted that Metaspace_lock is owned by this thread if it has been explicitly mentioned in the description. ?- same file: ?211???ChunkedHandleList _handles; // Handles to constant pool arrays, Modules, etc, which ?212???????????????????????????????// have the same life cycle of the corresponding ClassLoader. -> ... have the same life cycle _as_ the corresponding class loader. Looks good otherwise. I do not need a re-review for the comment changes. Thanks, ? Thomas From coleen.phillimore at oracle.com Wed Feb 15 15:43:37 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 10:43:37 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> Message-ID: <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> Hi Serguei, Thank you for reviewing this. On 2/15/17 2:18 AM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > > Thank you for the review discussion with David! > It helps to understand the webrev details. > > It looks good in general. > Some comments/questions below. > > > http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCodeInstaller.cpp.udiff.html > > > for (jint i = 0; i < values->length(); i++) { > ScopeValue* cur_second = NULL; > - Handle object = values->obj_at(i); > - BasicType type = JVMCIRuntime::kindToBasicType(slotKinds->obj_at(i), > CHECK); > + Handle object (THREAD, values->obj_at(i)); > + Handle slot_kind (THREAD, slotKinds->obj_at(i)); > + BasicType type = JVMCIRuntime::kindToBasicType(slot_kind, CHECK); > > > Do we need a HandleMark in the loop? > > Yes, I don't think it would hurt. Added. > > http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCompilerToVM.cpp.udiff.html > > > Handle result; > if (!klass.is_null()) { > - result = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); > + oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); > + result = Handle(THREAD, result_oop); > } else { > result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); > } > return JNIHandles::make_local(THREAD, result()); > > > Not clear, why does the result need to be a handle. > There's no reason this had to be a Handle, but java_lang_String::create_from_symbol returns a Handle. That's why. This seems better, agree? oop result_oop; if (!klass.is_null()) { oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); } else { Handle result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); result_oop = result(); } return JNIHandles::make_local(THREAD, result_oop); > > > http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.udiff.html > > > + const char* javapkg = "java/"; . . . - > strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { > + strncmp(class_name->as_C_string(), javapkg, strlen(javapkg)) == 0) { > > > Why this change is needed? > It is a reverse of the recent Rachel's fix. I messed this up in the integration. Thank you for finding it! This was the only diff in that function: @@ -2460,7 +2463,7 @@ TRAPS) { ResourceMark rm(THREAD); if (!class_loader.is_null() && - !SystemDictionary::is_platform_class_loader(class_loader) && + !SystemDictionary::is_platform_class_loader(class_loader()) && class_name != NULL && strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { TempNewSymbol pkg_name = InstanceKlass::package_from_name(class_name, CHECK); > > > http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html > > > 701 void InstanceKlass::initialize_impl(instanceKlassHandle this_k, > TRAPS) { > 702 HandleMark hm(THREAD); > . . . > 712 // refer to the JVM book page 47 for description of steps > 713 // Step 1 > 714 { > 715 HandleMark hm(THREAD); > > > It is not clear what is the reason for one more HandleMark at 715. > > I'll delete the second one. I added it for debugging number of Handles. Thanks for reviewing! Coleen > > Thanks, > Serguei > > > On 2/14/17 14:21, coleen.phillimore at oracle.com wrote: >> >> New webrev: >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev >> >> see coments below first. >> >> On 2/13/17 11:51 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>>> >>>> >>>> On 2/13/17 8:44 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> "actually S"? :) >>>> >>>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>>> backports. >>>> >>>>> >>>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>>> Thread::current() call. >>>>> >>>>> Well there's a bit more to it than that as you also change parameter >>>>> types so that we (sometimes) avoid: >>>>> >>>>> - oop -> Handle -> oop >>>>> - Handle -> oop -> Handle >>>> >>>> Yes, I put that in the bug but not in the RFR. The recommendation >>>> is to >>>> Handle the oop as far up as possible in the call stack and pass Handle >>>> around. >>> >>> I hear what you are saying but it is hard to actually see that >>> reflected in the code. >>> >>>>> across method calls. This leads to some puzzling differences eg: >>>>> >>>>> src/share/vm/aot/aotCodeHeap.cpp >>>>> src/share/vm/classfile/javaClasses.hpp >>>>> >>>>> The change in aotCodeHeap.cpp reverses the use of Handle and oop, but >>>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>>> have different signatures in that regard. They used to both take a >>>>> Handle but now one takes an oop instead. ?? >>>> >>>> Yes, you found the exceptions in Throwable. A lot of the >>>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop >>>> instead >>>> of Handle and I changed print to oop to make it consistent, and >>>> because >>>> they it being called from other functions in Throwable with oop. >>>> Throwable::print_stack_trace actually goes to a safepoint so has to be >>>> Handle. >>> >>> I think in javaClasses.hpp the rule should be oop unless Handle is >>> definitely needed. >> >> It's easiest to keep it an oop because most of the functions just do >> an obj_field, etc. >> >> oop java_lang_Throwable::message(oop throwable) { >> return throwable->obj_field(detailMessage_offset); >> } >> >> but it's likely that the oop throwable is a handle up the call >> stack. Interestingly looking at it's callers is one in >> javaClasses.cpp that's an oop, one in exceptions.cpp that's a Handle, >> >> oop msg = java_lang_Throwable::message(exception()); >> >> And then there's the one in universe.cpp that's a naked oop, which >> can get moved before the call. >> >> // get the error object at the slot and set set it to NULL so >> that the >> // array isn't keeping it alive anymore. >> oop exc = preallocated_out_of_memory_errors()->obj_at(next); >> assert(exc != NULL, "slot has been used already"); >> preallocated_out_of_memory_errors()->obj_at_put(next, NULL); >> <= This on G1 can take out a lock for the oop_store and GC and move >> the default_err >> >> // use the message from the default error >> oop msg = java_lang_Throwable::message(default_err); <= >> default_err is an oop. >> assert(msg != NULL, "no message"); >> java_lang_Throwable::set_message(exc, msg); >> >> exc is a naked oop too. I don't think the naked oop detector >> triggers for obj_at_put/oop_store because G1 wasn't in the code when >> it was added. >> >> Creating a Handle and passing it saves having to visually inspect >> functions to try to find naked oops. >> >> Fixing this to have a local handle though because the callers don't >> have THREAD. >>> >>>> >>>>> >>>>> More generally it is unclear why you made the changes you did in >>>>> places - see below for some specifics. I'm left questioning how to >>>>> know when to take an oop and when to take a Handle. Of course that >>>>> question already exists, but these changes didn't make it any clearer >>>>> for me. >>>> >>>> Pass a Handle early and often is still the rule. For all but GC code. >>>> There are still some functions that are very small and don't safepoint >>>> and take oop. Some of these have migrated to taking Handle, but some >>>> haven't. I didn't want to make this change larger for now. >>>> >>>>> >>>>> I understand why the oop to Handle implicit conversion could be >>>>> problematic due to the Thread::current() calls, but I don't >>>>> understand >>>>> why you couldn't keep (? add?) the Handle to oop implicit >>>>> conversions ?? >>>> >>>> There weren't Handle -> oop implicit conversions. The conversion back >>>> to oop uses the operator() which remains. >>> >>> Okay then it seems an implicit Handle -> oop would avoid the need to >>> change the caller from foo to foo() if the underlying API switched >>> from oop to Handle. >> >> Yes but it's better to just pass the oop along until something really >> needs to do something with it. >>> >>>> The oop -> Handle conversion has cost, both at runtime and also to the >>>> GC because it increases the number of Handles on the >>>> Thread->_handle_area. >>> >>> Sure. >>> >>>> I did this same experiment years ago when Thread::current() was at the >>>> top of the profiling call stack but back then I needed to add too many >>>> explicit Thread::current() calls. There aren't many in this >>>> change, so >>>> it's worth doing. Also, metadata handles have the same problem but >>>> there are still too many of them to remove the implicit conversion. >>>> And {instance}KlassHandle needs to be removed first. They do nothing >>>> now but hide the type. >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/c1/c1_Runtime1.cpp >>>>> >>>>> Aside: I'm a little surprised that there was not a Handle assignment >>>>> operator or copy constructor such that given: >>>>> >>>>> 863 Handle appendix(THREAD, NULL); >>>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>>> >>>>> that it simply updated the NULL to the desired value, while keeping >>>>> the THREAD intact. >>>> >>>> Handle doesn't save the THREAD so a NULL handle is just oop* NULL. The >>>> RHS of the expression has to be converted to Handle and the default >>>> assignment operator copies it to appendix. >>> >>> I had assumed Handle saved the Thread. >> >> No, then it would be two fields. It's only one so passing it is >> cheap now. >>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/ci/ciInstance.cpp >>>>> >>>>> Not at all obvious that replacing Handle with raw oop is correct. I'm >>>>> not saying it isn't, just that it isn't obvious - and I'll wonder why >>>>> the Handle was used in the first place. >>>> >>>> It is safe because it's immediately unhandled in all the case >>>> statements, and I wanted to avoid a Thread::current call. >>>>> >>>>> --- >>>>> >>>>> src/share/vm/classfile/javaClasses.cpp >>>>> >>>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>>> instead of an oop if you are simply going to unwrap the Handle to get >>>>> the oop back. ??? Particular when a caller like >>>>> src/share/vm/prims/methodHandles.cpp has to create the Handle from >>>>> oop >>>>> in the first place. >>>> >>>> I changed it to be the same as java_lang_String::as_symbol(Handle >>>> java_string ...) which took a handle. java_lang_String::as_symbol() >>>> was >>>> the same - it immediately unhandles the argument to pass to the >>>> rest of >>>> the functions. I changed java_lang_String::as_symbol() to take an oop >>>> to match, and there are few scattered changes from that. >>> >>> I have to disagree with this one. AFAICS there is no reason this: >>> >>> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >>> >>> needs to take a Handle in the first place. The java_string is >>> unwrapped to get the oop and the oop is only passed to a few other >>> String methods and then not touched. Looking at the API's in >>> javaClass.hpp the norm should be to take oop (as the bulk of methods >>> do) with Handle only used when actually required. >> >> I changed this one to take oop because it was obvious on inspection >> but note that the "required" is sometimes really hard to see! Chasing >> naked oop problems isn't something we do very much anymore because we >> converted many arguments to Handles. Note that the runtime only >> passes oops much of the time and doesn't usually do anything with >> them. The exception is javaClasses.cpp though. >> >>> >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>>> >>>>> It is not at all obvious that JavaArgumentUnboxers are >>>>> thread-confined! I'm also concerned by API's (unfortunately a number >>>>> pre-existing) that take a Thread/JavaThread argument but really >>>>> require it to be the current thread. To that end I'd rather see >>>>> _thread as _cur_thread and an assertion when it is set. >>>> >>>> Since it's a ResourceObj and not StackObj (setting aside debate about >>>> that), I agree with you. I changed it to use Thread::current() >>>> where it >>>> handles the object, which it explicitly did before. In this case, >>>> it is >>>> a stack allocated thing but it's not guaranteed to be that. >>> >>> Ok. >>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/oops/cpCache.cpp >>>>> >>>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>>> >>>>> Please remove space before ( >>>> >>>> fixed. >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jvm.cpp >>>>> >>>>> Nit: need to fix indent on StackWalk::walk call second line. >>>> >>>> fixed. >>>> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>>> >>>>> Ditto re _thread -> _cur_thread >>>>> >>>> >>>> This one is a StackObj (a Closure which should be used that way). So I >>>> changed _thread to _cur_thread and added an assert that it == >>>> Thread::current() in the constructor. >>> >>> Thanks. >>> >>>>> --- >>>>> >>>>> src/share/vm/prims/jvmtiImpl.cpp >>>>> >>>>> Nit: space before ( again (on handle constructor) >>>> >>>> fixed. >>>>> >>>>>> It's a very small change to a number of files. Notably the new >>>>>> JVMCI >>>>>> files have a lot of implicit conversions from oop to handle. I >>>>>> want to >>>>>> get this change in early for jdk10 to stop this usage. I also >>>>>> added a >>>>>> few HandleMarks where I found that the HandlesArea was growing >>>>>> large. >>>>> >>>>> The addition and removal of the HandleMarks seems problematic because >>>>> there are no obvious rules being applied. It is hard to know where >>>>> and >>>>> when a HandleMark is needed or not. Looking at >>>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>>> create/destroy the HandleMark on each loop iteration? That would only >>>>> seem necessary if we know the number of iterations is so high that we >>>>> might consume all our handle space. >>>> >>>> Nobody knows when and where to put HandleMarks. It's a dark art. I >>> >>> So a trade-off between too much cleanup overhead if we have too >>> many, and too much GC overhead if we have too few. And no real way >>> to know either way - Ouch! :( >>> >>>> also was testing with a JVM that restored the assert if HandleArea got >>>> over 200 handles and the numbers got very high in these functions. >>>> The >>>> GC has to walk these HandleAreas and I am trying to reduce work for >>>> them. >>>> >>>> I tried to write about this in the bug. I'd like to see if changing >>>> Handle to a proper constructor/destructor like methodHandles, so that >>>> only the Handles that are active need to be collected. Then we don't >>>> need HandleMark and their associated mystery. There are several >>>> HandleMarks in code that obviously doesn't allocate any Handles. I >>>> took >>>> out several but restored them so that this patch was smaller. To >>>> change Handle to a scoped object would require changing passing Handle >>>> as a const reference like I did with methodHandle which is too big >>>> of a >>>> change for early-jdk10. >>> >>> Ok. This sounds like a way forward though. In combination with an >>> easy way to detect when handles are required, this would allow very >>> precise management of handle lifetimes. >> >> Good, I'm glad you think so too. >>> >>>> The reason I worked on this RFE was because I was curious to see how >>>> many oops/Handles we're passing around and using from within the JVM >>>> these days. It's more than I thought. >>>> >>>> Thank you for reviewing this. I'm going to rerun some tests with the >>>> changes above and send another webrev. >>> >>> Ok. Thanks. >> >> Thanks for the discussion and review! >> >> Coleen >>> >>> David >>> ----- >>> >>>> Coleen >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> ------ >>>>> >>>>>> See bug for more on the motivation of this change. >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>>> >>>>>> Tested by running all hotspot jtreg tests with >>>>>> -XX:+CheckUnhandledOops. >>>>>> There weren't any unhandled oops amazingly. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>> >> > From daniel.daugherty at oracle.com Wed Feb 15 16:34:04 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 15 Feb 2017 09:34:04 -0700 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> Message-ID: On 2/12/17 9:25 PM, Kim Barrett wrote: > Here's the updated webrevs, incorporating feedback received so far. > > full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ src/share/vm/runtime/javaCalls.hpp No comments. src/share/vm/runtime/javaCalls.cpp No comments. src/share/vm/runtime/jniHandles.hpp No comments. src/share/vm/runtime/jniHandles.cpp No comments. src/share/vm/prims/jni.cpp No comments. src/share/vm/prims/jvmtiEnv.cpp No comments. make/test/JtregNative.gmk No comments. src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp No comments. src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp No comments. src/cpu/arm/vm/interp_masm_arm.cpp No comments. src/cpu/arm/vm/interp_masm_arm.hpp No comments. src/cpu/arm/vm/macroAssembler_arm.cpp No comments. src/cpu/arm/vm/macroAssembler_arm.hpp No comments. src/cpu/arm/vm/sharedRuntime_arm.cpp No comments. src/cpu/arm/vm/templateInterpreterGenerator_arm.cpp No comments. src/cpu/ppc/vm/frame_ppc.cpp No comments. src/cpu/ppc/vm/sharedRuntime_ppc.cpp No comments. src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp No comments. src/cpu/s390/vm/sharedRuntime_s390.cpp No comments. src/cpu/s390/vm/templateInterpreterGenerator_s390.cpp No comments. src/cpu/sparc/vm/sharedRuntime_sparc.cpp No comments. src/cpu/sparc/vm/templateInterpreterGenerator_sparc.cpp No comments. src/cpu/x86/vm/macroAssembler_x86.cpp No comments. src/cpu/x86/vm/macroAssembler_x86.hpp No comments. src/cpu/x86/vm/sharedRuntime_x86_32.cpp No comments. src/cpu/x86/vm/sharedRuntime_x86_64.cpp No comments. src/cpu/x86/vm/templateInterpreterGenerator_x86.cpp No comments. src/cpu/zero/vm/cppInterpreter_zero.cpp No comments. src/share/vm/shark/sharkNativeWrapper.cpp No comments. test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java No comments. test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c No comments. test/runtime/jni/ReturnJNIWeak/ReturnJNIWeak.java No comments. test/runtime/jni/ReturnJNIWeak/libReturnJNIWeak.c No comments. Thumbs up! Dan > incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ > > For x86 and ARM I added resolve_jobject functions to their respective > macroAssembler files, and call those to generate the return object > handling code. Merging the two cases for Sparc is harder and I'm > going to leave it to someone else. I'm also leaving any such merging > for other (non-Oracle) ports to their respective maintainers. > > Rewrote the error checking in SignatureChekker::check_obj to simplify > the control flow and provide more information. Rather than > conditionally selecting between ReportJNIFatalError or assert to > report problems, instead just always use guarantee. Recall that we > only get here if CheckJNICalls or in a debug build. This made > SignatureChekker::_thread unused, so removed it. > From serguei.spitsyn at oracle.com Wed Feb 15 16:50:10 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 15 Feb 2017 08:50:10 -0800 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> Message-ID: <519f62ac-d353-7d4c-5f6f-1acb722c1a35@oracle.com> Coleen, On 2/15/17 07:43, coleen.phillimore at oracle.com wrote: > > Hi Serguei, Thank you for reviewing this. > > On 2/15/17 2:18 AM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> >> Thank you for the review discussion with David! >> It helps to understand the webrev details. >> >> It looks good in general. >> Some comments/questions below. >> >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCodeInstaller.cpp.udiff.html >> >> >> for (jint i = 0; i < values->length(); i++) { >> ScopeValue* cur_second = NULL; >> - Handle object = values->obj_at(i); >> - BasicType type = >> JVMCIRuntime::kindToBasicType(slotKinds->obj_at(i), CHECK); >> + Handle object (THREAD, values->obj_at(i)); >> + Handle slot_kind (THREAD, slotKinds->obj_at(i)); >> + BasicType type = JVMCIRuntime::kindToBasicType(slot_kind, CHECK); >> >> >> Do we need a HandleMark in the loop? >> >> > Yes, I don't think it would hurt. Added. Thanks! > >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCompilerToVM.cpp.udiff.html >> >> >> Handle result; >> if (!klass.is_null()) { >> - result = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >> + oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >> + result = Handle(THREAD, result_oop); >> } else { >> result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); >> } >> return JNIHandles::make_local(THREAD, result()); >> >> >> Not clear, why does the result need to be a handle. >> > > There's no reason this had to be a Handle, but > java_lang_String::create_from_symbol returns a Handle. That's why. > This seems better, agree? > > oop result_oop; > if (!klass.is_null()) { > oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); > } else { > Handle result = java_lang_String::create_from_symbol(symbol, > CHECK_NULL); > result_oop = result(); > } > return JNIHandles::make_local(THREAD, result_oop); Agreed. Alternatively, this can be used: result_oop = java_lang_String::create_from_symbol(symbol, CHECK_NULL)(); > >> >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.udiff.html >> >> >> + const char* javapkg = "java/"; . . . - >> strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { >> + strncmp(class_name->as_C_string(), javapkg, strlen(javapkg)) == 0) { >> >> >> Why this change is needed? >> It is a reverse of the recent Rachel's fix. > > I messed this up in the integration. Thank you for finding it! This > was the only diff in that function: > > @@ -2460,7 +2463,7 @@ > TRAPS) { > ResourceMark rm(THREAD); > if (!class_loader.is_null() && > - !SystemDictionary::is_platform_class_loader(class_loader) && > + !SystemDictionary::is_platform_class_loader(class_loader()) && > class_name != NULL && > strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { > TempNewSymbol pkg_name = > InstanceKlass::package_from_name(class_name, CHECK); Ok, thanks! > >> >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html >> >> >> 701 void InstanceKlass::initialize_impl(instanceKlassHandle this_k, >> TRAPS) { >> 702 HandleMark hm(THREAD); >> . . . >> 712 // refer to the JVM book page 47 for description of steps >> 713 // Step 1 >> 714 { >> 715 HandleMark hm(THREAD); >> >> >> It is not clear what is the reason for one more HandleMark at 715. >> >> > > I'll delete the second one. I added it for debugging number of Handles. Thanks, Coleen! I have no more comments - reviewed. Thanks, Serguei > > Thanks for reviewing! > Coleen > >> >> Thanks, >> Serguei >> >> >> On 2/14/17 14:21, coleen.phillimore at oracle.com wrote: >>> >>> New webrev: >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev >>> >>> see coments below first. >>> >>> On 2/13/17 11:51 PM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>>>> >>>>> >>>>> On 2/13/17 8:44 PM, David Holmes wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> "actually S"? :) >>>>> >>>>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>>>> backports. >>>>> >>>>>> >>>>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>>>> Thread::current() call. >>>>>> >>>>>> Well there's a bit more to it than that as you also change parameter >>>>>> types so that we (sometimes) avoid: >>>>>> >>>>>> - oop -> Handle -> oop >>>>>> - Handle -> oop -> Handle >>>>> >>>>> Yes, I put that in the bug but not in the RFR. The recommendation >>>>> is to >>>>> Handle the oop as far up as possible in the call stack and pass >>>>> Handle >>>>> around. >>>> >>>> I hear what you are saying but it is hard to actually see that >>>> reflected in the code. >>>> >>>>>> across method calls. This leads to some puzzling differences eg: >>>>>> >>>>>> src/share/vm/aot/aotCodeHeap.cpp >>>>>> src/share/vm/classfile/javaClasses.hpp >>>>>> >>>>>> The change in aotCodeHeap.cpp reverses the use of Handle and oop, >>>>>> but >>>>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>>>> have different signatures in that regard. They used to both take a >>>>>> Handle but now one takes an oop instead. ?? >>>>> >>>>> Yes, you found the exceptions in Throwable. A lot of the >>>>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop >>>>> instead >>>>> of Handle and I changed print to oop to make it consistent, and >>>>> because >>>>> they it being called from other functions in Throwable with oop. >>>>> Throwable::print_stack_trace actually goes to a safepoint so has >>>>> to be >>>>> Handle. >>>> >>>> I think in javaClasses.hpp the rule should be oop unless Handle is >>>> definitely needed. >>> >>> It's easiest to keep it an oop because most of the functions just do >>> an obj_field, etc. >>> >>> oop java_lang_Throwable::message(oop throwable) { >>> return throwable->obj_field(detailMessage_offset); >>> } >>> >>> but it's likely that the oop throwable is a handle up the call >>> stack. Interestingly looking at it's callers is one in >>> javaClasses.cpp that's an oop, one in exceptions.cpp that's a Handle, >>> >>> oop msg = java_lang_Throwable::message(exception()); >>> >>> And then there's the one in universe.cpp that's a naked oop, which >>> can get moved before the call. >>> >>> // get the error object at the slot and set set it to NULL so >>> that the >>> // array isn't keeping it alive anymore. >>> oop exc = preallocated_out_of_memory_errors()->obj_at(next); >>> assert(exc != NULL, "slot has been used already"); >>> preallocated_out_of_memory_errors()->obj_at_put(next, NULL); >>> <= This on G1 can take out a lock for the oop_store and GC and move >>> the default_err >>> >>> // use the message from the default error >>> oop msg = java_lang_Throwable::message(default_err); <= >>> default_err is an oop. >>> assert(msg != NULL, "no message"); >>> java_lang_Throwable::set_message(exc, msg); >>> >>> exc is a naked oop too. I don't think the naked oop detector >>> triggers for obj_at_put/oop_store because G1 wasn't in the code when >>> it was added. >>> >>> Creating a Handle and passing it saves having to visually inspect >>> functions to try to find naked oops. >>> >>> Fixing this to have a local handle though because the callers don't >>> have THREAD. >>>> >>>>> >>>>>> >>>>>> More generally it is unclear why you made the changes you did in >>>>>> places - see below for some specifics. I'm left questioning how to >>>>>> know when to take an oop and when to take a Handle. Of course that >>>>>> question already exists, but these changes didn't make it any >>>>>> clearer >>>>>> for me. >>>>> >>>>> Pass a Handle early and often is still the rule. For all but GC >>>>> code. >>>>> There are still some functions that are very small and don't >>>>> safepoint >>>>> and take oop. Some of these have migrated to taking Handle, but some >>>>> haven't. I didn't want to make this change larger for now. >>>>> >>>>>> >>>>>> I understand why the oop to Handle implicit conversion could be >>>>>> problematic due to the Thread::current() calls, but I don't >>>>>> understand >>>>>> why you couldn't keep (? add?) the Handle to oop implicit >>>>>> conversions ?? >>>>> >>>>> There weren't Handle -> oop implicit conversions. The conversion >>>>> back >>>>> to oop uses the operator() which remains. >>>> >>>> Okay then it seems an implicit Handle -> oop would avoid the need >>>> to change the caller from foo to foo() if the underlying API >>>> switched from oop to Handle. >>> >>> Yes but it's better to just pass the oop along until something >>> really needs to do something with it. >>>> >>>>> The oop -> Handle conversion has cost, both at runtime and also to >>>>> the >>>>> GC because it increases the number of Handles on the >>>>> Thread->_handle_area. >>>> >>>> Sure. >>>> >>>>> I did this same experiment years ago when Thread::current() was at >>>>> the >>>>> top of the profiling call stack but back then I needed to add too >>>>> many >>>>> explicit Thread::current() calls. There aren't many in this >>>>> change, so >>>>> it's worth doing. Also, metadata handles have the same problem but >>>>> there are still too many of them to remove the implicit conversion. >>>>> And {instance}KlassHandle needs to be removed first. They do nothing >>>>> now but hide the type. >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/c1/c1_Runtime1.cpp >>>>>> >>>>>> Aside: I'm a little surprised that there was not a Handle assignment >>>>>> operator or copy constructor such that given: >>>>>> >>>>>> 863 Handle appendix(THREAD, NULL); >>>>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>>>> >>>>>> that it simply updated the NULL to the desired value, while keeping >>>>>> the THREAD intact. >>>>> >>>>> Handle doesn't save the THREAD so a NULL handle is just oop* NULL. >>>>> The >>>>> RHS of the expression has to be converted to Handle and the default >>>>> assignment operator copies it to appendix. >>>> >>>> I had assumed Handle saved the Thread. >>> >>> No, then it would be two fields. It's only one so passing it is >>> cheap now. >>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/ci/ciInstance.cpp >>>>>> >>>>>> Not at all obvious that replacing Handle with raw oop is correct. >>>>>> I'm >>>>>> not saying it isn't, just that it isn't obvious - and I'll wonder >>>>>> why >>>>>> the Handle was used in the first place. >>>>> >>>>> It is safe because it's immediately unhandled in all the case >>>>> statements, and I wanted to avoid a Thread::current call. >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/classfile/javaClasses.cpp >>>>>> >>>>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>>>> instead of an oop if you are simply going to unwrap the Handle to >>>>>> get >>>>>> the oop back. ??? Particular when a caller like >>>>>> src/share/vm/prims/methodHandles.cpp has to create the Handle >>>>>> from oop >>>>>> in the first place. >>>>> >>>>> I changed it to be the same as java_lang_String::as_symbol(Handle >>>>> java_string ...) which took a handle. >>>>> java_lang_String::as_symbol() was >>>>> the same - it immediately unhandles the argument to pass to the >>>>> rest of >>>>> the functions. I changed java_lang_String::as_symbol() to take an >>>>> oop >>>>> to match, and there are few scattered changes from that. >>>> >>>> I have to disagree with this one. AFAICS there is no reason this: >>>> >>>> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >>>> >>>> needs to take a Handle in the first place. The java_string is >>>> unwrapped to get the oop and the oop is only passed to a few other >>>> String methods and then not touched. Looking at the API's in >>>> javaClass.hpp the norm should be to take oop (as the bulk of >>>> methods do) with Handle only used when actually required. >>> >>> I changed this one to take oop because it was obvious on inspection >>> but note that the "required" is sometimes really hard to see! >>> Chasing naked oop problems isn't something we do very much anymore >>> because we converted many arguments to Handles. Note that the >>> runtime only passes oops much of the time and doesn't usually do >>> anything with them. The exception is javaClasses.cpp though. >>> >>>> >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>>>> >>>>>> It is not at all obvious that JavaArgumentUnboxers are >>>>>> thread-confined! I'm also concerned by API's (unfortunately a number >>>>>> pre-existing) that take a Thread/JavaThread argument but really >>>>>> require it to be the current thread. To that end I'd rather see >>>>>> _thread as _cur_thread and an assertion when it is set. >>>>> >>>>> Since it's a ResourceObj and not StackObj (setting aside debate about >>>>> that), I agree with you. I changed it to use Thread::current() >>>>> where it >>>>> handles the object, which it explicitly did before. In this case, >>>>> it is >>>>> a stack allocated thing but it's not guaranteed to be that. >>>> >>>> Ok. >>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/oops/cpCache.cpp >>>>>> >>>>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>>>> >>>>>> Please remove space before ( >>>>> >>>>> fixed. >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/prims/jvm.cpp >>>>>> >>>>>> Nit: need to fix indent on StackWalk::walk call second line. >>>>> >>>>> fixed. >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>>>> >>>>>> Ditto re _thread -> _cur_thread >>>>>> >>>>> >>>>> This one is a StackObj (a Closure which should be used that way). >>>>> So I >>>>> changed _thread to _cur_thread and added an assert that it == >>>>> Thread::current() in the constructor. >>>> >>>> Thanks. >>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/prims/jvmtiImpl.cpp >>>>>> >>>>>> Nit: space before ( again (on handle constructor) >>>>> >>>>> fixed. >>>>>> >>>>>>> It's a very small change to a number of files. Notably the new >>>>>>> JVMCI >>>>>>> files have a lot of implicit conversions from oop to handle. I >>>>>>> want to >>>>>>> get this change in early for jdk10 to stop this usage. I also >>>>>>> added a >>>>>>> few HandleMarks where I found that the HandlesArea was growing >>>>>>> large. >>>>>> >>>>>> The addition and removal of the HandleMarks seems problematic >>>>>> because >>>>>> there are no obvious rules being applied. It is hard to know >>>>>> where and >>>>>> when a HandleMark is needed or not. Looking at >>>>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>>>> create/destroy the HandleMark on each loop iteration? That would >>>>>> only >>>>>> seem necessary if we know the number of iterations is so high >>>>>> that we >>>>>> might consume all our handle space. >>>>> >>>>> Nobody knows when and where to put HandleMarks. It's a dark art. I >>>> >>>> So a trade-off between too much cleanup overhead if we have too >>>> many, and too much GC overhead if we have too few. And no real way >>>> to know either way - Ouch! :( >>>> >>>>> also was testing with a JVM that restored the assert if HandleArea >>>>> got >>>>> over 200 handles and the numbers got very high in these >>>>> functions. The >>>>> GC has to walk these HandleAreas and I am trying to reduce work >>>>> for them. >>>>> >>>>> I tried to write about this in the bug. I'd like to see if changing >>>>> Handle to a proper constructor/destructor like methodHandles, so that >>>>> only the Handles that are active need to be collected. Then we don't >>>>> need HandleMark and their associated mystery. There are several >>>>> HandleMarks in code that obviously doesn't allocate any Handles. >>>>> I took >>>>> out several but restored them so that this patch was smaller. To >>>>> change Handle to a scoped object would require changing passing >>>>> Handle >>>>> as a const reference like I did with methodHandle which is too big >>>>> of a >>>>> change for early-jdk10. >>>> >>>> Ok. This sounds like a way forward though. In combination with an >>>> easy way to detect when handles are required, this would allow very >>>> precise management of handle lifetimes. >>> >>> Good, I'm glad you think so too. >>>> >>>>> The reason I worked on this RFE was because I was curious to see how >>>>> many oops/Handles we're passing around and using from within the JVM >>>>> these days. It's more than I thought. >>>>> >>>>> Thank you for reviewing this. I'm going to rerun some tests with the >>>>> changes above and send another webrev. >>>> >>>> Ok. Thanks. >>> >>> Thanks for the discussion and review! >>> >>> Coleen >>>> >>>> David >>>> ----- >>>> >>>>> Coleen >>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ------ >>>>>> >>>>>>> See bug for more on the motivation of this change. >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>>>> >>>>>>> Tested by running all hotspot jtreg tests with >>>>>>> -XX:+CheckUnhandledOops. >>>>>>> There weren't any unhandled oops amazingly. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>> >>> >> > From coleen.phillimore at oracle.com Wed Feb 15 18:42:03 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 13:42:03 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <519f62ac-d353-7d4c-5f6f-1acb722c1a35@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> <519f62ac-d353-7d4c-5f6f-1acb722c1a35@oracle.com> Message-ID: Thank you Serguei, one tiny comment below. On 2/15/17 11:50 AM, serguei.spitsyn at oracle.com wrote: > Coleen, > > > On 2/15/17 07:43, coleen.phillimore at oracle.com wrote: >> >> Hi Serguei, Thank you for reviewing this. >> >> On 2/15/17 2:18 AM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> >>> Thank you for the review discussion with David! >>> It helps to understand the webrev details. >>> >>> It looks good in general. >>> Some comments/questions below. >>> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCodeInstaller.cpp.udiff.html >>> >>> >>> for (jint i = 0; i < values->length(); i++) { >>> ScopeValue* cur_second = NULL; >>> - Handle object = values->obj_at(i); >>> - BasicType type = >>> JVMCIRuntime::kindToBasicType(slotKinds->obj_at(i), CHECK); >>> + Handle object (THREAD, values->obj_at(i)); >>> + Handle slot_kind (THREAD, slotKinds->obj_at(i)); >>> + BasicType type = JVMCIRuntime::kindToBasicType(slot_kind, CHECK); >>> >>> >>> Do we need a HandleMark in the loop? >>> >>> >> Yes, I don't think it would hurt. Added. > > Thanks! >> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCompilerToVM.cpp.udiff.html >>> >>> >>> Handle result; >>> if (!klass.is_null()) { >>> - result = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >>> + oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >>> + result = Handle(THREAD, result_oop); >>> } else { >>> result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); >>> } >>> return JNIHandles::make_local(THREAD, result()); >>> >>> >>> Not clear, why does the result need to be a handle. >>> >> >> There's no reason this had to be a Handle, but >> java_lang_String::create_from_symbol returns a Handle. That's >> why. This seems better, agree? >> >> oop result_oop; >> if (!klass.is_null()) { >> oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >> } else { >> Handle result = java_lang_String::create_from_symbol(symbol, >> CHECK_NULL); >> result_oop = result(); >> } >> return JNIHandles::make_local(THREAD, result_oop); > > Agreed. > Alternatively, this can be used: > result_oop = java_lang_String::create_from_symbol(symbol, > CHECK_NULL)(); > The function create_from_symbol returns a Handle. It seems like it'd be better for all functions to pass Handle but return oops, but there are some that return Handle because the last argument is CHECK which becomes: THREAD); if (HAS_PENDING_EXCEPTION) { return NULL; } And if you try to Handle the result in one statement, like this: result = Handle(THREAD, java_lang_String::create_from_symbol(symbol, CHECK_NULL)); it won't compile because of the CHECK_NULL. So we made many functions like this return Handle. Thanks, Coleen >> >>> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.udiff.html >>> >>> >>> + const char* javapkg = "java/"; . . . - >>> strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { >>> + strncmp(class_name->as_C_string(), javapkg, strlen(javapkg)) == 0) { >>> >>> >>> Why this change is needed? >>> It is a reverse of the recent Rachel's fix. >> >> I messed this up in the integration. Thank you for finding it! This >> was the only diff in that function: >> >> @@ -2460,7 +2463,7 @@ >> TRAPS) { >> ResourceMark rm(THREAD); >> if (!class_loader.is_null() && >> - !SystemDictionary::is_platform_class_loader(class_loader) && >> + !SystemDictionary::is_platform_class_loader(class_loader()) && >> class_name != NULL && >> strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { >> TempNewSymbol pkg_name = >> InstanceKlass::package_from_name(class_name, CHECK); > > Ok, thanks! > >> >>> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html >>> >>> >>> 701 void InstanceKlass::initialize_impl(instanceKlassHandle this_k, >>> TRAPS) { >>> 702 HandleMark hm(THREAD); >>> . . . >>> 712 // refer to the JVM book page 47 for description of steps >>> 713 // Step 1 >>> 714 { >>> 715 HandleMark hm(THREAD); >>> >>> >>> It is not clear what is the reason for one more HandleMark at 715. >>> >>> >> >> I'll delete the second one. I added it for debugging number of Handles. > > Thanks, Coleen! > I have no more comments - reviewed. > > Thanks, > Serguei > >> >> Thanks for reviewing! >> Coleen >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 2/14/17 14:21, coleen.phillimore at oracle.com wrote: >>>> >>>> New webrev: >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev >>>> >>>> see coments below first. >>>> >>>> On 2/13/17 11:51 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>>>>> >>>>>> >>>>>> On 2/13/17 8:44 PM, David Holmes wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> "actually S"? :) >>>>>> >>>>>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>>>>> backports. >>>>>> >>>>>>> >>>>>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>>>>> Thread::current() call. >>>>>>> >>>>>>> Well there's a bit more to it than that as you also change >>>>>>> parameter >>>>>>> types so that we (sometimes) avoid: >>>>>>> >>>>>>> - oop -> Handle -> oop >>>>>>> - Handle -> oop -> Handle >>>>>> >>>>>> Yes, I put that in the bug but not in the RFR. The >>>>>> recommendation is to >>>>>> Handle the oop as far up as possible in the call stack and pass >>>>>> Handle >>>>>> around. >>>>> >>>>> I hear what you are saying but it is hard to actually see that >>>>> reflected in the code. >>>>> >>>>>>> across method calls. This leads to some puzzling differences eg: >>>>>>> >>>>>>> src/share/vm/aot/aotCodeHeap.cpp >>>>>>> src/share/vm/classfile/javaClasses.hpp >>>>>>> >>>>>>> The change in aotCodeHeap.cpp reverses the use of Handle and >>>>>>> oop, but >>>>>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>>>>> have different signatures in that regard. They used to both take a >>>>>>> Handle but now one takes an oop instead. ?? >>>>>> >>>>>> Yes, you found the exceptions in Throwable. A lot of the >>>>>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop >>>>>> instead >>>>>> of Handle and I changed print to oop to make it consistent, and >>>>>> because >>>>>> they it being called from other functions in Throwable with oop. >>>>>> Throwable::print_stack_trace actually goes to a safepoint so has >>>>>> to be >>>>>> Handle. >>>>> >>>>> I think in javaClasses.hpp the rule should be oop unless Handle is >>>>> definitely needed. >>>> >>>> It's easiest to keep it an oop because most of the functions just >>>> do an obj_field, etc. >>>> >>>> oop java_lang_Throwable::message(oop throwable) { >>>> return throwable->obj_field(detailMessage_offset); >>>> } >>>> >>>> but it's likely that the oop throwable is a handle up the call >>>> stack. Interestingly looking at it's callers is one in >>>> javaClasses.cpp that's an oop, one in exceptions.cpp that's a Handle, >>>> >>>> oop msg = java_lang_Throwable::message(exception()); >>>> >>>> And then there's the one in universe.cpp that's a naked oop, which >>>> can get moved before the call. >>>> >>>> // get the error object at the slot and set set it to NULL so >>>> that the >>>> // array isn't keeping it alive anymore. >>>> oop exc = preallocated_out_of_memory_errors()->obj_at(next); >>>> assert(exc != NULL, "slot has been used already"); >>>> preallocated_out_of_memory_errors()->obj_at_put(next, >>>> NULL); <= This on G1 can take out a lock for the oop_store and >>>> GC and move the default_err >>>> >>>> // use the message from the default error >>>> oop msg = java_lang_Throwable::message(default_err); <= >>>> default_err is an oop. >>>> assert(msg != NULL, "no message"); >>>> java_lang_Throwable::set_message(exc, msg); >>>> >>>> exc is a naked oop too. I don't think the naked oop detector >>>> triggers for obj_at_put/oop_store because G1 wasn't in the code >>>> when it was added. >>>> >>>> Creating a Handle and passing it saves having to visually inspect >>>> functions to try to find naked oops. >>>> >>>> Fixing this to have a local handle though because the callers don't >>>> have THREAD. >>>>> >>>>>> >>>>>>> >>>>>>> More generally it is unclear why you made the changes you did in >>>>>>> places - see below for some specifics. I'm left questioning how to >>>>>>> know when to take an oop and when to take a Handle. Of course that >>>>>>> question already exists, but these changes didn't make it any >>>>>>> clearer >>>>>>> for me. >>>>>> >>>>>> Pass a Handle early and often is still the rule. For all but GC >>>>>> code. >>>>>> There are still some functions that are very small and don't >>>>>> safepoint >>>>>> and take oop. Some of these have migrated to taking Handle, but >>>>>> some >>>>>> haven't. I didn't want to make this change larger for now. >>>>>> >>>>>>> >>>>>>> I understand why the oop to Handle implicit conversion could be >>>>>>> problematic due to the Thread::current() calls, but I don't >>>>>>> understand >>>>>>> why you couldn't keep (? add?) the Handle to oop implicit >>>>>>> conversions ?? >>>>>> >>>>>> There weren't Handle -> oop implicit conversions. The conversion >>>>>> back >>>>>> to oop uses the operator() which remains. >>>>> >>>>> Okay then it seems an implicit Handle -> oop would avoid the need >>>>> to change the caller from foo to foo() if the underlying API >>>>> switched from oop to Handle. >>>> >>>> Yes but it's better to just pass the oop along until something >>>> really needs to do something with it. >>>>> >>>>>> The oop -> Handle conversion has cost, both at runtime and also >>>>>> to the >>>>>> GC because it increases the number of Handles on the >>>>>> Thread->_handle_area. >>>>> >>>>> Sure. >>>>> >>>>>> I did this same experiment years ago when Thread::current() was >>>>>> at the >>>>>> top of the profiling call stack but back then I needed to add too >>>>>> many >>>>>> explicit Thread::current() calls. There aren't many in this >>>>>> change, so >>>>>> it's worth doing. Also, metadata handles have the same problem but >>>>>> there are still too many of them to remove the implicit conversion. >>>>>> And {instance}KlassHandle needs to be removed first. They do nothing >>>>>> now but hide the type. >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/c1/c1_Runtime1.cpp >>>>>>> >>>>>>> Aside: I'm a little surprised that there was not a Handle >>>>>>> assignment >>>>>>> operator or copy constructor such that given: >>>>>>> >>>>>>> 863 Handle appendix(THREAD, NULL); >>>>>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>>>>> >>>>>>> that it simply updated the NULL to the desired value, while keeping >>>>>>> the THREAD intact. >>>>>> >>>>>> Handle doesn't save the THREAD so a NULL handle is just oop* >>>>>> NULL. The >>>>>> RHS of the expression has to be converted to Handle and the default >>>>>> assignment operator copies it to appendix. >>>>> >>>>> I had assumed Handle saved the Thread. >>>> >>>> No, then it would be two fields. It's only one so passing it is >>>> cheap now. >>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/ci/ciInstance.cpp >>>>>>> >>>>>>> Not at all obvious that replacing Handle with raw oop is >>>>>>> correct. I'm >>>>>>> not saying it isn't, just that it isn't obvious - and I'll >>>>>>> wonder why >>>>>>> the Handle was used in the first place. >>>>>> >>>>>> It is safe because it's immediately unhandled in all the case >>>>>> statements, and I wanted to avoid a Thread::current call. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/classfile/javaClasses.cpp >>>>>>> >>>>>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>>>>> instead of an oop if you are simply going to unwrap the Handle >>>>>>> to get >>>>>>> the oop back. ??? Particular when a caller like >>>>>>> src/share/vm/prims/methodHandles.cpp has to create the Handle >>>>>>> from oop >>>>>>> in the first place. >>>>>> >>>>>> I changed it to be the same as java_lang_String::as_symbol(Handle >>>>>> java_string ...) which took a handle. >>>>>> java_lang_String::as_symbol() was >>>>>> the same - it immediately unhandles the argument to pass to the >>>>>> rest of >>>>>> the functions. I changed java_lang_String::as_symbol() to take >>>>>> an oop >>>>>> to match, and there are few scattered changes from that. >>>>> >>>>> I have to disagree with this one. AFAICS there is no reason this: >>>>> >>>>> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >>>>> >>>>> needs to take a Handle in the first place. The java_string is >>>>> unwrapped to get the oop and the oop is only passed to a few other >>>>> String methods and then not touched. Looking at the API's in >>>>> javaClass.hpp the norm should be to take oop (as the bulk of >>>>> methods do) with Handle only used when actually required. >>>> >>>> I changed this one to take oop because it was obvious on inspection >>>> but note that the "required" is sometimes really hard to see! >>>> Chasing naked oop problems isn't something we do very much anymore >>>> because we converted many arguments to Handles. Note that the >>>> runtime only passes oops much of the time and doesn't usually do >>>> anything with them. The exception is javaClasses.cpp though. >>>> >>>>> >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>>>>> >>>>>>> It is not at all obvious that JavaArgumentUnboxers are >>>>>>> thread-confined! I'm also concerned by API's (unfortunately a >>>>>>> number >>>>>>> pre-existing) that take a Thread/JavaThread argument but really >>>>>>> require it to be the current thread. To that end I'd rather see >>>>>>> _thread as _cur_thread and an assertion when it is set. >>>>>> >>>>>> Since it's a ResourceObj and not StackObj (setting aside debate >>>>>> about >>>>>> that), I agree with you. I changed it to use Thread::current() >>>>>> where it >>>>>> handles the object, which it explicitly did before. In this >>>>>> case, it is >>>>>> a stack allocated thing but it's not guaranteed to be that. >>>>> >>>>> Ok. >>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/oops/cpCache.cpp >>>>>>> >>>>>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>>>>> >>>>>>> Please remove space before ( >>>>>> >>>>>> fixed. >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/prims/jvm.cpp >>>>>>> >>>>>>> Nit: need to fix indent on StackWalk::walk call second line. >>>>>> >>>>>> fixed. >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>>>>> >>>>>>> Ditto re _thread -> _cur_thread >>>>>>> >>>>>> >>>>>> This one is a StackObj (a Closure which should be used that way). >>>>>> So I >>>>>> changed _thread to _cur_thread and added an assert that it == >>>>>> Thread::current() in the constructor. >>>>> >>>>> Thanks. >>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/prims/jvmtiImpl.cpp >>>>>>> >>>>>>> Nit: space before ( again (on handle constructor) >>>>>> >>>>>> fixed. >>>>>>> >>>>>>>> It's a very small change to a number of files. Notably the new >>>>>>>> JVMCI >>>>>>>> files have a lot of implicit conversions from oop to handle. I >>>>>>>> want to >>>>>>>> get this change in early for jdk10 to stop this usage. I also >>>>>>>> added a >>>>>>>> few HandleMarks where I found that the HandlesArea was growing >>>>>>>> large. >>>>>>> >>>>>>> The addition and removal of the HandleMarks seems problematic >>>>>>> because >>>>>>> there are no obvious rules being applied. It is hard to know >>>>>>> where and >>>>>>> when a HandleMark is needed or not. Looking at >>>>>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>>>>> create/destroy the HandleMark on each loop iteration? That would >>>>>>> only >>>>>>> seem necessary if we know the number of iterations is so high >>>>>>> that we >>>>>>> might consume all our handle space. >>>>>> >>>>>> Nobody knows when and where to put HandleMarks. It's a dark art. I >>>>> >>>>> So a trade-off between too much cleanup overhead if we have too >>>>> many, and too much GC overhead if we have too few. And no real way >>>>> to know either way - Ouch! :( >>>>> >>>>>> also was testing with a JVM that restored the assert if >>>>>> HandleArea got >>>>>> over 200 handles and the numbers got very high in these >>>>>> functions. The >>>>>> GC has to walk these HandleAreas and I am trying to reduce work >>>>>> for them. >>>>>> >>>>>> I tried to write about this in the bug. I'd like to see if >>>>>> changing >>>>>> Handle to a proper constructor/destructor like methodHandles, so >>>>>> that >>>>>> only the Handles that are active need to be collected. Then we don't >>>>>> need HandleMark and their associated mystery. There are several >>>>>> HandleMarks in code that obviously doesn't allocate any Handles. >>>>>> I took >>>>>> out several but restored them so that this patch was smaller. To >>>>>> change Handle to a scoped object would require changing passing >>>>>> Handle >>>>>> as a const reference like I did with methodHandle which is too >>>>>> big of a >>>>>> change for early-jdk10. >>>>> >>>>> Ok. This sounds like a way forward though. In combination with an >>>>> easy way to detect when handles are required, this would allow >>>>> very precise management of handle lifetimes. >>>> >>>> Good, I'm glad you think so too. >>>>> >>>>>> The reason I worked on this RFE was because I was curious to see how >>>>>> many oops/Handles we're passing around and using from within the JVM >>>>>> these days. It's more than I thought. >>>>>> >>>>>> Thank you for reviewing this. I'm going to rerun some tests with >>>>>> the >>>>>> changes above and send another webrev. >>>>> >>>>> Ok. Thanks. >>>> >>>> Thanks for the discussion and review! >>>> >>>> Coleen >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> Coleen >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ------ >>>>>>> >>>>>>>> See bug for more on the motivation of this change. >>>>>>>> >>>>>>>> open webrev at >>>>>>>> http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>>>>> >>>>>>>> Tested by running all hotspot jtreg tests with >>>>>>>> -XX:+CheckUnhandledOops. >>>>>>>> There weren't any unhandled oops amazingly. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>> >>>> >>> >> > From kim.barrett at oracle.com Wed Feb 15 19:47:43 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 15 Feb 2017 14:47:43 -0500 Subject: RFR: 8166188: G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <1519B166-C6A7-45A3-9C68-105177D9D601@oracle.com> <66F01296-FCA8-4C60-9AF3-FB937E5F3EDA@oracle.com> <1EA40A56-7BEB-44C8-9D24-98904DA003A9@oracle.com> <745EFBBD-AA95-4A5C-A4FB-56F41F8A778E@oracle.com> Message-ID: > On Feb 15, 2017, at 11:34 AM, Daniel D. Daugherty wrote: > > On 2/12/17 9:25 PM, Kim Barrett wrote: >> Here's the updated webrevs, incorporating feedback received so far. >> >> full: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05/ Thanks Dan. > > src/share/vm/runtime/javaCalls.hpp > No comments. > > src/share/vm/runtime/javaCalls.cpp > No comments. > > src/share/vm/runtime/jniHandles.hpp > No comments. > > src/share/vm/runtime/jniHandles.cpp > No comments. > > src/share/vm/prims/jni.cpp > No comments. > > src/share/vm/prims/jvmtiEnv.cpp > No comments. > > make/test/JtregNative.gmk > No comments. > > src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp > No comments. > > src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp > No comments. > > src/cpu/arm/vm/interp_masm_arm.cpp > No comments. > > src/cpu/arm/vm/interp_masm_arm.hpp > No comments. > > src/cpu/arm/vm/macroAssembler_arm.cpp > No comments. > > src/cpu/arm/vm/macroAssembler_arm.hpp > No comments. > > src/cpu/arm/vm/sharedRuntime_arm.cpp > No comments. > > src/cpu/arm/vm/templateInterpreterGenerator_arm.cpp > No comments. > > src/cpu/ppc/vm/frame_ppc.cpp > No comments. > > src/cpu/ppc/vm/sharedRuntime_ppc.cpp > No comments. > > src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp > No comments. > > src/cpu/s390/vm/sharedRuntime_s390.cpp > No comments. > > src/cpu/s390/vm/templateInterpreterGenerator_s390.cpp > No comments. > > src/cpu/sparc/vm/sharedRuntime_sparc.cpp > No comments. > > src/cpu/sparc/vm/templateInterpreterGenerator_sparc.cpp > No comments. > > src/cpu/x86/vm/macroAssembler_x86.cpp > No comments. > > src/cpu/x86/vm/macroAssembler_x86.hpp > No comments. > > src/cpu/x86/vm/sharedRuntime_x86_32.cpp > No comments. > > src/cpu/x86/vm/sharedRuntime_x86_64.cpp > No comments. > > src/cpu/x86/vm/templateInterpreterGenerator_x86.cpp > No comments. > > src/cpu/zero/vm/cppInterpreter_zero.cpp > No comments. > > src/share/vm/shark/sharkNativeWrapper.cpp > No comments. > > test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java > No comments. > > test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c > No comments. > > test/runtime/jni/ReturnJNIWeak/ReturnJNIWeak.java > No comments. > > test/runtime/jni/ReturnJNIWeak/libReturnJNIWeak.c > No comments. > > Thumbs up! > > Dan > > >> incr: http://cr.openjdk.java.net/~kbarrett/8166188/hotspot.05.inc/ >> >> For x86 and ARM I added resolve_jobject functions to their respective >> macroAssembler files, and call those to generate the return object >> handling code. Merging the two cases for Sparc is harder and I'm >> going to leave it to someone else. I'm also leaving any such merging >> for other (non-Oracle) ports to their respective maintainers. >> >> Rewrote the error checking in SignatureChekker::check_obj to simplify >> the control flow and provide more information. Rather than >> conditionally selecting between ReportJNIFatalError or assert to >> report problems, instead just always use guarantee. Recall that we >> only get here if CheckJNICalls or in a debug build. This made >> SignatureChekker::_thread unused, so removed it. From serguei.spitsyn at oracle.com Wed Feb 15 20:36:24 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 15 Feb 2017 12:36:24 -0800 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> <519f62ac-d353-7d4c-5f6f-1acb722c1a35@oracle.com> Message-ID: <373d6153-e0d1-1e12-634e-64f1f09fd5ee@oracle.com> On 2/15/17 10:42, coleen.phillimore at oracle.com wrote: > Thank you Serguei, one tiny comment below. > > On 2/15/17 11:50 AM, serguei.spitsyn at oracle.com wrote: >> Coleen, >> >> >> On 2/15/17 07:43, coleen.phillimore at oracle.com wrote: >>> >>> Hi Serguei, Thank you for reviewing this. >>> >>> On 2/15/17 2:18 AM, serguei.spitsyn at oracle.com wrote: . . . >>>> >>>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCompilerToVM.cpp.udiff.html >>>> >>>> >>>> Handle result; >>>> if (!klass.is_null()) { >>>> - result = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >>>> + oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >>>> + result = Handle(THREAD, result_oop); >>>> } else { >>>> result = java_lang_String::create_from_symbol(symbol, >>>> CHECK_NULL); >>>> } >>>> return JNIHandles::make_local(THREAD, result()); >>>> >>>> >>>> Not clear, why does the result need to be a handle. >>>> >>> >>> There's no reason this had to be a Handle, but >>> java_lang_String::create_from_symbol returns a Handle. That's >>> why. This seems better, agree? >>> >>> oop result_oop; >>> if (!klass.is_null()) { >>> oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >>> } else { >>> Handle result = java_lang_String::create_from_symbol(symbol, >>> CHECK_NULL); >>> result_oop = result(); >>> } >>> return JNIHandles::make_local(THREAD, result_oop); >> >> Agreed. >> Alternatively, this can be used: >> result_oop = java_lang_String::create_from_symbol(symbol, >> CHECK_NULL)(); >> > > The function create_from_symbol returns a Handle. It seems like it'd > be better for all functions to pass Handle but return oops, but there > are some that return Handle because the last argument is CHECK which > becomes: > > THREAD); if (HAS_PENDING_EXCEPTION) { > return NULL; > } > > And if you try to Handle the result in one statement, like this: > > result = Handle(THREAD, > java_lang_String::create_from_symbol(symbol, CHECK_NULL)); > > it won't compile because of the CHECK_NULL. > So we made many functions like this return Handle. Ok, thanks. Serguei > > Thanks, > Coleen From coleen.phillimore at oracle.com Wed Feb 15 20:38:54 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 15:38:54 -0500 Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> Message-ID: <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> Shafi, This looks good to me also. Thank you for fixing this. Coleen On 2/15/17 6:35 AM, Shafi Ahmad wrote: > Hi All, > > Summary: Adding return value check and update index variable. It's a very small change to single file. > > Webrev link: http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ > bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 > > Testing: jprt and jtreg test. > > Regards, > Shafi From coleen.phillimore at oracle.com Wed Feb 15 20:46:35 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 15:46:35 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> Message-ID: On 2/15/17 10:07 AM, Erik Helin wrote: > On 02/15/2017 02:48 AM, David Holmes wrote: >> Hi Erik, > > Hi David, > > thanks for having a look! Please see new patches at: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ > >> oops_do can miss an oop that is in the process of being added - is that >> a problem? > > Nope, that is not an issue. Even if `add_handle` and `oops_do` where > synchronized with each other (for example by using lock), you can > still have the scenario where a GC thread first runs (and finishes) > `oops_do` and then `add_handle` is called (a linearized execution). > > This scenario is something the concurrent marking algorithms already > should be prepared to handle. > >> 140 if (c->data[i] != NULL) { >> 141 f->do_oop(&c->data[i]); >> 142 } >> >> Given a slot can be nulled out concurrently at any time, is it worth >> does this NULL check? The OopClosure will have to do its own NULL check >> anyway. > > Sure, this was mostly to be a bit consistent with JNIHandleBlock (it > does the same NULL check). Think of it as an optimization, there is no > reason to call `f->do_oop` if `c->data[i] == NULL`. Do you think it > reads better if the NULL check is removed? I don't have a strong > opinion on this one. > >> 144 c = (Chunk*) OrderAccess::load_ptr_acquire((volatile >> intptr_t*)&c->next); >> >> This doesn't need to be a load-acquire. You already loaded 'c' via >> load-acquire of _head (or chained therefrom) and its next field is set >> prior to the setting of the _head that you read. > > Agree. I just discussed this with Erik ? as well, and we all agree on > this. I also removed the `volatile` specifier for `next`. > >> 624 jobject ClassLoaderData::add_handle(Handle h) { >> 625 MutexLockerEx ml(metaspace_lock(), >> Mutex::_no_safepoint_check_flag); >> 626 return (jobject) _handles.add(h()); >> 627 } >> 628 >> 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { >> 630 assert(_handles.contains((oop*) h), "Got unexpected handle " >> PTR_FORMAT, p2i((oop*) h)); >> 631 *((oop*) h) = NULL; >> 632 } >> >> I'm a bit unclear on the typing here. Previously we use a JNI Handle >> which was a jobject. But now we've simply stored an oop into a slot in >> an array. We pass the address of that slot back as a jobject, but is it >> really? > > So, this is a bit confusing, and I discussed this with Coleen a few > days ago. jobject was used originally used because that is what > `JNIHandleBlock::allocate_handle` returns. Using something other than > jobject means updating `ModuleEntry` and in particular users > `ModuleEntry::_pd`. This is not something we are keen on doing right > now for JDK 9, and Coleen is planning to clean up how the runtime > handles oops for 10 (or at least improve the situation). Yes, these are not really jobject if you think of jobject and jweak as being native handles to oops. These really are indirect pointers to oops that we use in the runtime. We're still discussing how we want these to look for JDK10. Your suggestions are welcome - we'll include you on some email as we discuss this (and openjdk also). > > I don't know what jobject is meant to represent, in the code it is > just an empty class (an opaque type). > > ClassLoaderData::add_handle could just as well return an oop*, but > AFAIU we try to avoid oops and oop* in the runtime code (because that > might mean that some code forgot to tell the GC about the oop). Maybe > I'm wrong here? You are right. We don't want to have oop* in the runtime code because it also is unclear if it was supposed to be compressed or not. We want a special name for these. Thanks, Coleen > > > I would also expect contains to take an oop not an oop* - it > > seems to be asking "is this an address of a slot in our > > ChunkedHandleList" rather than asking "is this oop in our > > ChunkedHandleList". ?? > > I actually wanted the assert to check whether the passed jobject > (which really is an oop*) actually originated from a call to > ClassLoaderData::add_handle. I don't want any code to pass a jobject > to ClassLoaderData::remove_handle_unsafe that doesn't come from a call > to ClassLoaderData::add_handle. > >> A few minor comments: >> >> Copyrights need updating. > > Aaah, it is that time of the year again. Fixed. > >> src/share/vm/classfile/classLoaderData.hpp >> >> 177 // Only on thread ... >> >> Typo: on -> one > > Thanks, fixed. > >> src/share/vm/classfile/moduleEntry.cpp >> >> 90 // Create a JNI handle for the shared ProtectionDomain and save it >> atomically. >> 91 // If someone beats us setting the _pd cache, the created JNI >> handle is destroyed. >> >> These are no longer JNI Handles. > > Thanks, I updated the comment. > > Thanks, > Erik > >> Thanks, >> David >> ----- >> >>> Testing: >>> - Tier 1 (aka JPRT), 2, 3, 4, 5. >>> >>> I would appreciate quite a few reviews on this patch, it contains a >>> nice >>> mixture of: >>> - me venturing into the runtime code :) >>> - lock free code >>> - unreproducible bug (even though I'm sure of the root cause) >>> >>> Thanks for everyone participating in the discussions around this bug >>> and >>> potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and >>> Jiangli. >>> >>> Thanks! >>> Erik >>> >>> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From coleen.phillimore at oracle.com Wed Feb 15 21:08:12 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 16:08:12 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> Message-ID: <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> Hi Erik, I thought you were going to hide the declaration of class ChunkedHandleList in classLoaderData.cpp since it is only used internally? And have a field in ClassLoaderData: ChunkedHandleList* _chunked_handle_list; The class name "Chunk" is so overused! Here ChunkedHandleList should not be a subtype of CHeapObj. It's a ValueObj. I don't think it should be a struct either because even though it's internal to ClassLoaderData, the fields could still be accessed outside the class. Why is 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { it called unsafe? It is safe to zero the element in the block, isn't it? One other reason to make it internal is that the CAPACITY of 64 is way too big for the ClassLoaderData of anonymous classes which I think will only have presently 1 (or 2 maybe) oops in it. And there are zillions of them. Thanks, Coleen On 2/14/17 11:40 AM, Erik Helin wrote: > Hi all, > > this patch aims to solve the bug [0] by making it safe for a GC to > concurrently traverse the oops in a ClassLoaderData. > > The problem right now is that ClassLoaderData::_handles is a > JNIHandleBlock. It is not safe for one thread to iterate over the oops > in a JNIHandleBlock while another thread concurrently adds a new oop > to the JNIHandleBlock. > > My proposed solution is to replace JNIHandleBlock with another data > structure for ClassLoaderData. ClassLoaderData only needs a place to > store oops that a GC can iterate over, it has no use for the rest of > the methods in the JNIHandleBlock class. I have implemented a minimal > chunked linked list data structure (internal to ClassLoaderData) with > only two public methods: > - add (only executed by one thread at a time) > - oops_do (can be executed by any number of threads, possibly > concurrently with a call to `add`) > > ChunkedHandleList uses load_acquire and release_store to synchronize > access to the underlying chunks (arrays). Since ChunkedHandleList > utilizes the (rather) minimal requirements of its user > (ClassLoaderData), I decided to keep the class internal to > ClassLoaderData for now. If other find it useful elsewhere, the we can > try to create a more generalized version (this is trickier than it > appears at first sight, I tried ;)). > > I also changed ClassLoaderData::remove_handle to write NULL to the > oop* (that is represented by a jobject), instead of using a sentinel > oop as done by JNIHandleBlock. The GC closures should be prepared to > handle a field in a Java object becoming NULL, so this should be fine. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8168914 > > Webrev: > http://cr.openjdk.java.net/~ehelin/8168914/00/ > > Testing: > - Tier 1 (aka JPRT), 2, 3, 4, 5. > > I would appreciate quite a few reviews on this patch, it contains a > nice mixture of: > - me venturing into the runtime code :) > - lock free code > - unreproducible bug (even though I'm sure of the root cause) > > Thanks for everyone participating in the discussions around this bug > and potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and > Jiangli. > > Thanks! > Erik > > [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From coleen.phillimore at oracle.com Wed Feb 15 21:11:42 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 16:11:42 -0500 Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: References: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> <9d506110-91a1-4827-99a8-e46e9363eb3d@default> <5bec7006-4301-7f2a-88c9-dd685e447d15@oracle.com> Message-ID: <73ff2d81-0791-658b-3ef3-7f6191e0b8a9@oracle.com> Yes this looks very good. Do you have a test for it? Or is it in the jck tests and verified manually for each case? Thanks, Coleen On 2/15/17 1:29 AM, Shafi Ahmad wrote: > Thank you David for reviewing it. > > All, > > May I get second reviewer for this trivial change. > > Regards, > Shafi > > > >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, February 15, 2017 11:54 AM >> To: Shafi Ahmad ; hotspot- >> dev at openjdk.java.net >> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field >> name&signature in class file" should report the name and signature of the >> field >> >> Looks good to me! Nice and neat. >> >> Thanks, >> David >> >> On 15/02/2017 4:02 PM, Shafi Ahmad wrote: >>> Hi David, >>> >>> Please find updated webrev at >>> http://cr.openjdk.java.net/~shshahma/8171194/webrev.01/ >>> Added overloaded method classfile_parse_error. >>> >>> Regards, >>> Shafi >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Wednesday, February 15, 2017 3:55 AM >>>> To: Shafi Ahmad ; hotspot- >>>> dev at openjdk.java.net >>>> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field >>>> name&signature in class file" should report the name and signature of >>>> the field >>>> >>>> Hi Shafi, >>>> >>>> On 14/02/2017 11:20 PM, Shafi Ahmad wrote: >>>>> Hi David, >>>>> >>>>> Thanks for reviewing it. >>>>> >>>>> Initially I started with fixed size of local char array but later I >>>>> changed my >>>> mind and make it dynamic. >>>>> Let me know if I have to make it local char array like. >>>>> >>>>> + unsigned int siglength = sig->utf8_length(); >>>>> + unsigned int namelength = name->utf8_length(); >>>>> + unsigned int length = siglength + namelength + 64; >>>>> + char *buff = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, >>>> length); >>>>> + jio_snprintf(buff, length, >>>>> + "Duplicate method name \"%s\" with signature >>>>> + \"%s\" in class >>>> file", >>>>> + name->as_C_string(), sig->as_klass_external_name()); >>>>> + classfile_parse_error("%s %s", buff, CHECK); >>>>> } >>>>> >>>>> to >>>>> >>>>> + char buff[fixedsize]; // say fixedsize is 512 >>>>> + jio_snprintf(buff, 512, >>>>> + "Duplicate method name \"%s\" with signature >>>>> + \"%s\" in class >>>> file", >>>>> + name->as_C_string(), sig->as_klass_external_name()); >>>>> + classfile_parse_error("%s %s", buff, CHECK); >>>>> } >>>> It could still be dynamic, you just need to use >>>> NEW_RESOURCE_ARRAY_IN_THREAD_RETURN_NULL and check for a >> NULL return, >>>> and fallback to not including the additional info. But the underlying >>>> Exceptions::fthrow uses a local buffer itself (1024 max), so you >>>> could just do the same as you propose above. >>>> >>>> Though it is very annoying to have to allocate a buffer just to pass >>>> it through to classfile_parse_error which passes it through to >>>> Exceptions::fthrow which is a varargs method. So another possibility >>>> is to just add another overload of classfile_parse_error that takes >>>> in the two additional strings you want to print. >>>> >>>> Thanks, >>>> David >>>> >>>>> Regards, >>>>> Shafi >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Tuesday, February 14, 2017 6:34 PM >>>>>> To: Shafi Ahmad ; hotspot- >>>>>> dev at openjdk.java.net >>>>>> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field >>>>>> name&signature in class file" should report the name and signature >>>>>> of the field >>>>>> >>>>>> Hi Shafi, >>>>>> >>>>>> I'm concerned about the use of NEW_RESOURCE_ARRAY_IN_THREAD. >> If >>>> it >>>>>> can't allocate it will abort the VM. That seems like a bad thing to >> happen. >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> On 14/02/2017 7:19 PM, Shafi Ahmad wrote: >>>>>>> Summary: java.lang.ClassFormatError: Exception "Duplicate field >>>>>> name&signature in class file" should report the name and signature >>>>>> of the field. >>>>>>> It's a very small change to single file. >>>>>>> In the current implementation name and signature of duplicate >>>>>>> field is >>>>>> missing in java.lang.ClassFormatError exception message. >>>>>>> Without a field name + signature it is hard to triggering the problem. >>>>>>> >>>>>>> Webrev link: >>>> http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ >>>>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 >>>>>>> >>>>>>> Testing: jprt and jtreg test. >>>>>>> >>>>>>> I have verified my changes with the reproduces of >>>>>> https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 >>>>>> code base as I was not able to write reproducer of current issue. >>>>>>> With the fix I am getting below exception message. >>>>>>> >>>>>>> $ java CreateBadClassFile >>>>>>> .foreach() call: Exception in thread "main" java.lang.ClassFormatError: >>>>>> Duplicate field name "hasNext" with signature "Ljava.lang.Object;" >>>>>> in class file WidgetCollection$1 >>>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:760) >>>>>>> at >>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java: >>>>>> 14 >>>>>> 2) >>>>>>> . . . >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> From coleen.phillimore at oracle.com Wed Feb 15 21:52:49 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 16:52:49 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> Message-ID: <27c3ac91-082c-da07-d694-1748cc08b84a@oracle.com> On 2/14/17 8:18 PM, Kim Barrett wrote: > Not a review yet (though I will try), but saw this plus your question to me earlier today... > >> On Feb 14, 2017, at 5:21 PM, coleen.phillimore at oracle.com wrote: >> And then there's the one in universe.cpp that's a naked oop, which can get moved before the call. >> >> // get the error object at the slot and set set it to NULL so that the >> // array isn't keeping it alive anymore. >> oop exc = preallocated_out_of_memory_errors()->obj_at(next); >> assert(exc != NULL, "slot has been used already"); >> preallocated_out_of_memory_errors()->obj_at_put(next, NULL); <= This on G1 can take out a lock for the oop_store and GC and move the default_err > I don?t think there?s a problem here. Yes, it takes locks, but always without safepoint checks. Thank you for the clarification. I knew there was a lock taken. Coleen > > From cthalinger at twitter.com Wed Feb 15 22:04:32 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Wed, 15 Feb 2017 12:04:32 -1000 Subject: RFR: JDK-8172670: AOT Platform Support for Windows and Mac OS X x64 (Linux and Solaris too) In-Reply-To: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> References: <12F3770E-F647-4905-B3E9-8CE5EFA48D87@oracle.com> Message-ID: <61DE8944-4A5F-4D0C-A7B0-F26DC86EB2CA@twitter.com> This is amazing! Awesome work. I?m glad this got done so soon. > On Feb 8, 2017, at 9:29 AM, Bob Vandette wrote: > > SUMMARY: > > Please review the following set of changes that adds Ahead-of-time compilation support for Mac OSX and > Windows x64 in JDK 10. Linux and Solaris x64 AOT support has also been updated to be consistent with > the new 100% Java based binary container support included in this changeset. > > This change also removes the build and runtime dependency on the external libelf library and header files. > > Once this change is integrated Ahead of time support will be available for the following set of Platforms: > > - Linux x64 > - Windows x64 > - MacOSX x64 > - Solaris x64 > > RFE: > > https://bugs.openjdk.java.net/browse/JDK-8172670 > > WEBREVS: > > TOP LEVEL > ????? > > http://cr.openjdk.java.net/~bobv/8172670/webrev.01/ > > JDK > ?? > > http://cr.openjdk.java.net/~bobv/8172670/jdk/webrev.01/ > > HOTSPOT > ????? > > Full Hotspot Webrev: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev.01/ > > > If you?d prefer to review things in smaller chunks, here are the hotspot changes for Linux and > Solaris including the libelf removal: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-linux.01/ > > Files added to support Mac OSX: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-mac.01/ > > Files added to provide Windows support: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-win.01/ > > Changes to the jtreg tests for AOT: > http://cr.openjdk.java.net/~bobv/8172670/hotspot/webrev-test.01/ > > > TESTING: > > 1. JPRT has been run and passes with these changes > 2. jtreg AOT tests pass on all supported platforms > 3. AOT compiling of java.base completes and can run basic Java programs on all supported platforms > > NOTE: > > 1. Although these test run correctly on Windows, jtreg AOT tests have been temporarily disabled for this platform. > This is due to an internal JPRT system configuration issue which will hopefully get resolved soon. AOT requires > access to a local toolchain and our JPRT systems do not regularly install Visual Studio. > > > Thanks, > Bob. From max.ockner at oracle.com Wed Feb 15 22:18:50 2017 From: max.ockner at oracle.com (Max Ockner) Date: Wed, 15 Feb 2017 17:18:50 -0500 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <58A4D2C8.80603@oracle.com> References: <58A4D2C8.80603@oracle.com> Message-ID: <58A4D3CA.2060700@oracle.com> Hello all, We have filed a bug to remove the interpreter stack caching optimization for jdk10. Ideally we can make this change *early* during the jdk10 development cycle. See below for justification: Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 Stack caching has been around for a long time and is intended to replace some of the load/store (pop/push) operations with corresponding register operations. The need for this optimization arose before caching could adequately lessen the burden of memory access. We have reevaluated the JVM stack caching optimization and have found that it has a high memory footprint and is very costly to maintain, but does not provide significant measurable or theoretical benefit for us when used with modern hardware. Minimal Theoretical Benefit. Because modern hardware does not slap us with the same cost for accessing memory as it once did, the benefit of replacing memory access with register access is far less dramatic now than it once was. Additionally, the interpreter runs for a relatively short time before relevant code sections are compiled. When the VM starts running compiled code instead of interpreted code, performance should begin to move asymptotically towards that of compiled code, diluting any performance penalties from the interpreter to small performance variations. No Measurable Benefit. Please see the results files attached in the bug page. This change was adapted for x86 and sparc, and interpreter performance was measured with Specjvm98 (run with -Xint). No significant decrease in performance was observed. Memory footprint and code complexity. Stack caching in the JVM is implemented by switching the instruction look-up table depending on the tos (top-of-stack) state. At any moment there are is an active table consisting of one dispatch table for each of the 10 tos states. When we enter a safepoint, we copy all 10 safepoint dispatch tables into the active table. The additional entry code makes this copy less efficient and makes any work in the interpreter harder to debug. If we remove this optimization, we will: - decrease memory usage in the interpreter, - eliminated wasteful memory transactions during safepoints, - decrease code complexity (a lot). Please let me know what you think. Thanks, Max From cthalinger at twitter.com Wed Feb 15 23:18:32 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Wed, 15 Feb 2017 13:18:32 -1000 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <58A4D3CA.2060700@oracle.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> Message-ID: <821F9A52-16F2-4AFB-9B6A-27A0AC9EE9F5@twitter.com> Yes, that?s a good idea. And with AOT it should be even less of a problem. > On Feb 15, 2017, at 12:18 PM, Max Ockner wrote: > > Hello all, > > We have filed a bug to remove the interpreter stack caching optimization for jdk10. Ideally we can make this change *early* during the jdk10 development cycle. See below for justification: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 > > Stack caching has been around for a long time and is intended to replace some of the load/store (pop/push) operations with corresponding register operations. The need for this optimization arose before caching could adequately lessen the burden of memory access. We have reevaluated the JVM stack caching optimization and have found that it has a high memory footprint and is very costly to maintain, but does not provide significant measurable or theoretical benefit for us when used with modern hardware. > > Minimal Theoretical Benefit. > Because modern hardware does not slap us with the same cost for accessing memory as it once did, the benefit of replacing memory access with register access is far less dramatic now than it once was. Additionally, the interpreter runs for a relatively short time before relevant code sections are compiled. When the VM starts running compiled code instead of interpreted code, performance should begin to move asymptotically towards that of compiled code, diluting any performance penalties from the interpreter to small performance variations. > > No Measurable Benefit. > Please see the results files attached in the bug page. This change was adapted for x86 and sparc, and interpreter performance was measured with Specjvm98 (run with -Xint). No significant decrease in performance was observed. > > Memory footprint and code complexity. > Stack caching in the JVM is implemented by switching the instruction look-up table depending on the tos (top-of-stack) state. At any moment there are is an active table consisting of one dispatch table for each of the 10 tos states. When we enter a safepoint, we copy all 10 safepoint dispatch tables into the active table. The additional entry code makes this copy less efficient and makes any work in the interpreter harder to debug. > > If we remove this optimization, we will: > - decrease memory usage in the interpreter, > - eliminated wasteful memory transactions during safepoints, > - decrease code complexity (a lot). > > Please let me know what you think. > Thanks, > Max > From coleen.phillimore at oracle.com Thu Feb 16 01:46:16 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Feb 2017 20:46:16 -0500 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> Message-ID: <60eecb82-c56b-67d1-5353-d8de18d227af@oracle.com> On 2/15/17 10:43 AM, coleen.phillimore at oracle.com wrote: > > Hi Serguei, Thank you for reviewing this. > > On 2/15/17 2:18 AM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> >> Thank you for the review discussion with David! >> It helps to understand the webrev details. >> >> It looks good in general. >> Some comments/questions below. >> >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCodeInstaller.cpp.udiff.html >> >> >> for (jint i = 0; i < values->length(); i++) { >> ScopeValue* cur_second = NULL; >> - Handle object = values->obj_at(i); >> - BasicType type = >> JVMCIRuntime::kindToBasicType(slotKinds->obj_at(i), CHECK); >> + Handle object (THREAD, values->obj_at(i)); >> + Handle slot_kind (THREAD, slotKinds->obj_at(i)); >> + BasicType type = JVMCIRuntime::kindToBasicType(slot_kind, CHECK); >> >> >> Do we need a HandleMark in the loop? >> >> > Yes, I don't think it would hurt. Added. > >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCompilerToVM.cpp.udiff.html >> >> >> Handle result; >> if (!klass.is_null()) { >> - result = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >> + oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >> + result = Handle(THREAD, result_oop); >> } else { >> result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); >> } >> return JNIHandles::make_local(THREAD, result()); >> >> >> Not clear, why does the result need to be a handle. >> > > There's no reason this had to be a Handle, but > java_lang_String::create_from_symbol returns a Handle. That's why. > This seems better, agree? > > oop result_oop; > if (!klass.is_null()) { > oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); > } else { > Handle result = java_lang_String::create_from_symbol(symbol, > CHECK_NULL); > result_oop = result(); > } > return JNIHandles::make_local(THREAD, result_oop); > Correction, I redeclared result_oop above. oop result_oop; if (!klass.is_null()) { result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); } else { Handle result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); result_oop = result(); } Coleen >> >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.udiff.html >> >> >> + const char* javapkg = "java/"; . . . - >> strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { >> + strncmp(class_name->as_C_string(), javapkg, strlen(javapkg)) == 0) { >> >> >> Why this change is needed? >> It is a reverse of the recent Rachel's fix. > > I messed this up in the integration. Thank you for finding it! This > was the only diff in that function: > > @@ -2460,7 +2463,7 @@ > TRAPS) { > ResourceMark rm(THREAD); > if (!class_loader.is_null() && > - !SystemDictionary::is_platform_class_loader(class_loader) && > + !SystemDictionary::is_platform_class_loader(class_loader()) && > class_name != NULL && > strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { > TempNewSymbol pkg_name = > InstanceKlass::package_from_name(class_name, CHECK); > >> >> >> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html >> >> >> 701 void InstanceKlass::initialize_impl(instanceKlassHandle this_k, >> TRAPS) { >> 702 HandleMark hm(THREAD); >> . . . >> 712 // refer to the JVM book page 47 for description of steps >> 713 // Step 1 >> 714 { >> 715 HandleMark hm(THREAD); >> >> >> It is not clear what is the reason for one more HandleMark at 715. >> >> > > I'll delete the second one. I added it for debugging number of Handles. > > Thanks for reviewing! > Coleen > >> >> Thanks, >> Serguei >> >> >> On 2/14/17 14:21, coleen.phillimore at oracle.com wrote: >>> >>> New webrev: >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev >>> >>> see coments below first. >>> >>> On 2/13/17 11:51 PM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>>>> >>>>> >>>>> On 2/13/17 8:44 PM, David Holmes wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> "actually S"? :) >>>>> >>>>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>>>> backports. >>>>> >>>>>> >>>>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>>>> Thread::current() call. >>>>>> >>>>>> Well there's a bit more to it than that as you also change parameter >>>>>> types so that we (sometimes) avoid: >>>>>> >>>>>> - oop -> Handle -> oop >>>>>> - Handle -> oop -> Handle >>>>> >>>>> Yes, I put that in the bug but not in the RFR. The recommendation >>>>> is to >>>>> Handle the oop as far up as possible in the call stack and pass >>>>> Handle >>>>> around. >>>> >>>> I hear what you are saying but it is hard to actually see that >>>> reflected in the code. >>>> >>>>>> across method calls. This leads to some puzzling differences eg: >>>>>> >>>>>> src/share/vm/aot/aotCodeHeap.cpp >>>>>> src/share/vm/classfile/javaClasses.hpp >>>>>> >>>>>> The change in aotCodeHeap.cpp reverses the use of Handle and oop, >>>>>> but >>>>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>>>> have different signatures in that regard. They used to both take a >>>>>> Handle but now one takes an oop instead. ?? >>>>> >>>>> Yes, you found the exceptions in Throwable. A lot of the >>>>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop >>>>> instead >>>>> of Handle and I changed print to oop to make it consistent, and >>>>> because >>>>> they it being called from other functions in Throwable with oop. >>>>> Throwable::print_stack_trace actually goes to a safepoint so has >>>>> to be >>>>> Handle. >>>> >>>> I think in javaClasses.hpp the rule should be oop unless Handle is >>>> definitely needed. >>> >>> It's easiest to keep it an oop because most of the functions just do >>> an obj_field, etc. >>> >>> oop java_lang_Throwable::message(oop throwable) { >>> return throwable->obj_field(detailMessage_offset); >>> } >>> >>> but it's likely that the oop throwable is a handle up the call >>> stack. Interestingly looking at it's callers is one in >>> javaClasses.cpp that's an oop, one in exceptions.cpp that's a Handle, >>> >>> oop msg = java_lang_Throwable::message(exception()); >>> >>> And then there's the one in universe.cpp that's a naked oop, which >>> can get moved before the call. >>> >>> // get the error object at the slot and set set it to NULL so >>> that the >>> // array isn't keeping it alive anymore. >>> oop exc = preallocated_out_of_memory_errors()->obj_at(next); >>> assert(exc != NULL, "slot has been used already"); >>> preallocated_out_of_memory_errors()->obj_at_put(next, NULL); >>> <= This on G1 can take out a lock for the oop_store and GC and move >>> the default_err >>> >>> // use the message from the default error >>> oop msg = java_lang_Throwable::message(default_err); <= >>> default_err is an oop. >>> assert(msg != NULL, "no message"); >>> java_lang_Throwable::set_message(exc, msg); >>> >>> exc is a naked oop too. I don't think the naked oop detector >>> triggers for obj_at_put/oop_store because G1 wasn't in the code when >>> it was added. >>> >>> Creating a Handle and passing it saves having to visually inspect >>> functions to try to find naked oops. >>> >>> Fixing this to have a local handle though because the callers don't >>> have THREAD. >>>> >>>>> >>>>>> >>>>>> More generally it is unclear why you made the changes you did in >>>>>> places - see below for some specifics. I'm left questioning how to >>>>>> know when to take an oop and when to take a Handle. Of course that >>>>>> question already exists, but these changes didn't make it any >>>>>> clearer >>>>>> for me. >>>>> >>>>> Pass a Handle early and often is still the rule. For all but GC >>>>> code. >>>>> There are still some functions that are very small and don't >>>>> safepoint >>>>> and take oop. Some of these have migrated to taking Handle, but some >>>>> haven't. I didn't want to make this change larger for now. >>>>> >>>>>> >>>>>> I understand why the oop to Handle implicit conversion could be >>>>>> problematic due to the Thread::current() calls, but I don't >>>>>> understand >>>>>> why you couldn't keep (? add?) the Handle to oop implicit >>>>>> conversions ?? >>>>> >>>>> There weren't Handle -> oop implicit conversions. The conversion >>>>> back >>>>> to oop uses the operator() which remains. >>>> >>>> Okay then it seems an implicit Handle -> oop would avoid the need >>>> to change the caller from foo to foo() if the underlying API >>>> switched from oop to Handle. >>> >>> Yes but it's better to just pass the oop along until something >>> really needs to do something with it. >>>> >>>>> The oop -> Handle conversion has cost, both at runtime and also to >>>>> the >>>>> GC because it increases the number of Handles on the >>>>> Thread->_handle_area. >>>> >>>> Sure. >>>> >>>>> I did this same experiment years ago when Thread::current() was at >>>>> the >>>>> top of the profiling call stack but back then I needed to add too >>>>> many >>>>> explicit Thread::current() calls. There aren't many in this >>>>> change, so >>>>> it's worth doing. Also, metadata handles have the same problem but >>>>> there are still too many of them to remove the implicit conversion. >>>>> And {instance}KlassHandle needs to be removed first. They do nothing >>>>> now but hide the type. >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/c1/c1_Runtime1.cpp >>>>>> >>>>>> Aside: I'm a little surprised that there was not a Handle assignment >>>>>> operator or copy constructor such that given: >>>>>> >>>>>> 863 Handle appendix(THREAD, NULL); >>>>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>>>> >>>>>> that it simply updated the NULL to the desired value, while keeping >>>>>> the THREAD intact. >>>>> >>>>> Handle doesn't save the THREAD so a NULL handle is just oop* NULL. >>>>> The >>>>> RHS of the expression has to be converted to Handle and the default >>>>> assignment operator copies it to appendix. >>>> >>>> I had assumed Handle saved the Thread. >>> >>> No, then it would be two fields. It's only one so passing it is >>> cheap now. >>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/ci/ciInstance.cpp >>>>>> >>>>>> Not at all obvious that replacing Handle with raw oop is correct. >>>>>> I'm >>>>>> not saying it isn't, just that it isn't obvious - and I'll wonder >>>>>> why >>>>>> the Handle was used in the first place. >>>>> >>>>> It is safe because it's immediately unhandled in all the case >>>>> statements, and I wanted to avoid a Thread::current call. >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/classfile/javaClasses.cpp >>>>>> >>>>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>>>> instead of an oop if you are simply going to unwrap the Handle to >>>>>> get >>>>>> the oop back. ??? Particular when a caller like >>>>>> src/share/vm/prims/methodHandles.cpp has to create the Handle >>>>>> from oop >>>>>> in the first place. >>>>> >>>>> I changed it to be the same as java_lang_String::as_symbol(Handle >>>>> java_string ...) which took a handle. >>>>> java_lang_String::as_symbol() was >>>>> the same - it immediately unhandles the argument to pass to the >>>>> rest of >>>>> the functions. I changed java_lang_String::as_symbol() to take an >>>>> oop >>>>> to match, and there are few scattered changes from that. >>>> >>>> I have to disagree with this one. AFAICS there is no reason this: >>>> >>>> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >>>> >>>> needs to take a Handle in the first place. The java_string is >>>> unwrapped to get the oop and the oop is only passed to a few other >>>> String methods and then not touched. Looking at the API's in >>>> javaClass.hpp the norm should be to take oop (as the bulk of >>>> methods do) with Handle only used when actually required. >>> >>> I changed this one to take oop because it was obvious on inspection >>> but note that the "required" is sometimes really hard to see! >>> Chasing naked oop problems isn't something we do very much anymore >>> because we converted many arguments to Handles. Note that the >>> runtime only passes oops much of the time and doesn't usually do >>> anything with them. The exception is javaClasses.cpp though. >>> >>>> >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>>>> >>>>>> It is not at all obvious that JavaArgumentUnboxers are >>>>>> thread-confined! I'm also concerned by API's (unfortunately a number >>>>>> pre-existing) that take a Thread/JavaThread argument but really >>>>>> require it to be the current thread. To that end I'd rather see >>>>>> _thread as _cur_thread and an assertion when it is set. >>>>> >>>>> Since it's a ResourceObj and not StackObj (setting aside debate about >>>>> that), I agree with you. I changed it to use Thread::current() >>>>> where it >>>>> handles the object, which it explicitly did before. In this case, >>>>> it is >>>>> a stack allocated thing but it's not guaranteed to be that. >>>> >>>> Ok. >>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/oops/cpCache.cpp >>>>>> >>>>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>>>> >>>>>> Please remove space before ( >>>>> >>>>> fixed. >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/prims/jvm.cpp >>>>>> >>>>>> Nit: need to fix indent on StackWalk::walk call second line. >>>>> >>>>> fixed. >>>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>>>> >>>>>> Ditto re _thread -> _cur_thread >>>>>> >>>>> >>>>> This one is a StackObj (a Closure which should be used that way). >>>>> So I >>>>> changed _thread to _cur_thread and added an assert that it == >>>>> Thread::current() in the constructor. >>>> >>>> Thanks. >>>> >>>>>> --- >>>>>> >>>>>> src/share/vm/prims/jvmtiImpl.cpp >>>>>> >>>>>> Nit: space before ( again (on handle constructor) >>>>> >>>>> fixed. >>>>>> >>>>>>> It's a very small change to a number of files. Notably the new >>>>>>> JVMCI >>>>>>> files have a lot of implicit conversions from oop to handle. I >>>>>>> want to >>>>>>> get this change in early for jdk10 to stop this usage. I also >>>>>>> added a >>>>>>> few HandleMarks where I found that the HandlesArea was growing >>>>>>> large. >>>>>> >>>>>> The addition and removal of the HandleMarks seems problematic >>>>>> because >>>>>> there are no obvious rules being applied. It is hard to know >>>>>> where and >>>>>> when a HandleMark is needed or not. Looking at >>>>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>>>> create/destroy the HandleMark on each loop iteration? That would >>>>>> only >>>>>> seem necessary if we know the number of iterations is so high >>>>>> that we >>>>>> might consume all our handle space. >>>>> >>>>> Nobody knows when and where to put HandleMarks. It's a dark art. I >>>> >>>> So a trade-off between too much cleanup overhead if we have too >>>> many, and too much GC overhead if we have too few. And no real way >>>> to know either way - Ouch! :( >>>> >>>>> also was testing with a JVM that restored the assert if HandleArea >>>>> got >>>>> over 200 handles and the numbers got very high in these >>>>> functions. The >>>>> GC has to walk these HandleAreas and I am trying to reduce work >>>>> for them. >>>>> >>>>> I tried to write about this in the bug. I'd like to see if changing >>>>> Handle to a proper constructor/destructor like methodHandles, so that >>>>> only the Handles that are active need to be collected. Then we don't >>>>> need HandleMark and their associated mystery. There are several >>>>> HandleMarks in code that obviously doesn't allocate any Handles. >>>>> I took >>>>> out several but restored them so that this patch was smaller. To >>>>> change Handle to a scoped object would require changing passing >>>>> Handle >>>>> as a const reference like I did with methodHandle which is too big >>>>> of a >>>>> change for early-jdk10. >>>> >>>> Ok. This sounds like a way forward though. In combination with an >>>> easy way to detect when handles are required, this would allow very >>>> precise management of handle lifetimes. >>> >>> Good, I'm glad you think so too. >>>> >>>>> The reason I worked on this RFE was because I was curious to see how >>>>> many oops/Handles we're passing around and using from within the JVM >>>>> these days. It's more than I thought. >>>>> >>>>> Thank you for reviewing this. I'm going to rerun some tests with the >>>>> changes above and send another webrev. >>>> >>>> Ok. Thanks. >>> >>> Thanks for the discussion and review! >>> >>> Coleen >>>> >>>> David >>>> ----- >>>> >>>>> Coleen >>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ------ >>>>>> >>>>>>> See bug for more on the motivation of this change. >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>>>> >>>>>>> Tested by running all hotspot jtreg tests with >>>>>>> -XX:+CheckUnhandledOops. >>>>>>> There weren't any unhandled oops amazingly. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>> >>> >> > From david.holmes at oracle.com Thu Feb 16 02:03:15 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Feb 2017 12:03:15 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> Message-ID: <5de68a97-8fb3-7a01-6909-1c13ccd0fc24@oracle.com> On 16/02/2017 1:07 AM, Erik Helin wrote: > On 02/15/2017 02:48 AM, David Holmes wrote: >> Hi Erik, > > Hi David, > > thanks for having a look! Please see new patches at: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ Updates look fine to me. >> oops_do can miss an oop that is in the process of being added - is that >> a problem? > > Nope, that is not an issue. Even if `add_handle` and `oops_do` where > synchronized with each other (for example by using lock), you can still > have the scenario where a GC thread first runs (and finishes) `oops_do` > and then `add_handle` is called (a linearized execution). > > This scenario is something the concurrent marking algorithms already > should be prepared to handle. I'm unclear about the exact hand-over process for the oop. Before the add the oop will be found through some existing path - presumably the Handle - and after the add it will be found through oops_do. But at some point that "existing path" will no longer have the oop, and it will be assumed to be found via the oops_do. The success of that seems to depend on what order the concurrent marking checks things. Or it may be that once concurrent marking starts the Handle can't lose the oop without concurrent marking being informed? The scenario I'm concerned about is this: - add() is executing concurrently with oops_do - oops_do reads _head just before new Chunk is added - oops_do thread is preempted - add() completes and call chain unwinds and at some point the Handle is released - oops_do continues but has missed the new oop - Handle traversal continues but the oop is no longer there Thanks, David ----- >> 140 if (c->data[i] != NULL) { >> 141 f->do_oop(&c->data[i]); >> 142 } >> >> Given a slot can be nulled out concurrently at any time, is it worth >> does this NULL check? The OopClosure will have to do its own NULL check >> anyway. > > Sure, this was mostly to be a bit consistent with JNIHandleBlock (it > does the same NULL check). Think of it as an optimization, there is no > reason to call `f->do_oop` if `c->data[i] == NULL`. Do you think it > reads better if the NULL check is removed? I don't have a strong opinion > on this one. > >> 144 c = (Chunk*) OrderAccess::load_ptr_acquire((volatile >> intptr_t*)&c->next); >> >> This doesn't need to be a load-acquire. You already loaded 'c' via >> load-acquire of _head (or chained therefrom) and its next field is set >> prior to the setting of the _head that you read. > > Agree. I just discussed this with Erik ? as well, and we all agree on > this. I also removed the `volatile` specifier for `next`. > >> 624 jobject ClassLoaderData::add_handle(Handle h) { >> 625 MutexLockerEx ml(metaspace_lock(), >> Mutex::_no_safepoint_check_flag); >> 626 return (jobject) _handles.add(h()); >> 627 } >> 628 >> 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { >> 630 assert(_handles.contains((oop*) h), "Got unexpected handle " >> PTR_FORMAT, p2i((oop*) h)); >> 631 *((oop*) h) = NULL; >> 632 } >> >> I'm a bit unclear on the typing here. Previously we use a JNI Handle >> which was a jobject. But now we've simply stored an oop into a slot in >> an array. We pass the address of that slot back as a jobject, but is it >> really? > > So, this is a bit confusing, and I discussed this with Coleen a few days > ago. jobject was used originally used because that is what > `JNIHandleBlock::allocate_handle` returns. Using something other than > jobject means updating `ModuleEntry` and in particular users > `ModuleEntry::_pd`. This is not something we are keen on doing right now > for JDK 9, and Coleen is planning to clean up how the runtime handles > oops for 10 (or at least improve the situation). > > I don't know what jobject is meant to represent, in the code it is just > an empty class (an opaque type). > > ClassLoaderData::add_handle could just as well return an oop*, but AFAIU > we try to avoid oops and oop* in the runtime code (because that might > mean that some code forgot to tell the GC about the oop). Maybe I'm > wrong here? > >> I would also expect contains to take an oop not an oop* - it >> seems to be asking "is this an address of a slot in our >> ChunkedHandleList" rather than asking "is this oop in our >> ChunkedHandleList". ?? > > I actually wanted the assert to check whether the passed jobject (which > really is an oop*) actually originated from a call to > ClassLoaderData::add_handle. I don't want any code to pass a jobject to > ClassLoaderData::remove_handle_unsafe that doesn't come from a call to > ClassLoaderData::add_handle. > >> A few minor comments: >> >> Copyrights need updating. > > Aaah, it is that time of the year again. Fixed. > >> src/share/vm/classfile/classLoaderData.hpp >> >> 177 // Only on thread ... >> >> Typo: on -> one > > Thanks, fixed. > >> src/share/vm/classfile/moduleEntry.cpp >> >> 90 // Create a JNI handle for the shared ProtectionDomain and save it >> atomically. >> 91 // If someone beats us setting the _pd cache, the created JNI >> handle is destroyed. >> >> These are no longer JNI Handles. > > Thanks, I updated the comment. > > Thanks, > Erik > >> Thanks, >> David >> ----- >> >>> Testing: >>> - Tier 1 (aka JPRT), 2, 3, 4, 5. >>> >>> I would appreciate quite a few reviews on this patch, it contains a nice >>> mixture of: >>> - me venturing into the runtime code :) >>> - lock free code >>> - unreproducible bug (even though I'm sure of the root cause) >>> >>> Thanks for everyone participating in the discussions around this bug and >>> potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and >>> Jiangli. >>> >>> Thanks! >>> Erik >>> >>> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From kim.barrett at oracle.com Thu Feb 16 02:41:45 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 15 Feb 2017 21:41:45 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> Message-ID: <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> > On Feb 15, 2017, at 10:07 AM, Erik Helin wrote: > > On 02/15/2017 02:48 AM, David Holmes wrote: >> Hi Erik, > > Hi David, > > thanks for having a look! Please see new patches at: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ ------------------------------------------------------------------------------ /src/share/vm/classfile/classLoaderData.cpp 138 const juint size = OrderAccess::load_acquire(&c->size); I think all but the first chunk have a constant size of the maximum size, so no need for acquire except for the first. ------------------------------------------------------------------------------ /src/share/vm/classfile/classLoaderData.cpp 173 VerifyContainsOopClosure cl(op); 174 oops_do(&cl); We don't care that this iterates over the entire list even if found early? ------------------------------------------------------------------------------ /src/share/vm/classfile/classLoaderData.hpp 162 struct ChunkedHandleList : public CHeapObj { As used, this doesn't seem to need to be CHeapObj. ------------------------------------------------------------------------------ /src/share/vm/classfile/classLoaderData.hpp 163 struct Chunk : public CHeapObj { 164 static const size_t CAPACITY = 64; 165 oop data[CAPACITY]; Large fixed capacity seems problematic? I think there are lots of "small" CLDs. Don't anonymous classes have there own class loader? A variable sized chunk with a capacity field doesn't seem like it should introduce any new problems. The whole Chunk definition could also be moved to the .cpp file, with only a forward declaration here. ------------------------------------------------------------------------------ /src/share/vm/classfile/classLoaderData.cpp 626 return (jobject) _handles.add(h()); and 631 *((oop*) h) = NULL; Having spent a bunch of time recently dealing with and cleaning up a bunch of places that break the jobject abstraction (though by no means all), I'm not happy to see new ones. As Coleen said, there's discussion about how to refer to indirect pointers to oops in runtime code. jobject has been used as an answer for that in some places, but they might need to be changed. Hiding the conversions as casts will just make that harder. I'm thinking some named conversion functions are in order, instead of bare casts. From serguei.spitsyn at oracle.com Thu Feb 16 03:07:22 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 15 Feb 2017 19:07:22 -0800 Subject: (10) RFR (actually S) 8169881: Remove implicit Handle conversions oop->Handle In-Reply-To: <60eecb82-c56b-67d1-5353-d8de18d227af@oracle.com> References: <9dfc45f6-5513-e033-3c78-43c9c858ae89@oracle.com> <59a88341-33db-9566-8110-2537e8e7ae17@oracle.com> <68fdbee3-79b5-8bc4-4b4f-d8d36cd23ef9@oracle.com> <85e95739-9afc-7dfd-4da1-75707c1794c9@oracle.com> <60eecb82-c56b-67d1-5353-d8de18d227af@oracle.com> Message-ID: <6e8a8d8a-6bfa-3377-afa9-2e2efc21ec11@oracle.com> On 2/15/17 17:46, coleen.phillimore at oracle.com wrote: > > > On 2/15/17 10:43 AM, coleen.phillimore at oracle.com wrote: >> >> Hi Serguei, Thank you for reviewing this. >> >> On 2/15/17 2:18 AM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> >>> Thank you for the review discussion with David! >>> It helps to understand the webrev details. >>> >>> It looks good in general. >>> Some comments/questions below. >>> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCodeInstaller.cpp.udiff.html >>> >>> >>> for (jint i = 0; i < values->length(); i++) { >>> ScopeValue* cur_second = NULL; >>> - Handle object = values->obj_at(i); >>> - BasicType type = >>> JVMCIRuntime::kindToBasicType(slotKinds->obj_at(i), CHECK); >>> + Handle object (THREAD, values->obj_at(i)); >>> + Handle slot_kind (THREAD, slotKinds->obj_at(i)); >>> + BasicType type = JVMCIRuntime::kindToBasicType(slot_kind, CHECK); >>> >>> >>> Do we need a HandleMark in the loop? >>> >>> >> Yes, I don't think it would hurt. Added. >> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/jvmci/jvmciCompilerToVM.cpp.udiff.html >>> >>> >>> Handle result; >>> if (!klass.is_null()) { >>> - result = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >>> + oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >>> + result = Handle(THREAD, result_oop); >>> } else { >>> result = java_lang_String::create_from_symbol(symbol, CHECK_NULL); >>> } >>> return JNIHandles::make_local(THREAD, result()); >>> >>> >>> Not clear, why does the result need to be a handle. >>> >> >> There's no reason this had to be a Handle, but >> java_lang_String::create_from_symbol returns a Handle. That's >> why. This seems better, agree? >> >> oop result_oop; >> if (!klass.is_null()) { >> oop result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); >> } else { >> Handle result = java_lang_String::create_from_symbol(symbol, >> CHECK_NULL); >> result_oop = result(); >> } >> return JNIHandles::make_local(THREAD, result_oop); >> > > Correction, I redeclared result_oop above. > > oop result_oop; > if (!klass.is_null()) { > result_oop = CompilerToVM::get_jvmci_type(klass, CHECK_NULL); > } else { > Handle result = java_lang_String::create_from_symbol(symbol, > CHECK_NULL); > result_oop = result(); > } Yes, of course. Sorry, I did not pay my attention to this detail. I hope, you have caught it quickly. Thanks, Serguei > > Coleen >>> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.udiff.html >>> >>> >>> + const char* javapkg = "java/"; . . . - >>> strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { >>> + strncmp(class_name->as_C_string(), javapkg, strlen(javapkg)) == 0) { >>> >>> >>> Why this change is needed? >>> It is a reverse of the recent Rachel's fix. >> >> I messed this up in the integration. Thank you for finding it! This >> was the only diff in that function: >> >> @@ -2460,7 +2463,7 @@ >> TRAPS) { >> ResourceMark rm(THREAD); >> if (!class_loader.is_null() && >> - !SystemDictionary::is_platform_class_loader(class_loader) && >> + !SystemDictionary::is_platform_class_loader(class_loader()) && >> class_name != NULL && >> strncmp(class_name->as_C_string(), JAVAPKG, JAVAPKG_LEN) == 0) { >> TempNewSymbol pkg_name = >> InstanceKlass::package_from_name(class_name, CHECK); >> >>> >>> >>> http://cr.openjdk.java.net/~coleenp/8169881.02/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html >>> >>> >>> 701 void InstanceKlass::initialize_impl(instanceKlassHandle this_k, >>> TRAPS) { >>> 702 HandleMark hm(THREAD); >>> . . . >>> 712 // refer to the JVM book page 47 for description of steps >>> 713 // Step 1 >>> 714 { >>> 715 HandleMark hm(THREAD); >>> >>> >>> It is not clear what is the reason for one more HandleMark at 715. >>> >>> >> >> I'll delete the second one. I added it for debugging number of Handles. >> >> Thanks for reviewing! >> Coleen >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 2/14/17 14:21, coleen.phillimore at oracle.com wrote: >>>> >>>> New webrev: >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8169881.02/webrev >>>> >>>> see coments below first. >>>> >>>> On 2/13/17 11:51 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> On 14/02/2017 1:53 PM, Coleen Phillimore wrote: >>>>>> >>>>>> >>>>>> On 2/13/17 8:44 PM, David Holmes wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> "actually S"? :) >>>>>> >>>>>> Well, maybe not but it shouldn't have a lot of merge conflicts for >>>>>> backports. >>>>>> >>>>>>> >>>>>>> On 14/02/2017 6:10 AM, Coleen Phillimore wrote: >>>>>>>> Summary: Pass THREAD to Handle as argument instead of implicit >>>>>>>> Thread::current() call. >>>>>>> >>>>>>> Well there's a bit more to it than that as you also change >>>>>>> parameter >>>>>>> types so that we (sometimes) avoid: >>>>>>> >>>>>>> - oop -> Handle -> oop >>>>>>> - Handle -> oop -> Handle >>>>>> >>>>>> Yes, I put that in the bug but not in the RFR. The >>>>>> recommendation is to >>>>>> Handle the oop as far up as possible in the call stack and pass >>>>>> Handle >>>>>> around. >>>>> >>>>> I hear what you are saying but it is hard to actually see that >>>>> reflected in the code. >>>>> >>>>>>> across method calls. This leads to some puzzling differences eg: >>>>>>> >>>>>>> src/share/vm/aot/aotCodeHeap.cpp >>>>>>> src/share/vm/classfile/javaClasses.hpp >>>>>>> >>>>>>> The change in aotCodeHeap.cpp reverses the use of Handle and >>>>>>> oop, but >>>>>>> it is unclear why Throwable::print and Throwable::print_stack_trace >>>>>>> have different signatures in that regard. They used to both take a >>>>>>> Handle but now one takes an oop instead. ?? >>>>>> >>>>>> Yes, you found the exceptions in Throwable. A lot of the >>>>>> java_lang_Throwable (and other javaClasses.hpp classes) pass oop >>>>>> instead >>>>>> of Handle and I changed print to oop to make it consistent, and >>>>>> because >>>>>> they it being called from other functions in Throwable with oop. >>>>>> Throwable::print_stack_trace actually goes to a safepoint so has >>>>>> to be >>>>>> Handle. >>>>> >>>>> I think in javaClasses.hpp the rule should be oop unless Handle is >>>>> definitely needed. >>>> >>>> It's easiest to keep it an oop because most of the functions just >>>> do an obj_field, etc. >>>> >>>> oop java_lang_Throwable::message(oop throwable) { >>>> return throwable->obj_field(detailMessage_offset); >>>> } >>>> >>>> but it's likely that the oop throwable is a handle up the call >>>> stack. Interestingly looking at it's callers is one in >>>> javaClasses.cpp that's an oop, one in exceptions.cpp that's a Handle, >>>> >>>> oop msg = java_lang_Throwable::message(exception()); >>>> >>>> And then there's the one in universe.cpp that's a naked oop, which >>>> can get moved before the call. >>>> >>>> // get the error object at the slot and set set it to NULL so >>>> that the >>>> // array isn't keeping it alive anymore. >>>> oop exc = preallocated_out_of_memory_errors()->obj_at(next); >>>> assert(exc != NULL, "slot has been used already"); >>>> preallocated_out_of_memory_errors()->obj_at_put(next, >>>> NULL); <= This on G1 can take out a lock for the oop_store and >>>> GC and move the default_err >>>> >>>> // use the message from the default error >>>> oop msg = java_lang_Throwable::message(default_err); <= >>>> default_err is an oop. >>>> assert(msg != NULL, "no message"); >>>> java_lang_Throwable::set_message(exc, msg); >>>> >>>> exc is a naked oop too. I don't think the naked oop detector >>>> triggers for obj_at_put/oop_store because G1 wasn't in the code >>>> when it was added. >>>> >>>> Creating a Handle and passing it saves having to visually inspect >>>> functions to try to find naked oops. >>>> >>>> Fixing this to have a local handle though because the callers don't >>>> have THREAD. >>>>> >>>>>> >>>>>>> >>>>>>> More generally it is unclear why you made the changes you did in >>>>>>> places - see below for some specifics. I'm left questioning how to >>>>>>> know when to take an oop and when to take a Handle. Of course that >>>>>>> question already exists, but these changes didn't make it any >>>>>>> clearer >>>>>>> for me. >>>>>> >>>>>> Pass a Handle early and often is still the rule. For all but GC >>>>>> code. >>>>>> There are still some functions that are very small and don't >>>>>> safepoint >>>>>> and take oop. Some of these have migrated to taking Handle, but >>>>>> some >>>>>> haven't. I didn't want to make this change larger for now. >>>>>> >>>>>>> >>>>>>> I understand why the oop to Handle implicit conversion could be >>>>>>> problematic due to the Thread::current() calls, but I don't >>>>>>> understand >>>>>>> why you couldn't keep (? add?) the Handle to oop implicit >>>>>>> conversions ?? >>>>>> >>>>>> There weren't Handle -> oop implicit conversions. The conversion >>>>>> back >>>>>> to oop uses the operator() which remains. >>>>> >>>>> Okay then it seems an implicit Handle -> oop would avoid the need >>>>> to change the caller from foo to foo() if the underlying API >>>>> switched from oop to Handle. >>>> >>>> Yes but it's better to just pass the oop along until something >>>> really needs to do something with it. >>>>> >>>>>> The oop -> Handle conversion has cost, both at runtime and also >>>>>> to the >>>>>> GC because it increases the number of Handles on the >>>>>> Thread->_handle_area. >>>>> >>>>> Sure. >>>>> >>>>>> I did this same experiment years ago when Thread::current() was >>>>>> at the >>>>>> top of the profiling call stack but back then I needed to add too >>>>>> many >>>>>> explicit Thread::current() calls. There aren't many in this >>>>>> change, so >>>>>> it's worth doing. Also, metadata handles have the same problem but >>>>>> there are still too many of them to remove the implicit conversion. >>>>>> And {instance}KlassHandle needs to be removed first. They do nothing >>>>>> now but hide the type. >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/c1/c1_Runtime1.cpp >>>>>>> >>>>>>> Aside: I'm a little surprised that there was not a Handle >>>>>>> assignment >>>>>>> operator or copy constructor such that given: >>>>>>> >>>>>>> 863 Handle appendix(THREAD, NULL); >>>>>>> 970 appendix = cpce->appendix_if_resolved(pool); >>>>>>> >>>>>>> that it simply updated the NULL to the desired value, while keeping >>>>>>> the THREAD intact. >>>>>> >>>>>> Handle doesn't save the THREAD so a NULL handle is just oop* >>>>>> NULL. The >>>>>> RHS of the expression has to be converted to Handle and the default >>>>>> assignment operator copies it to appendix. >>>>> >>>>> I had assumed Handle saved the Thread. >>>> >>>> No, then it would be two fields. It's only one so passing it is >>>> cheap now. >>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/ci/ciInstance.cpp >>>>>>> >>>>>>> Not at all obvious that replacing Handle with raw oop is >>>>>>> correct. I'm >>>>>>> not saying it isn't, just that it isn't obvious - and I'll >>>>>>> wonder why >>>>>>> the Handle was used in the first place. >>>>>> >>>>>> It is safe because it's immediately unhandled in all the case >>>>>> statements, and I wanted to avoid a Thread::current call. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/classfile/javaClasses.cpp >>>>>>> >>>>>>> Why change java_lang_String::as_symbol_or_null to take a Handle >>>>>>> instead of an oop if you are simply going to unwrap the Handle >>>>>>> to get >>>>>>> the oop back. ??? Particular when a caller like >>>>>>> src/share/vm/prims/methodHandles.cpp has to create the Handle >>>>>>> from oop >>>>>>> in the first place. >>>>>> >>>>>> I changed it to be the same as java_lang_String::as_symbol(Handle >>>>>> java_string ...) which took a handle. >>>>>> java_lang_String::as_symbol() was >>>>>> the same - it immediately unhandles the argument to pass to the >>>>>> rest of >>>>>> the functions. I changed java_lang_String::as_symbol() to take >>>>>> an oop >>>>>> to match, and there are few scattered changes from that. >>>>> >>>>> I have to disagree with this one. AFAICS there is no reason this: >>>>> >>>>> 517 Symbol* java_lang_String::as_symbol(Handle java_string, TRAPS) { >>>>> >>>>> needs to take a Handle in the first place. The java_string is >>>>> unwrapped to get the oop and the oop is only passed to a few other >>>>> String methods and then not touched. Looking at the API's in >>>>> javaClass.hpp the norm should be to take oop (as the bulk of >>>>> methods do) with Handle only used when actually required. >>>> >>>> I changed this one to take oop because it was obvious on inspection >>>> but note that the "required" is sometimes really hard to see! >>>> Chasing naked oop problems isn't something we do very much anymore >>>> because we converted many arguments to Handles. Note that the >>>> runtime only passes oops much of the time and doesn't usually do >>>> anything with them. The exception is javaClasses.cpp though. >>>> >>>>> >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/jvmci/jvmciCompilerToVM.hpp >>>>>>> >>>>>>> It is not at all obvious that JavaArgumentUnboxers are >>>>>>> thread-confined! I'm also concerned by API's (unfortunately a >>>>>>> number >>>>>>> pre-existing) that take a Thread/JavaThread argument but really >>>>>>> require it to be the current thread. To that end I'd rather see >>>>>>> _thread as _cur_thread and an assertion when it is set. >>>>>> >>>>>> Since it's a ResourceObj and not StackObj (setting aside debate >>>>>> about >>>>>> that), I agree with you. I changed it to use Thread::current() >>>>>> where it >>>>>> handles the object, which it explicitly did before. In this >>>>>> case, it is >>>>>> a stack allocated thing but it's not guaranteed to be that. >>>>> >>>>> Ok. >>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/oops/cpCache.cpp >>>>>>> >>>>>>> Nit: objArrayHandle resolved_references (Thread::current(), ... >>>>>>> >>>>>>> Please remove space before ( >>>>>> >>>>>> fixed. >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/prims/jvm.cpp >>>>>>> >>>>>>> Nit: need to fix indent on StackWalk::walk call second line. >>>>>> >>>>>> fixed. >>>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/prims/jvmtiGetLoadedClasses.cpp >>>>>>> >>>>>>> Ditto re _thread -> _cur_thread >>>>>>> >>>>>> >>>>>> This one is a StackObj (a Closure which should be used that way). >>>>>> So I >>>>>> changed _thread to _cur_thread and added an assert that it == >>>>>> Thread::current() in the constructor. >>>>> >>>>> Thanks. >>>>> >>>>>>> --- >>>>>>> >>>>>>> src/share/vm/prims/jvmtiImpl.cpp >>>>>>> >>>>>>> Nit: space before ( again (on handle constructor) >>>>>> >>>>>> fixed. >>>>>>> >>>>>>>> It's a very small change to a number of files. Notably the new >>>>>>>> JVMCI >>>>>>>> files have a lot of implicit conversions from oop to handle. I >>>>>>>> want to >>>>>>>> get this change in early for jdk10 to stop this usage. I also >>>>>>>> added a >>>>>>>> few HandleMarks where I found that the HandlesArea was growing >>>>>>>> large. >>>>>>> >>>>>>> The addition and removal of the HandleMarks seems problematic >>>>>>> because >>>>>>> there are no obvious rules being applied. It is hard to know >>>>>>> where and >>>>>>> when a HandleMark is needed or not. Looking at >>>>>>> src/share/vm/jvmci/jvmciCompilerToVM.cpp do we really want to >>>>>>> create/destroy the HandleMark on each loop iteration? That would >>>>>>> only >>>>>>> seem necessary if we know the number of iterations is so high >>>>>>> that we >>>>>>> might consume all our handle space. >>>>>> >>>>>> Nobody knows when and where to put HandleMarks. It's a dark art. I >>>>> >>>>> So a trade-off between too much cleanup overhead if we have too >>>>> many, and too much GC overhead if we have too few. And no real way >>>>> to know either way - Ouch! :( >>>>> >>>>>> also was testing with a JVM that restored the assert if >>>>>> HandleArea got >>>>>> over 200 handles and the numbers got very high in these >>>>>> functions. The >>>>>> GC has to walk these HandleAreas and I am trying to reduce work >>>>>> for them. >>>>>> >>>>>> I tried to write about this in the bug. I'd like to see if >>>>>> changing >>>>>> Handle to a proper constructor/destructor like methodHandles, so >>>>>> that >>>>>> only the Handles that are active need to be collected. Then we don't >>>>>> need HandleMark and their associated mystery. There are several >>>>>> HandleMarks in code that obviously doesn't allocate any Handles. >>>>>> I took >>>>>> out several but restored them so that this patch was smaller. To >>>>>> change Handle to a scoped object would require changing passing >>>>>> Handle >>>>>> as a const reference like I did with methodHandle which is too >>>>>> big of a >>>>>> change for early-jdk10. >>>>> >>>>> Ok. This sounds like a way forward though. In combination with an >>>>> easy way to detect when handles are required, this would allow >>>>> very precise management of handle lifetimes. >>>> >>>> Good, I'm glad you think so too. >>>>> >>>>>> The reason I worked on this RFE was because I was curious to see how >>>>>> many oops/Handles we're passing around and using from within the JVM >>>>>> these days. It's more than I thought. >>>>>> >>>>>> Thank you for reviewing this. I'm going to rerun some tests with >>>>>> the >>>>>> changes above and send another webrev. >>>>> >>>>> Ok. Thanks. >>>> >>>> Thanks for the discussion and review! >>>> >>>> Coleen >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> Coleen >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ------ >>>>>>> >>>>>>>> See bug for more on the motivation of this change. >>>>>>>> >>>>>>>> open webrev at >>>>>>>> http://cr.openjdk.java.net/~coleenp/8169881.01/webrev >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8169881 >>>>>>>> >>>>>>>> Tested by running all hotspot jtreg tests with >>>>>>>> -XX:+CheckUnhandledOops. >>>>>>>> There weren't any unhandled oops amazingly. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>> >>>> >>> >> > From david.holmes at oracle.com Thu Feb 16 03:31:53 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Feb 2017 13:31:53 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> Message-ID: <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> Hi Kim, If I may ... On 16/02/2017 12:41 PM, Kim Barrett wrote: >> On Feb 15, 2017, at 10:07 AM, Erik Helin wrote: >> >> On 02/15/2017 02:48 AM, David Holmes wrote: >>> Hi Erik, >> >> Hi David, >> >> thanks for having a look! Please see new patches at: >> - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ >> - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ > > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.cpp > 138 const juint size = OrderAccess::load_acquire(&c->size); > > I think all but the first chunk have a constant size of the > maximum size, so no need for acquire except for the first. The size is incremented on each add, so this acquire sync's with the release of the new size, so that we are sure to see the oop that was just added. Though given the races between add and oops_do it may not actually matter. David ----- > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.cpp > 173 VerifyContainsOopClosure cl(op); > 174 oops_do(&cl); > > We don't care that this iterates over the entire list even if found > early? > > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.hpp > 162 struct ChunkedHandleList : public CHeapObj { > > As used, this doesn't seem to need to be CHeapObj. > > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.hpp > 163 struct Chunk : public CHeapObj { > 164 static const size_t CAPACITY = 64; > 165 oop data[CAPACITY]; > > Large fixed capacity seems problematic? I think there are lots of > "small" CLDs. Don't anonymous classes have there own class loader? > A variable sized chunk with a capacity field doesn't seem like it > should introduce any new problems. > > The whole Chunk definition could also be moved to the .cpp file, with > only a forward declaration here. > > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.cpp > 626 return (jobject) _handles.add(h()); > and > 631 *((oop*) h) = NULL; > > Having spent a bunch of time recently dealing with and cleaning up a > bunch of places that break the jobject abstraction (though by no means > all), I'm not happy to see new ones. > > As Coleen said, there's discussion about how to refer to indirect > pointers to oops in runtime code. jobject has been used as an answer > for that in some places, but they might need to be changed. Hiding > the conversions as casts will just make that harder. I'm thinking > some named conversion functions are in order, instead of bare casts. > From kim.barrett at oracle.com Thu Feb 16 05:53:54 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 Feb 2017 00:53:54 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> Message-ID: > On Feb 15, 2017, at 10:31 PM, David Holmes wrote: > > Hi Kim, > > If I may ... > > On 16/02/2017 12:41 PM, Kim Barrett wrote: >>> On Feb 15, 2017, at 10:07 AM, Erik Helin wrote: >>> >>> On 02/15/2017 02:48 AM, David Holmes wrote: >>>> Hi Erik, >>> >>> Hi David, >>> >>> thanks for having a look! Please see new patches at: >>> - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ >>> - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ >> >> ------------------------------------------------------------------------------ >> /src/share/vm/classfile/classLoaderData.cpp >> 138 const juint size = OrderAccess::load_acquire(&c->size); >> >> I think all but the first chunk have a constant size of the >> maximum size, so no need for acquire except for the first. > > The size is incremented on each add, so this acquire sync's with the release of the new size, so that we are sure to see the oop that was just added. > > Though given the races between add and oops_do it may not actually matter. By ?first? I mean the first chunk. only the first chunk?s size is subject to modification. The release_store of _head that made the first chunk accessible will be after the setting of the next chunk?s size to its final (max) value. The corresponding load_acquire of _head ensures any _next chunks are visibly complete. It?s the same rationale for _next not requiring any special handling. At least, that?s how it looks to me right now; am I missing something. From david.holmes at oracle.com Thu Feb 16 07:41:01 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Feb 2017 17:41:01 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> Message-ID: <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> On 16/02/2017 3:53 PM, Kim Barrett wrote: >> On Feb 15, 2017, at 10:31 PM, David Holmes wrote: >> >> Hi Kim, >> >> If I may ... >> >> On 16/02/2017 12:41 PM, Kim Barrett wrote: >>>> On Feb 15, 2017, at 10:07 AM, Erik Helin wrote: >>>> >>>> On 02/15/2017 02:48 AM, David Holmes wrote: >>>>> Hi Erik, >>>> >>>> Hi David, >>>> >>>> thanks for having a look! Please see new patches at: >>>> - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ >>>> - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ >>> >>> ------------------------------------------------------------------------------ >>> /src/share/vm/classfile/classLoaderData.cpp >>> 138 const juint size = OrderAccess::load_acquire(&c->size); >>> >>> I think all but the first chunk have a constant size of the >>> maximum size, so no need for acquire except for the first. >> >> The size is incremented on each add, so this acquire sync's with the release of the new size, so that we are sure to see the oop that was just added. >> >> Though given the races between add and oops_do it may not actually matter. > > By ?first? I mean the first chunk. only the first chunk?s size is subject to modification. The release_store of _head that made the first chunk accessible will be after the setting of the next chunk?s size to its final (max) value. The corresponding load_acquire of _head ensures any _next chunks are visibly complete. It?s the same rationale for _next not requiring any special handling. At least, that?s how it looks to me right now; am I missing something. The _head chunk has its _size incremented on each add(). David From kim.barrett at oracle.com Thu Feb 16 07:46:01 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 Feb 2017 02:46:01 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> Message-ID: <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> > On Feb 16, 2017, at 2:41 AM, David Holmes wrote: > > On 16/02/2017 3:53 PM, Kim Barrett wrote: >>>> /src/share/vm/classfile/classLoaderData.cpp >>>> 138 const juint size = OrderAccess::load_acquire(&c->size); >>>> >>>> I think all but the first chunk have a constant size of the >>>> maximum size, so no need for acquire except for the first. >>> >>> The size is incremented on each add, so this acquire sync's with the release of the new size, so that we are sure to see the oop that was just added. >>> >>> Though given the races between add and oops_do it may not actually matter. >> >> By ?first? I mean the first chunk. only the first chunk?s size is subject to modification. The release_store of _head that made the first chunk accessible will be after the setting of the next chunk?s size to its final (max) value. The corresponding load_acquire of _head ensures any _next chunks are visibly complete. It?s the same rationale for _next not requiring any special handling. At least, that?s how it looks to me right now; am I missing something. > > The _head chunk has its _size incremented on each add(). > > David Agreed, and a load_acquire is needed for the _head chunk. I?m suggesting an ordinary load is sufficient for any _next chunks as we loop down the list. From david.holmes at oracle.com Thu Feb 16 07:54:06 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Feb 2017 17:54:06 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> Message-ID: On 16/02/2017 5:46 PM, Kim Barrett wrote: >> On Feb 16, 2017, at 2:41 AM, David Holmes wrote: >> >> On 16/02/2017 3:53 PM, Kim Barrett wrote: >>>>> /src/share/vm/classfile/classLoaderData.cpp >>>>> 138 const juint size = OrderAccess::load_acquire(&c->size); >>>>> >>>>> I think all but the first chunk have a constant size of the >>>>> maximum size, so no need for acquire except for the first. >>>> >>>> The size is incremented on each add, so this acquire sync's with the release of the new size, so that we are sure to see the oop that was just added. >>>> >>>> Though given the races between add and oops_do it may not actually matter. >>> >>> By ?first? I mean the first chunk. only the first chunk?s size is subject to modification. The release_store of _head that made the first chunk accessible will be after the setting of the next chunk?s size to its final (max) value. The corresponding load_acquire of _head ensures any _next chunks are visibly complete. It?s the same rationale for _next not requiring any special handling. At least, that?s how it looks to me right now; am I missing something. >> >> The _head chunk has its _size incremented on each add(). >> >> David > > Agreed, and a load_acquire is needed for the _head chunk. > > I?m suggesting an ordinary load is sufficient for any _next chunks as we loop down the list. Ah! Now I get what you mean - sorry. Yes only the first loop iteration needs the load-acquire. So just: Chunk* c = (Chunk*) OrderAccess::load_ptr_acquire((volatile intptr_t*)&_head); const juint size = (c == NULL ? 0 : OrderAccess::load_acquire(&c->size)); while (c != NULL) { size = OrderAccess::load_acquire(&c->size); ... } Cheers, David > From david.holmes at oracle.com Thu Feb 16 08:09:36 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Feb 2017 18:09:36 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> Message-ID: <2b8c7837-0e9f-6c15-2be3-fa06f95ee47e@oracle.com> Bah! didn't finish my edit ... On 16/02/2017 5:54 PM, David Holmes wrote: > On 16/02/2017 5:46 PM, Kim Barrett wrote: >>> On Feb 16, 2017, at 2:41 AM, David Holmes >>> wrote: >>> >>> On 16/02/2017 3:53 PM, Kim Barrett wrote: >>>>>> /src/share/vm/classfile/classLoaderData.cpp >>>>>> 138 const juint size = OrderAccess::load_acquire(&c->size); >>>>>> >>>>>> I think all but the first chunk have a constant size of the >>>>>> maximum size, so no need for acquire except for the first. >>>>> >>>>> The size is incremented on each add, so this acquire sync's with >>>>> the release of the new size, so that we are sure to see the oop >>>>> that was just added. >>>>> >>>>> Though given the races between add and oops_do it may not actually >>>>> matter. >>>> >>>> By ?first? I mean the first chunk. only the first chunk?s size is >>>> subject to modification. The release_store of _head that made the >>>> first chunk accessible will be after the setting of the next chunk?s >>>> size to its final (max) value. The corresponding load_acquire of >>>> _head ensures any _next chunks are visibly complete. It?s the same >>>> rationale for _next not requiring any special handling. At least, >>>> that?s how it looks to me right now; am I missing something. >>> >>> The _head chunk has its _size incremented on each add(). >>> >>> David >> >> Agreed, and a load_acquire is needed for the _head chunk. >> >> I?m suggesting an ordinary load is sufficient for any _next chunks as >> we loop down the list. > > Ah! Now I get what you mean - sorry. Yes only the first loop iteration > needs the load-acquire. So just: > > Chunk* c = (Chunk*) OrderAccess::load_ptr_acquire((volatile > intptr_t*)&_head); > const juint size = (c == NULL ? 0 : OrderAccess::load_acquire(&c->size)); > while (c != NULL) { > size = OrderAccess::load_acquire(&c->size); size = c->size; David ----- > ... > } > > > Cheers, > David >> From shafi.s.ahmad at oracle.com Thu Feb 16 08:26:12 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 16 Feb 2017 00:26:12 -0800 (PST) Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> Message-ID: <56a78bd9-bac8-4fff-a1bf-1417e709426f@default> Hi Coleen, Thank you for the review. Regards, Shafi > -----Original Message----- > From: Coleen Phillimore > Sent: Thursday, February 16, 2017 2:09 AM > To: hotspot-dev at openjdk.java.net > Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of > JDK_Version::to_string OR proper return statement is missing OR proper > comment is needed > > Shafi, > This looks good to me also. Thank you for fixing this. > Coleen > > On 2/15/17 6:35 AM, Shafi Ahmad wrote: > > Hi All, > > > > Summary: Adding return value check and update index variable. It's a very > small change to single file. > > > > Webrev link: http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ > > bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 > > > > Testing: jprt and jtreg test. > > > > Regards, > > Shafi > From shafi.s.ahmad at oracle.com Thu Feb 16 08:35:36 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 16 Feb 2017 00:35:36 -0800 (PST) Subject: JDK 10 RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: <73ff2d81-0791-658b-3ef3-7f6191e0b8a9@oracle.com> References: <26b29345-f606-580d-22ae-9540d7036051@oracle.com> <9d506110-91a1-4827-99a8-e46e9363eb3d@default> <5bec7006-4301-7f2a-88c9-dd685e447d15@oracle.com> <73ff2d81-0791-658b-3ef3-7f6191e0b8a9@oracle.com> Message-ID: Hi Coleen, I don't have test case. I tried to write one but couldn't able to. I have verified it with reproducer of old fixed bug https://bugs.openjdk.java.net/browse/JDK-8080842. I applied my changes on the revision just before the fix of JDK-8080842. Without the fix: java CreateBadClassFile .foreach() call: Exception in thread "main" java.lang.ClassFormatError: Duplicate field name&signature in class file WidgetCollection$1 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:759) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) . . . With the fix: .foreach() call: Exception in thread "main" java.lang.ClassFormatError: Duplicate field name "hasNext" with signature "Ljava.lang.Object;" in class file WidgetCollection$1 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:760) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) . . . Regards, Shafi > -----Original Message----- > From: Coleen Phillimore > Sent: Thursday, February 16, 2017 2:42 AM > To: hotspot-dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > > Yes this looks very good. Do you have a test for it? Or is it in the jck tests and > verified manually for each case? > > Thanks, > Coleen > > On 2/15/17 1:29 AM, Shafi Ahmad wrote: > > Thank you David for reviewing it. > > > > All, > > > > May I get second reviewer for this trivial change. > > > > Regards, > > Shafi > > > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Wednesday, February 15, 2017 11:54 AM > >> To: Shafi Ahmad ; hotspot- > >> dev at openjdk.java.net > >> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > >> name&signature in class file" should report the name and signature of > >> the field > >> > >> Looks good to me! Nice and neat. > >> > >> Thanks, > >> David > >> > >> On 15/02/2017 4:02 PM, Shafi Ahmad wrote: > >>> Hi David, > >>> > >>> Please find updated webrev at > >>> http://cr.openjdk.java.net/~shshahma/8171194/webrev.01/ > >>> Added overloaded method classfile_parse_error. > >>> > >>> Regards, > >>> Shafi > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Wednesday, February 15, 2017 3:55 AM > >>>> To: Shafi Ahmad ; hotspot- > >>>> dev at openjdk.java.net > >>>> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > >>>> name&signature in class file" should report the name and signature > >>>> of the field > >>>> > >>>> Hi Shafi, > >>>> > >>>> On 14/02/2017 11:20 PM, Shafi Ahmad wrote: > >>>>> Hi David, > >>>>> > >>>>> Thanks for reviewing it. > >>>>> > >>>>> Initially I started with fixed size of local char array but later > >>>>> I changed my > >>>> mind and make it dynamic. > >>>>> Let me know if I have to make it local char array like. > >>>>> > >>>>> + unsigned int siglength = sig->utf8_length(); > >>>>> + unsigned int namelength = name->utf8_length(); > >>>>> + unsigned int length = siglength + namelength + 64; > >>>>> + char *buff = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, > char, > >>>> length); > >>>>> + jio_snprintf(buff, length, > >>>>> + "Duplicate method name \"%s\" with signature > >>>>> + \"%s\" in class > >>>> file", > >>>>> + name->as_C_string(), sig->as_klass_external_name()); > >>>>> + classfile_parse_error("%s %s", buff, CHECK); > >>>>> } > >>>>> > >>>>> to > >>>>> > >>>>> + char buff[fixedsize]; // say fixedsize is 512 > >>>>> + jio_snprintf(buff, 512, > >>>>> + "Duplicate method name \"%s\" with signature > >>>>> + \"%s\" in class > >>>> file", > >>>>> + name->as_C_string(), sig->as_klass_external_name()); > >>>>> + classfile_parse_error("%s %s", buff, CHECK); > >>>>> } > >>>> It could still be dynamic, you just need to use > >>>> NEW_RESOURCE_ARRAY_IN_THREAD_RETURN_NULL and check for a > >> NULL return, > >>>> and fallback to not including the additional info. But the > >>>> underlying Exceptions::fthrow uses a local buffer itself (1024 > >>>> max), so you could just do the same as you propose above. > >>>> > >>>> Though it is very annoying to have to allocate a buffer just to > >>>> pass it through to classfile_parse_error which passes it through to > >>>> Exceptions::fthrow which is a varargs method. So another > >>>> possibility is to just add another overload of > >>>> classfile_parse_error that takes in the two additional strings you want > to print. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> Regards, > >>>>> Shafi > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: David Holmes > >>>>>> Sent: Tuesday, February 14, 2017 6:34 PM > >>>>>> To: Shafi Ahmad ; hotspot- > >>>>>> dev at openjdk.java.net > >>>>>> Subject: Re: JDK 10 RFR JDK-8171194: Exception "Duplicate field > >>>>>> name&signature in class file" should report the name and > >>>>>> signature of the field > >>>>>> > >>>>>> Hi Shafi, > >>>>>> > >>>>>> I'm concerned about the use of > NEW_RESOURCE_ARRAY_IN_THREAD. > >> If > >>>> it > >>>>>> can't allocate it will abort the VM. That seems like a bad thing > >>>>>> to > >> happen. > >>>>>> Thanks, > >>>>>> David > >>>>>> > >>>>>> On 14/02/2017 7:19 PM, Shafi Ahmad wrote: > >>>>>>> Summary: java.lang.ClassFormatError: Exception "Duplicate field > >>>>>> name&signature in class file" should report the name and > >>>>>> signature of the field. > >>>>>>> It's a very small change to single file. > >>>>>>> In the current implementation name and signature of duplicate > >>>>>>> field is > >>>>>> missing in java.lang.ClassFormatError exception message. > >>>>>>> Without a field name + signature it is hard to triggering the > problem. > >>>>>>> > >>>>>>> Webrev link: > >>>> http://cr.openjdk.java.net/~shshahma/8171194/webrev.00/ > >>>>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8171194 > >>>>>>> > >>>>>>> Testing: jprt and jtreg test. > >>>>>>> > >>>>>>> I have verified my changes with the reproduces of > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8080842 on jdk8u60-b01 > >>>>>> code base as I was not able to write reproducer of current issue. > >>>>>>> With the fix I am getting below exception message. > >>>>>>> > >>>>>>> $ java CreateBadClassFile > >>>>>>> .foreach() call: Exception in thread "main" > java.lang.ClassFormatError: > >>>>>> Duplicate field name "hasNext" with signature "Ljava.lang.Object;" > >>>>>> in class file WidgetCollection$1 > >>>>>>> at java.lang.ClassLoader.defineClass1(Native Method) > >>>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > >>>>>>> at > >>>>>> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java: > >>>>>> 14 > >>>>>> 2) > From thomas.schatzl at oracle.com Thu Feb 16 08:55:17 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 16 Feb 2017 09:55:17 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <2b8c7837-0e9f-6c15-2be3-fa06f95ee47e@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> <2b8c7837-0e9f-6c15-2be3-fa06f95ee47e@oracle.com> Message-ID: <1487235317.3407.7.camel@oracle.com> Hi, On Thu, 2017-02-16 at 18:09 +1000, David Holmes wrote: > Bah! didn't finish my edit ... > > On 16/02/2017 5:54 PM, David Holmes wrote: > > > > On 16/02/2017 5:46 PM, Kim Barrett wrote: > > > > > > > > > > > On Feb 16, 2017, at 2:41 AM, David Holmes > > > com> > > > > wrote: > > > > > > > > On 16/02/2017 3:53 PM, Kim Barrett wrote: > > > > > > > > > > > > > > > > > > > > > > > > > /src/share/vm/classfile/classLoaderData.cpp > > > > > > > 138?????const juint size = OrderAccess::load_acquire(&c- > > > > > > > >size); > > > > > > > > > > > > > > I think all but the first chunk have a constant size of > > > > > > > the > > > > > > > maximum size, so no need for acquire except for the > > > > > > > first. > > > > > > The size is incremented on each add, so this acquire sync's > > > > > > with > > > > > > the release of the new size, so that we are sure to see the > > > > > > oop > > > > > > that was just added. > > > > > > > > > > > > Though given the races between add and oops_do it may not > > > > > > actually > > > > > > matter. > > > > > By ?first? I mean the first chunk.??only the first chunk?s > > > > > size is > > > > > subject to modification.??The release_store of _head that > > > > > made the > > > > > first chunk accessible will be after the setting of the next > > > > > chunk?s > > > > > size to its final (max) value.??The corresponding > > > > > load_acquire of > > > > > _head ensures any _next chunks are visibly complete.??It?s > > > > > the same > > > > > rationale for _next not requiring any special handling. At > > > > > least, > > > > > that?s how it looks to me right now; am I missing something. > > > > The _head chunk has its _size incremented on each add(). > > > > > > > > David > > > Agreed, and a load_acquire is needed for the _head chunk. > > > > > > I?m suggesting an ordinary load is sufficient for any _next > > > chunks as > > > we loop down the list. > > Ah! Now I get what you mean - sorry. Yes only the first loop > > iteration > > needs the load-acquire. So just: > > > > Chunk* c = (Chunk*) OrderAccess::load_ptr_acquire((volatile > > intptr_t*)&_head); > > const juint size = (c == NULL ? 0 : OrderAccess::load_acquire(&c- > > >size)); > > while (c != NULL) { > > ????size = OrderAccess::load_acquire(&c->size); > ???????size = c->size; Maybe I'm wrong, but just looking at this suggestion without more thought: what if the value of size changes just between the above load_acquire and the re-read of the size, and the actual value in the array has not been updated yet? If that optimization is done, please just peel out the first iteration. Thanks, ? Thomas From thomas.schatzl at oracle.com Thu Feb 16 09:06:30 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 16 Feb 2017 10:06:30 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> Message-ID: <1487235990.3407.17.camel@oracle.com> Hi, On Wed, 2017-02-15 at 21:41 -0500, Kim Barrett wrote: > > > > On Feb 15, 2017, at 10:07 AM, Erik Helin > > wrote: > > > > On 02/15/2017 02:48 AM, David Holmes wrote: > > > > > > Hi Erik, > > Hi David, > > > > thanks for having a look! Please see new patches at: > > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ > > - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ > ------------------------------------------------------------------- > ----------- > /src/share/vm/classfile/classLoaderData.cpp > ?138?????const juint size = OrderAccess::load_acquire(&c->size); > > I think all but the first chunk have a constant size of the > maximum size, so no need for acquire except for the first. > > ------------------------------------------------------------------- > ----------- > /src/share/vm/classfile/classLoaderData.cpp > ?173???VerifyContainsOopClosure cl(op); > ?174???oops_do(&cl); > > We don't care that this iterates over the entire list even if found > early? Not trying to defend the change: I thought the same, but it is only verification code, and as Coleen and you suggested, there are very few entries anyway. Also, the current interface (including the used OopClosure) do not allow returning a value anyway. Unless there is some AbortableOopClosure that can be used, I would recommend moving this to an RFE this late in the release. > ------------------------------------------------------------------- > ----------- > /src/share/vm/classfile/classLoaderData.hpp > ?162???struct ChunkedHandleList : public CHeapObj { > > As used, this doesn't seem to need to be CHeapObj. > > ------------------------------------------------------------------- > ----------- > /src/share/vm/classfile/classLoaderData.hpp > ?163?????struct Chunk : public CHeapObj { > ?164???????static const size_t CAPACITY = 64; > ?165???????oop data[CAPACITY]; > > Large fixed capacity seems problematic???I think there are lots of > "small" CLDs.??Don't anonymous classes have there own class loader? > A variable sized chunk with a capacity field doesn't seem like it > should introduce any new problems. While the default value might be too high, particularly a variable sized chunk size with very few entries will to add the overhead of the additional capacity field. For those ClassLoaders with very few entries, the existing Chunk already seems to have a quite high overhead. Maybe some specialization for anonymous vs. regular class loaders? Do you (Kim, Coleen) know typical sizes of this list for both kinds of class loaders? > The whole Chunk definition could also be moved to the .cpp file, with > only a forward declaration here. I would personally kind of prefer to keep them together here. I am fine with either way though. Thanks, ? Thomas From david.holmes at oracle.com Thu Feb 16 09:09:53 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Feb 2017 19:09:53 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <1487235317.3407.7.camel@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> <2b8c7837-0e9f-6c15-2be3-fa06f95ee47e@oracle.com> <1487235317.3407.7.camel@oracle.com> Message-ID: <2222b416-0cde-ac31-1d40-f5a04ad64cf0@oracle.com> On 16/02/2017 6:55 PM, Thomas Schatzl wrote: > Hi, > > On Thu, 2017-02-16 at 18:09 +1000, David Holmes wrote: >> Bah! didn't finish my edit ... >> >> On 16/02/2017 5:54 PM, David Holmes wrote: >>> >>> On 16/02/2017 5:46 PM, Kim Barrett wrote: >>>> >>>>> >>>>> On Feb 16, 2017, at 2:41 AM, David Holmes >>>> com> >>>>> wrote: >>>>> >>>>> On 16/02/2017 3:53 PM, Kim Barrett wrote: >>>>>> >>>>>>> >>>>>>>> >>>>>>>> /src/share/vm/classfile/classLoaderData.cpp >>>>>>>> 138 const juint size = OrderAccess::load_acquire(&c- >>>>>>>>> size); >>>>>>>> >>>>>>>> I think all but the first chunk have a constant size of >>>>>>>> the >>>>>>>> maximum size, so no need for acquire except for the >>>>>>>> first. >>>>>>> The size is incremented on each add, so this acquire sync's >>>>>>> with >>>>>>> the release of the new size, so that we are sure to see the >>>>>>> oop >>>>>>> that was just added. >>>>>>> >>>>>>> Though given the races between add and oops_do it may not >>>>>>> actually >>>>>>> matter. >>>>>> By ?first? I mean the first chunk. only the first chunk?s >>>>>> size is >>>>>> subject to modification. The release_store of _head that >>>>>> made the >>>>>> first chunk accessible will be after the setting of the next >>>>>> chunk?s >>>>>> size to its final (max) value. The corresponding >>>>>> load_acquire of >>>>>> _head ensures any _next chunks are visibly complete. It?s >>>>>> the same >>>>>> rationale for _next not requiring any special handling. At >>>>>> least, >>>>>> that?s how it looks to me right now; am I missing something. >>>>> The _head chunk has its _size incremented on each add(). >>>>> >>>>> David >>>> Agreed, and a load_acquire is needed for the _head chunk. >>>> >>>> I?m suggesting an ordinary load is sufficient for any _next >>>> chunks as >>>> we loop down the list. >>> Ah! Now I get what you mean - sorry. Yes only the first loop >>> iteration >>> needs the load-acquire. So just: >>> >>> Chunk* c = (Chunk*) OrderAccess::load_ptr_acquire((volatile >>> intptr_t*)&_head); >>> const juint size = (c == NULL ? 0 : OrderAccess::load_acquire(&c- >>>> size)); >>> while (c != NULL) { >>> size = OrderAccess::load_acquire(&c->size); >> size = c->size; > > Maybe I'm wrong, but just looking at this suggestion without more > thought: what if the value of size changes just between the above > load_acquire and the re-read of the size, and the actual value in the > array has not been updated yet? Yeah I screwed up peeling out the initial read of size. That line should come at the end of the loop after c=c->next; Thanks, David > If that optimization is done, please just peel out the first iteration. > > Thanks, > Thomas > From erik.helin at oracle.com Thu Feb 16 10:20:34 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 16 Feb 2017 11:20:34 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <1487235990.3407.17.camel@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1487235990.3407.17.camel@oracle.com> Message-ID: <2248b723-9e12-27a1-9552-2911c3e14bde@oracle.com> On 02/16/2017 10:06 AM, Thomas Schatzl wrote: > Hi, > > On Wed, 2017-02-15 at 21:41 -0500, Kim Barrett wrote: >>> >>> On Feb 15, 2017, at 10:07 AM, Erik Helin >>> wrote: >>> >>> On 02/15/2017 02:48 AM, David Holmes wrote: >>>> >>>> Hi Erik, >>> Hi David, >>> >>> thanks for having a look! Please see new patches at: >>> - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ >>> - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ >> ------------------------------------------------------------------- >> ----------- >> /src/share/vm/classfile/classLoaderData.cpp >> 138 const juint size = OrderAccess::load_acquire(&c->size); >> >> I think all but the first chunk have a constant size of the >> maximum size, so no need for acquire except for the first. >> >> ------------------------------------------------------------------- >> ----------- >> /src/share/vm/classfile/classLoaderData.cpp >> 173 VerifyContainsOopClosure cl(op); >> 174 oops_do(&cl); >> >> We don't care that this iterates over the entire list even if found >> early? > > Not trying to defend the change: I thought the same, but it is only > verification code, and as Coleen and you suggested, there are very few > entries anyway. Also, the current interface (including the used > OopClosure) do not allow returning a value anyway. Yep, this was my reasoning, there is no need to optimize this. > Unless there is some AbortableOopClosure that can be used, I would > recommend moving this to an RFE this late in the release. We do want AbortableOopClosure (I had a patch circling around a long time ago), but I don't think the optimization is worth it here. >> ------------------------------------------------------------------- >> ----------- >> /src/share/vm/classfile/classLoaderData.hpp >> 162 struct ChunkedHandleList : public CHeapObj { >> >> As used, this doesn't seem to need to be CHeapObj. Correct (also noticed by Coleen). >> ------------------------------------------------------------------- >> ----------- >> /src/share/vm/classfile/classLoaderData.hpp >> 163 struct Chunk : public CHeapObj { >> 164 static const size_t CAPACITY = 64; >> 165 oop data[CAPACITY]; >> >> Large fixed capacity seems problematic? I think there are lots of >> "small" CLDs. Don't anonymous classes have there own class loader? >> A variable sized chunk with a capacity field doesn't seem like it >> should introduce any new problems. > > While the default value might be too high, particularly a variable > sized chunk size with very few entries will to add the overhead of the > additional capacity field. > For those ClassLoaders with very few entries, the existing Chunk > already seems to have a quite high overhead. > > Maybe some specialization for anonymous vs. regular class loaders? > > Do you (Kim, Coleen) know typical sizes of this list for both kinds of > class loaders? > >> The whole Chunk definition could also be moved to the .cpp file, with >> only a forward declaration here. > > I would personally kind of prefer to keep them together here. I am fine > with either way though. I would also prefer to keep them together (but don't have a very strong opinion). Thanks, Erik > Thanks, > Thomas > From erik.helin at oracle.com Thu Feb 16 13:14:25 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 16 Feb 2017 14:14:25 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <2222b416-0cde-ac31-1d40-f5a04ad64cf0@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> <2b8c7837-0e9f-6c15-2be3-fa06f95ee47e@oracle.com> <1487235317.3407.7.camel@oracle.com> <2222b416-0cde-ac31-1d40-f5a04ad64cf0@oracle.com> Message-ID: On 02/16/2017 10:09 AM, David Holmes wrote: > On 16/02/2017 6:55 PM, Thomas Schatzl wrote: >> Hi, >> >> On Thu, 2017-02-16 at 18:09 +1000, David Holmes wrote: >>> Bah! didn't finish my edit ... >>> >>> On 16/02/2017 5:54 PM, David Holmes wrote: >>>> >>>> On 16/02/2017 5:46 PM, Kim Barrett wrote: >>>>> >>>>>> >>>>>> On Feb 16, 2017, at 2:41 AM, David Holmes >>>>> com> >>>>>> wrote: >>>>>> >>>>>> On 16/02/2017 3:53 PM, Kim Barrett wrote: >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> /src/share/vm/classfile/classLoaderData.cpp >>>>>>>>> 138 const juint size = OrderAccess::load_acquire(&c- >>>>>>>>>> size); >>>>>>>>> >>>>>>>>> I think all but the first chunk have a constant size of >>>>>>>>> the >>>>>>>>> maximum size, so no need for acquire except for the >>>>>>>>> first. >>>>>>>> The size is incremented on each add, so this acquire sync's >>>>>>>> with >>>>>>>> the release of the new size, so that we are sure to see the >>>>>>>> oop >>>>>>>> that was just added. >>>>>>>> >>>>>>>> Though given the races between add and oops_do it may not >>>>>>>> actually >>>>>>>> matter. >>>>>>> By ?first? I mean the first chunk. only the first chunk?s >>>>>>> size is >>>>>>> subject to modification. The release_store of _head that >>>>>>> made the >>>>>>> first chunk accessible will be after the setting of the next >>>>>>> chunk?s >>>>>>> size to its final (max) value. The corresponding >>>>>>> load_acquire of >>>>>>> _head ensures any _next chunks are visibly complete. It?s >>>>>>> the same >>>>>>> rationale for _next not requiring any special handling. At >>>>>>> least, >>>>>>> that?s how it looks to me right now; am I missing something. >>>>>> The _head chunk has its _size incremented on each add(). >>>>>> >>>>>> David >>>>> Agreed, and a load_acquire is needed for the _head chunk. >>>>> >>>>> I?m suggesting an ordinary load is sufficient for any _next >>>>> chunks as >>>>> we loop down the list. >>>> Ah! Now I get what you mean - sorry. Yes only the first loop >>>> iteration >>>> needs the load-acquire. So just: >>>> >>>> Chunk* c = (Chunk*) OrderAccess::load_ptr_acquire((volatile >>>> intptr_t*)&_head); >>>> const juint size = (c == NULL ? 0 : OrderAccess::load_acquire(&c- >>>>> size)); >>>> while (c != NULL) { >>>> size = OrderAccess::load_acquire(&c->size); >>> size = c->size; >> >> Maybe I'm wrong, but just looking at this suggestion without more >> thought: what if the value of size changes just between the above >> load_acquire and the re-read of the size, and the actual value in the >> array has not been updated yet? > > Yeah I screwed up peeling out the initial read of size. That line should > come at the end of the loop after c=c->next; I agree that we can implement this optimization, but I'm not sure it is worth it? The code will become quite a bit more cumbersome vs the straightforward code (well, as straightforward as lock free code gets) that is in the patch right now. Thanks, Erik > Thanks, > David > >> If that optimization is done, please just peel out the first iteration. >> >> Thanks, >> Thomas >> From david.holmes at oracle.com Thu Feb 16 13:15:16 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Feb 2017 23:15:16 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <1493e549-f8bf-2cd1-fd6d-28b0a1e52f16@oracle.com> <88680aba-527d-b46e-6b02-3a9b0047153a@oracle.com> <1950B9F7-80D3-45A3-89BA-2D2E95C8BF2D@oracle.com> <2b8c7837-0e9f-6c15-2be3-fa06f95ee47e@oracle.com> <1487235317.3407.7.camel@oracle.com> <2222b416-0cde-ac31-1d40-f5a04ad64cf0@oracle.com> Message-ID: <10ab1236-df66-e373-685a-840552306ced@oracle.com> On 16/02/2017 11:14 PM, Erik Helin wrote: > On 02/16/2017 10:09 AM, David Holmes wrote: >> On 16/02/2017 6:55 PM, Thomas Schatzl wrote: >>> Hi, >>> >>> On Thu, 2017-02-16 at 18:09 +1000, David Holmes wrote: >>>> Bah! didn't finish my edit ... >>>> >>>> On 16/02/2017 5:54 PM, David Holmes wrote: >>>>> >>>>> On 16/02/2017 5:46 PM, Kim Barrett wrote: >>>>>> >>>>>>> >>>>>>> On Feb 16, 2017, at 2:41 AM, David Holmes >>>>>> com> >>>>>>> wrote: >>>>>>> >>>>>>> On 16/02/2017 3:53 PM, Kim Barrett wrote: >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> /src/share/vm/classfile/classLoaderData.cpp >>>>>>>>>> 138 const juint size = OrderAccess::load_acquire(&c- >>>>>>>>>>> size); >>>>>>>>>> >>>>>>>>>> I think all but the first chunk have a constant size of >>>>>>>>>> the >>>>>>>>>> maximum size, so no need for acquire except for the >>>>>>>>>> first. >>>>>>>>> The size is incremented on each add, so this acquire sync's >>>>>>>>> with >>>>>>>>> the release of the new size, so that we are sure to see the >>>>>>>>> oop >>>>>>>>> that was just added. >>>>>>>>> >>>>>>>>> Though given the races between add and oops_do it may not >>>>>>>>> actually >>>>>>>>> matter. >>>>>>>> By ?first? I mean the first chunk. only the first chunk?s >>>>>>>> size is >>>>>>>> subject to modification. The release_store of _head that >>>>>>>> made the >>>>>>>> first chunk accessible will be after the setting of the next >>>>>>>> chunk?s >>>>>>>> size to its final (max) value. The corresponding >>>>>>>> load_acquire of >>>>>>>> _head ensures any _next chunks are visibly complete. It?s >>>>>>>> the same >>>>>>>> rationale for _next not requiring any special handling. At >>>>>>>> least, >>>>>>>> that?s how it looks to me right now; am I missing something. >>>>>>> The _head chunk has its _size incremented on each add(). >>>>>>> >>>>>>> David >>>>>> Agreed, and a load_acquire is needed for the _head chunk. >>>>>> >>>>>> I?m suggesting an ordinary load is sufficient for any _next >>>>>> chunks as >>>>>> we loop down the list. >>>>> Ah! Now I get what you mean - sorry. Yes only the first loop >>>>> iteration >>>>> needs the load-acquire. So just: >>>>> >>>>> Chunk* c = (Chunk*) OrderAccess::load_ptr_acquire((volatile >>>>> intptr_t*)&_head); >>>>> const juint size = (c == NULL ? 0 : OrderAccess::load_acquire(&c- >>>>>> size)); >>>>> while (c != NULL) { >>>>> size = OrderAccess::load_acquire(&c->size); >>>> size = c->size; >>> >>> Maybe I'm wrong, but just looking at this suggestion without more >>> thought: what if the value of size changes just between the above >>> load_acquire and the re-read of the size, and the actual value in the >>> array has not been updated yet? >> >> Yeah I screwed up peeling out the initial read of size. That line should >> come at the end of the loop after c=c->next; > > I agree that we can implement this optimization, but I'm not sure it is > worth it? The code will become quite a bit more cumbersome vs the > straightforward code (well, as straightforward as lock free code gets) > that is in the patch right now. I don't think it gets cumbersome. The removal of load_acquire simplifies things :) David > Thanks, > Erik > >> Thanks, >> David >> >>> If that optimization is done, please just peel out the first iteration. >>> >>> Thanks, >>> Thomas >>> From erik.helin at oracle.com Thu Feb 16 13:18:43 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 16 Feb 2017 14:18:43 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> Message-ID: <26597eee-2b9f-bc7e-ca3d-ac426d40e54b@oracle.com> Hi Kim, thanks for reviewing! On 02/16/2017 03:41 AM, Kim Barrett wrote: >> On Feb 15, 2017, at 10:07 AM, Erik Helin wrote: >> >> On 02/15/2017 02:48 AM, David Holmes wrote: >>> Hi Erik, >> >> Hi David, >> >> thanks for having a look! Please see new patches at: >> - incremental: http://cr.openjdk.java.net/~ehelin/8168914/00-01/ >> - full: http://cr.openjdk.java.net/~ehelin/8168914/01/ > > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.cpp > 138 const juint size = OrderAccess::load_acquire(&c->size); > > I think all but the first chunk have a constant size of the > maximum size, so no need for acquire except for the first. Please see my reply to my David. I agree that this can be optimized, but I'm not sure if it is worth it (the code will not become harder to understand)? > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.cpp > 173 VerifyContainsOopClosure cl(op); > 174 oops_do(&cl); > > We don't care that this iterates over the entire list even if found > early? Please see my reply to Thomas. We don't care :) > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.hpp > 162 struct ChunkedHandleList : public CHeapObj { > > As used, this doesn't seem to need to be CHeapObj. Yep, agree! > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.hpp > 163 struct Chunk : public CHeapObj { > 164 static const size_t CAPACITY = 64; > 165 oop data[CAPACITY]; > > Large fixed capacity seems problematic? I think there are lots of > "small" CLDs. Don't anonymous classes have there own class loader? > A variable sized chunk with a capacity field doesn't seem like it > should introduce any new problems. > > The whole Chunk definition could also be moved to the .cpp file, with > only a forward declaration here. Please see my reply to Coleen, I wanted to keep the same characteristics as the code has today when it uses a JNIHandleBlock. > ------------------------------------------------------------------------------ > /src/share/vm/classfile/classLoaderData.cpp > 626 return (jobject) _handles.add(h()); > and > 631 *((oop*) h) = NULL; > > Having spent a bunch of time recently dealing with and cleaning up a > bunch of places that break the jobject abstraction (though by no means > all), I'm not happy to see new ones. > > As Coleen said, there's discussion about how to refer to indirect > pointers to oops in runtime code. jobject has been used as an answer > for that in some places, but they might need to be changed. Hiding > the conversions as casts will just make that harder. I'm thinking > some named conversion functions are in order, instead of bare casts. Hmm, I would not like to introduce too much of Coleen's work in this patch. Do you have something minimal in mind that you think improves the situation? Or should we just note this place in the code and come back to it in 10? Thanks, Erik From erik.helin at oracle.com Thu Feb 16 14:13:53 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 16 Feb 2017 15:13:53 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> Message-ID: <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> On 02/15/2017 10:08 PM, coleen.phillimore at oracle.com wrote: > Hi Erik, Hey Coleen, thanks for reviewing! Please see new webrevs at: - incremental: http://cr.openjdk.java.net/~ehelin/8168914/01-02/ - full: http://cr.openjdk.java.net/~ehelin/8168914/02/ > I thought you were going to hide the declaration of class > ChunkedHandleList in classLoaderData.cpp since it is only used internally? > > And have a field in ClassLoaderData: ChunkedHandleList* > _chunked_handle_list; I didn't want to have a pointer to ChunkedHandleList just because I want to "hide" the implementation, I prefer to embed an instance (even thought that means I have to declare the struct/class in the .hpp file). > The class name "Chunk" is so overused! Haha, I agree that it is quite commonly used :) Fortunately there is some context here: Chunk is an inner class/struct to ChunkedHandleList. That hopefully gives the reader enough insight to distinguish this Chunk from other Chunks in the VM. > Here ChunkedHandleList should not be a subtype of CHeapObj. > It's a ValueObj. Agree, this is a left-over from an earlier state of the patch. Thanks for noticing! > I don't think it should be a struct either because even though it's > internal to ClassLoaderData, the fields could still be accessed outside > the class. Agree, I cleaned that up a bit. Thanks. > Why is > > 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { > > > it called unsafe? It is safe to zero the element in the block, isn't it? Because code using remove_handle_unsafe should be cautious. It is only safe to write NULL to the *((oop*) op) if: - the jobject originates from a call to ClassLoaderData::add_handle (the assert verifies this). - *op is the sole remaining pointer to **op (the object). - The native memory backing the chunked linked list is not freed. It is unfortunate that we even need to have the remove_handle_unsafe at all. An alternative is to let the only code using remove_handle_unsafe (ModuleEntry) do *((oop*) op) = NULL on its own. Would you prefer that? > One other reason to make it internal is that the CAPACITY of 64 is way > too big for the ClassLoaderData of anonymous classes which I think will > only have presently 1 (or 2 maybe) oops in it. And there are zillions > of them. Oops, thanks for catching this. I intended to use the same capacity as JNIHandleBlock, which uses 32 (don't recall where that 64 came from). As for using a smaller size, that seems to be a separate patch? For now I would prefer to keep the characteristics the same as for JNIHandleBlock. Thanks, Erik > Thanks, > Coleen > > > On 2/14/17 11:40 AM, Erik Helin wrote: >> Hi all, >> >> this patch aims to solve the bug [0] by making it safe for a GC to >> concurrently traverse the oops in a ClassLoaderData. >> >> The problem right now is that ClassLoaderData::_handles is a >> JNIHandleBlock. It is not safe for one thread to iterate over the oops >> in a JNIHandleBlock while another thread concurrently adds a new oop >> to the JNIHandleBlock. >> >> My proposed solution is to replace JNIHandleBlock with another data >> structure for ClassLoaderData. ClassLoaderData only needs a place to >> store oops that a GC can iterate over, it has no use for the rest of >> the methods in the JNIHandleBlock class. I have implemented a minimal >> chunked linked list data structure (internal to ClassLoaderData) with >> only two public methods: >> - add (only executed by one thread at a time) >> - oops_do (can be executed by any number of threads, possibly >> concurrently with a call to `add`) >> >> ChunkedHandleList uses load_acquire and release_store to synchronize >> access to the underlying chunks (arrays). Since ChunkedHandleList >> utilizes the (rather) minimal requirements of its user >> (ClassLoaderData), I decided to keep the class internal to >> ClassLoaderData for now. If other find it useful elsewhere, the we can >> try to create a more generalized version (this is trickier than it >> appears at first sight, I tried ;)). >> >> I also changed ClassLoaderData::remove_handle to write NULL to the >> oop* (that is represented by a jobject), instead of using a sentinel >> oop as done by JNIHandleBlock. The GC closures should be prepared to >> handle a field in a Java object becoming NULL, so this should be fine. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8168914 >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8168914/00/ >> >> Testing: >> - Tier 1 (aka JPRT), 2, 3, 4, 5. >> >> I would appreciate quite a few reviews on this patch, it contains a >> nice mixture of: >> - me venturing into the runtime code :) >> - lock free code >> - unreproducible bug (even though I'm sure of the root cause) >> >> Thanks for everyone participating in the discussions around this bug >> and potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and >> Jiangli. >> >> Thanks! >> Erik >> >> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 > From daniel.daugherty at oracle.com Thu Feb 16 15:40:49 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 16 Feb 2017 08:40:49 -0700 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <58A4D3CA.2060700@oracle.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> Message-ID: <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> Hi Max, Added a note to your bug. Interesting idea, but I think your data is a bit incomplete at the moment. Dan On 2/15/17 3:18 PM, Max Ockner wrote: > Hello all, > > We have filed a bug to remove the interpreter stack caching > optimization for jdk10. Ideally we can make this change *early* > during the jdk10 development cycle. See below for justification: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 > > Stack caching has been around for a long time and is intended to > replace some of the load/store (pop/push) operations with > corresponding register operations. The need for this optimization > arose before caching could adequately lessen the burden of memory > access. We have reevaluated the JVM stack caching optimization and > have found that it has a high memory footprint and is very costly to > maintain, but does not provide significant measurable or theoretical > benefit for us when used with modern hardware. > > Minimal Theoretical Benefit. > Because modern hardware does not slap us with the same cost for > accessing memory as it once did, the benefit of replacing memory > access with register access is far less dramatic now than it once was. > Additionally, the interpreter runs for a relatively short time before > relevant code sections are compiled. When the VM starts running > compiled code instead of interpreted code, performance should begin to > move asymptotically towards that of compiled code, diluting any > performance penalties from the interpreter to small performance > variations. > > No Measurable Benefit. > Please see the results files attached in the bug page. This change > was adapted for x86 and sparc, and interpreter performance was > measured with Specjvm98 (run with -Xint). No significant decrease in > performance was observed. > > Memory footprint and code complexity. > Stack caching in the JVM is implemented by switching the instruction > look-up table depending on the tos (top-of-stack) state. At any moment > there are is an active table consisting of one dispatch table for each > of the 10 tos states. When we enter a safepoint, we copy all 10 > safepoint dispatch tables into the active table. The additional entry > code makes this copy less efficient and makes any work in the > interpreter harder to debug. > > If we remove this optimization, we will: > - decrease memory usage in the interpreter, > - eliminated wasteful memory transactions during safepoints, > - decrease code complexity (a lot). > > Please let me know what you think. > Thanks, > Max > From kim.barrett at oracle.com Thu Feb 16 17:55:55 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 Feb 2017 12:55:55 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <26597eee-2b9f-bc7e-ca3d-ac426d40e54b@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <039a6866-cdb5-e494-da22-c471d7d2fe85@oracle.com> <459666a2-41d1-a1ab-82bc-e3c13a2c5ea8@oracle.com> <4815DF39-7E59-491C-BF45-D9F5BFD032F2@oracle.com> <26597eee-2b9f-bc7e-ca3d-ac426d40e54b@oracle.com> Message-ID: > On Feb 16, 2017, at 8:18 AM, Erik Helin wrote: > > Hi Kim, > > thanks for reviewing! > > On 02/16/2017 03:41 AM, Kim Barrett wrote: >> ------------------------------------------------------------------------------ >> /src/share/vm/classfile/classLoaderData.cpp >> 138 const juint size = OrderAccess::load_acquire(&c->size); >> >> I think all but the first chunk have a constant size of the >> maximum size, so no need for acquire except for the first. > > Please see my reply to my David. I agree that this can be optimized, but I'm not sure if it is worth it (the code will not become harder to understand)? I knew when I made the suggestion that it might increase the raw footprint of the source code. But I agree with David?s sentiment that removing dynamically unnecessary load_acquires helps readability. >> ------------------------------------------------------------------------------ >> /src/share/vm/classfile/classLoaderData.cpp >> 173 VerifyContainsOopClosure cl(op); >> 174 oops_do(&cl); >> >> We don't care that this iterates over the entire list even if found >> early? > > Please see my reply to Thomas. We don't care :) That?s what I guessed, hence making it a question. Okay. > >> ------------------------------------------------------------------------------ >> /src/share/vm/classfile/classLoaderData.cpp >> 626 return (jobject) _handles.add(h()); >> and >> 631 *((oop*) h) = NULL; >> >> Having spent a bunch of time recently dealing with and cleaning up a >> bunch of places that break the jobject abstraction (though by no means >> all), I'm not happy to see new ones. >> >> As Coleen said, there's discussion about how to refer to indirect >> pointers to oops in runtime code. jobject has been used as an answer >> for that in some places, but they might need to be changed. Hiding >> the conversions as casts will just make that harder. I'm thinking >> some named conversion functions are in order, instead of bare casts. > > Hmm, I would not like to introduce too much of Coleen's work in this patch. Do you have something minimal in mind that you think improves the situation? Or should we just note this place in the code and come back to it in 10? Some sort of ?convert_x_to_y? functions in JNIHandles. I added parts of one direction in 8166188, though that change has run into problems because more testing uncovered more abstraction breakage. > > Thanks, > Erik From kim.barrett at oracle.com Thu Feb 16 18:03:14 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 Feb 2017 13:03:14 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> Message-ID: <136801C3-1621-431A-98DB-16015D7F8FF0@oracle.com> > On Feb 16, 2017, at 9:13 AM, Erik Helin wrote: > > On 02/15/2017 10:08 PM, coleen.phillimore at oracle.com wrote: > >> Why is >> >> 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { >> >> >> it called unsafe? It is safe to zero the element in the block, isn't it? > > Because code using remove_handle_unsafe should be cautious. It is only safe to write NULL to the *((oop*) op) if: > - the jobject originates from a call to ClassLoaderData::add_handle > (the assert verifies this). > - *op is the sole remaining pointer to **op (the object). > - The native memory backing the chunked linked list is not freed. > > It is unfortunate that we even need to have the remove_handle_unsafe at all. An alternative is to let the only code using remove_handle_unsafe (ModuleEntry) do *((oop*) op) = NULL on its own. > > Would you prefer that? I think I prefer the (hopefully temporary?) API wart of this special function. From coleen.phillimore at oracle.com Thu Feb 16 18:07:43 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 16 Feb 2017 13:07:43 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <136801C3-1621-431A-98DB-16015D7F8FF0@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> <136801C3-1621-431A-98DB-16015D7F8FF0@oracle.com> Message-ID: <63e64c10-4396-ef3c-caea-e4d350bf34c5@oracle.com> On 2/16/17 1:03 PM, Kim Barrett wrote: >> On Feb 16, 2017, at 9:13 AM, Erik Helin wrote: >> >> On 02/15/2017 10:08 PM, coleen.phillimore at oracle.com wrote: >> >>> Why is >>> >>> 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { >>> >>> >>> it called unsafe? It is safe to zero the element in the block, isn't it? >> Because code using remove_handle_unsafe should be cautious. It is only safe to write NULL to the *((oop*) op) if: >> - the jobject originates from a call to ClassLoaderData::add_handle >> (the assert verifies this). >> - *op is the sole remaining pointer to **op (the object). >> - The native memory backing the chunked linked list is not freed. >> >> It is unfortunate that we even need to have the remove_handle_unsafe at all. An alternative is to let the only code using remove_handle_unsafe (ModuleEntry) do *((oop*) op) = NULL on its own. >> >> Would you prefer that? > I think I prefer the (hopefully temporary?) API wart of this special function. I like the existance of a function but the "unsafe" wording in the API says that calling it is unsafe, not that the contents could be unsafe if they did something other than zero the oop. So it's the API/name that I don't like. We will have this API for a long time because we will need to remove things from the _handles list (ie. resolved_references for redefintion or mirrors if indirect mirrors are used). thanks, Coleen > From coleen.phillimore at oracle.com Thu Feb 16 18:14:01 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 16 Feb 2017 13:14:01 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> Message-ID: On 2/16/17 9:13 AM, Erik Helin wrote: > On 02/15/2017 10:08 PM, coleen.phillimore at oracle.com wrote: >> Hi Erik, > > Hey Coleen, thanks for reviewing! Please see new webrevs at: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/01-02/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/02/ > >> I thought you were going to hide the declaration of class >> ChunkedHandleList in classLoaderData.cpp since it is only used >> internally? >> >> And have a field in ClassLoaderData: ChunkedHandleList* >> _chunked_handle_list; > > I didn't want to have a pointer to ChunkedHandleList just because I > want to "hide" the implementation, I prefer to embed an instance (even > thought that means I have to declare the struct/class in the .hpp file). My preference is different but this is fine. > >> The class name "Chunk" is so overused! > > Haha, I agree that it is quite commonly used :) Fortunately there is > some context here: Chunk is an inner class/struct to > ChunkedHandleList. That hopefully gives the reader enough insight to > distinguish this Chunk from other Chunks in the VM. > >> Here ChunkedHandleList should not be a subtype of CHeapObj. >> It's a ValueObj. > > Agree, this is a left-over from an earlier state of the patch. Thanks > for noticing! > >> I don't think it should be a struct either because even though it's >> internal to ClassLoaderData, the fields could still be accessed outside >> the class. > > Agree, I cleaned that up a bit. Thanks. > >> Why is >> >> 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { >> >> >> it called unsafe? It is safe to zero the element in the block, isn't >> it? > > Because code using remove_handle_unsafe should be cautious. It is only > safe to write NULL to the *((oop*) op) if: > - the jobject originates from a call to ClassLoaderData::add_handle > (the assert verifies this). > - *op is the sole remaining pointer to **op (the object). > - The native memory backing the chunked linked list is not freed. > > It is unfortunate that we even need to have the remove_handle_unsafe > at all. An alternative is to let the only code using > remove_handle_unsafe (ModuleEntry) do *((oop*) op) = NULL on its own. > > Would you prefer that? > I answered this in my last mail. >> One other reason to make it internal is that the CAPACITY of 64 is way >> too big for the ClassLoaderData of anonymous classes which I think will >> only have presently 1 (or 2 maybe) oops in it. And there are zillions >> of them. > > Oops, thanks for catching this. I intended to use the same capacity as > JNIHandleBlock, which uses 32 (don't recall where that 64 came from). > As for using a smaller size, that seems to be a separate patch? For > now I would prefer to keep the characteristics the same as for > JNIHandleBlock. If the whole ChunkHandleList class was internal, then you could pick 10 or something. I guess we'll see if anyone notices footprint and fix it then. JNIHandleBlock was 32 so this won't increase footprint. Thanks - the new webrev looks good. Coleen > > > Thanks, > Erik > >> Thanks, >> Coleen >> >> >> On 2/14/17 11:40 AM, Erik Helin wrote: >>> Hi all, >>> >>> this patch aims to solve the bug [0] by making it safe for a GC to >>> concurrently traverse the oops in a ClassLoaderData. >>> >>> The problem right now is that ClassLoaderData::_handles is a >>> JNIHandleBlock. It is not safe for one thread to iterate over the oops >>> in a JNIHandleBlock while another thread concurrently adds a new oop >>> to the JNIHandleBlock. >>> >>> My proposed solution is to replace JNIHandleBlock with another data >>> structure for ClassLoaderData. ClassLoaderData only needs a place to >>> store oops that a GC can iterate over, it has no use for the rest of >>> the methods in the JNIHandleBlock class. I have implemented a minimal >>> chunked linked list data structure (internal to ClassLoaderData) with >>> only two public methods: >>> - add (only executed by one thread at a time) >>> - oops_do (can be executed by any number of threads, possibly >>> concurrently with a call to `add`) >>> >>> ChunkedHandleList uses load_acquire and release_store to synchronize >>> access to the underlying chunks (arrays). Since ChunkedHandleList >>> utilizes the (rather) minimal requirements of its user >>> (ClassLoaderData), I decided to keep the class internal to >>> ClassLoaderData for now. If other find it useful elsewhere, the we can >>> try to create a more generalized version (this is trickier than it >>> appears at first sight, I tried ;)). >>> >>> I also changed ClassLoaderData::remove_handle to write NULL to the >>> oop* (that is represented by a jobject), instead of using a sentinel >>> oop as done by JNIHandleBlock. The GC closures should be prepared to >>> handle a field in a Java object becoming NULL, so this should be fine. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8168914 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8168914/00/ >>> >>> Testing: >>> - Tier 1 (aka JPRT), 2, 3, 4, 5. >>> >>> I would appreciate quite a few reviews on this patch, it contains a >>> nice mixture of: >>> - me venturing into the runtime code :) >>> - lock free code >>> - unreproducible bug (even though I'm sure of the root cause) >>> >>> Thanks for everyone participating in the discussions around this bug >>> and potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and >>> Jiangli. >>> >>> Thanks! >>> Erik >>> >>> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 >> From kim.barrett at oracle.com Thu Feb 16 18:51:50 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 Feb 2017 13:51:50 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> Message-ID: <2FC6A0BB-7125-43FC-9E3D-5CB67426AE9B@oracle.com> src/share/vm/classfile/classLoaderData.hpp 179 // Only one thread can add a time, guarded by the Metaspace_lock. Only one thread at a time time can add? > On Feb 16, 2017, at 9:13 AM, Erik Helin wrote: > > On 02/15/2017 10:08 PM, coleen.phillimore at oracle.com wrote: >> Hi Erik, > > Hey Coleen, thanks for reviewing! Please see new webrevs at: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/01-02/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/02/ > From daniel.daugherty at oracle.com Thu Feb 16 19:21:22 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 16 Feb 2017 12:21:22 -0700 Subject: URGENT: RFR: [BACKOUT] fix for JDK-8166188 Message-ID: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> Greetings, Sending this backout code review to the same aliases used for: JDK-8166188 G1 Needs pre barrier on dereference of weak JNI handles https://bugs.openjdk.java.net/browse/JDK-8166188 more as a heads up that JDK-8166188 might be getting backed out. I only need reviews from Kim and Jesper. Doing a contingent backout of Kim's fix for the above due to some unexpected test failures with some JDK tests: JDK-8175085 8166188 causes JDK test failures https://bugs.openjdk.java.net/browse/JDK-8175085 JDK-8175086 [BACKOUT] fix for JDK-8166188 https://bugs.openjdk.java.net/browse/JDK-8175086 Webrev URL: http://cr.openjdk.java.net/~dcubed/8175086-webrev/0-jdk9-hs/ I used "hg backout 3bc68ff68250" which created a child changeset of Kim's: changeset: 12639:3bc68ff68250 user: kbarrett date: Wed Feb 15 22:19:13 2017 -0500 summary: 8166188: G1 Needs pre barrier on dereference of weak JNI handles The backout changeset: changeset: 12644:ee34d6aabf5f parent: 12639:3bc68ff68250 user: dcubed date: Thu Feb 16 10:41:19 2017 -0800 summary: 8175086: [BACKOUT] fix for JDK-8166188 and then I used "hg merge" to merge with current tip. This was a "branch" merge with no conflicts. Sanity checks: - The parent of Kim's changeset is: changeset: 12638:35db0413819a user: roland date: Wed Feb 15 17:26:37 2017 -0800 summary: 8174164: SafePointNode::_replaced_nodes breaks with irreducible loops - hg diff -r 35db0413819a -r ee34d6aabf5f shows no output so the backout is clean. - $ wc -l files.list 37 files.list - $ hg diff -r 35db0413819a -r tip `cat files.list` shows no output so the merge with the tip is clean. Thanks, in advance, for reviews from Jesper and Kim!! Dan From kim.barrett at oracle.com Thu Feb 16 19:43:59 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 16 Feb 2017 14:43:59 -0500 Subject: URGENT: RFR: [BACKOUT] fix for JDK-8166188 In-Reply-To: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> References: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> Message-ID: <41EAF2BB-30D8-4D9B-8134-F557D4D83543@oracle.com> looks good. > On Feb 16, 2017, at 2:21 PM, Daniel D. Daugherty wrote: > > Greetings, > > Sending this backout code review to the same aliases used for: > > JDK-8166188 G1 Needs pre barrier on dereference of weak JNI handles > https://bugs.openjdk.java.net/browse/JDK-8166188 > > more as a heads up that JDK-8166188 might be getting backed out. > > I only need reviews from Kim and Jesper. > > Doing a contingent backout of Kim's fix for the above due to some > unexpected test failures with some JDK tests: > > JDK-8175085 8166188 causes JDK test failures > https://bugs.openjdk.java.net/browse/JDK-8175085 > > JDK-8175086 [BACKOUT] fix for JDK-8166188 > https://bugs.openjdk.java.net/browse/JDK-8175086 > > > Webrev URL: http://cr.openjdk.java.net/~dcubed/8175086-webrev/0-jdk9-hs/ > > > I used "hg backout 3bc68ff68250" which created a child > changeset of Kim's: > > changeset: 12639:3bc68ff68250 > user: kbarrett > date: Wed Feb 15 22:19:13 2017 -0500 > summary: 8166188: G1 Needs pre barrier on dereference of weak JNI handles > > The backout changeset: > > changeset: 12644:ee34d6aabf5f > parent: 12639:3bc68ff68250 > user: dcubed > date: Thu Feb 16 10:41:19 2017 -0800 > summary: 8175086: [BACKOUT] fix for JDK-8166188 > > and then I used "hg merge" to merge with current tip. > This was a "branch" merge with no conflicts. > > > Sanity checks: > > - The parent of Kim's changeset is: > > changeset: 12638:35db0413819a > user: roland > date: Wed Feb 15 17:26:37 2017 -0800 > summary: 8174164: SafePointNode::_replaced_nodes breaks with irreducible loops > > - hg diff -r 35db0413819a -r ee34d6aabf5f > > shows no output so the backout is clean. > > - $ wc -l files.list > 37 files.list > > - $ hg diff -r 35db0413819a -r tip `cat files.list` > > shows no output so the merge with the tip is clean. > > Thanks, in advance, for reviews from Jesper and Kim!! > > Dan From jesper.wilhelmsson at oracle.com Thu Feb 16 19:56:32 2017 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 16 Feb 2017 20:56:32 +0100 Subject: URGENT: RFR: [BACKOUT] fix for JDK-8166188 In-Reply-To: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> References: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> Message-ID: <1a37f025-1101-50e5-0171-c7e28a941668@oracle.com> Looks good! /Jesper On 2017-02-16 20:21, Daniel D. Daugherty wrote: > Greetings, > > Sending this backout code review to the same aliases used for: > > JDK-8166188 G1 Needs pre barrier on dereference of weak JNI handles > https://bugs.openjdk.java.net/browse/JDK-8166188 > > more as a heads up that JDK-8166188 might be getting backed out. > > I only need reviews from Kim and Jesper. > > Doing a contingent backout of Kim's fix for the above due to some > unexpected test failures with some JDK tests: > > JDK-8175085 8166188 causes JDK test failures > https://bugs.openjdk.java.net/browse/JDK-8175085 > > JDK-8175086 [BACKOUT] fix for JDK-8166188 > https://bugs.openjdk.java.net/browse/JDK-8175086 > > > Webrev URL: http://cr.openjdk.java.net/~dcubed/8175086-webrev/0-jdk9-hs/ > > > I used "hg backout 3bc68ff68250" which created a child > changeset of Kim's: > > changeset: 12639:3bc68ff68250 > user: kbarrett > date: Wed Feb 15 22:19:13 2017 -0500 > summary: 8166188: G1 Needs pre barrier on dereference of weak JNI > handles > > The backout changeset: > > changeset: 12644:ee34d6aabf5f > parent: 12639:3bc68ff68250 > user: dcubed > date: Thu Feb 16 10:41:19 2017 -0800 > summary: 8175086: [BACKOUT] fix for JDK-8166188 > > and then I used "hg merge" to merge with current tip. > This was a "branch" merge with no conflicts. > > > Sanity checks: > > - The parent of Kim's changeset is: > > changeset: 12638:35db0413819a > user: roland > date: Wed Feb 15 17:26:37 2017 -0800 > summary: 8174164: SafePointNode::_replaced_nodes breaks with > irreducible loops > > - hg diff -r 35db0413819a -r ee34d6aabf5f > > shows no output so the backout is clean. > > - $ wc -l files.list > 37 files.list > > - $ hg diff -r 35db0413819a -r tip `cat files.list` > > shows no output so the merge with the tip is clean. > > Thanks, in advance, for reviews from Jesper and Kim!! > > Dan > From daniel.daugherty at oracle.com Thu Feb 16 20:43:41 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 16 Feb 2017 13:43:41 -0700 Subject: URGENT: RFR: [BACKOUT] fix for JDK-8166188 In-Reply-To: <41EAF2BB-30D8-4D9B-8134-F557D4D83543@oracle.com> References: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> <41EAF2BB-30D8-4D9B-8134-F557D4D83543@oracle.com> Message-ID: <1335e5aa-2256-d414-c65b-68f8d3e32c14@oracle.com> Thanks for the quick review! Dan On 2/16/17 12:43 PM, Kim Barrett wrote: > looks good. > >> On Feb 16, 2017, at 2:21 PM, Daniel D. Daugherty wrote: >> >> Greetings, >> >> Sending this backout code review to the same aliases used for: >> >> JDK-8166188 G1 Needs pre barrier on dereference of weak JNI handles >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> more as a heads up that JDK-8166188 might be getting backed out. >> >> I only need reviews from Kim and Jesper. >> >> Doing a contingent backout of Kim's fix for the above due to some >> unexpected test failures with some JDK tests: >> >> JDK-8175085 8166188 causes JDK test failures >> https://bugs.openjdk.java.net/browse/JDK-8175085 >> >> JDK-8175086 [BACKOUT] fix for JDK-8166188 >> https://bugs.openjdk.java.net/browse/JDK-8175086 >> >> >> Webrev URL: http://cr.openjdk.java.net/~dcubed/8175086-webrev/0-jdk9-hs/ >> >> >> I used "hg backout 3bc68ff68250" which created a child >> changeset of Kim's: >> >> changeset: 12639:3bc68ff68250 >> user: kbarrett >> date: Wed Feb 15 22:19:13 2017 -0500 >> summary: 8166188: G1 Needs pre barrier on dereference of weak JNI handles >> >> The backout changeset: >> >> changeset: 12644:ee34d6aabf5f >> parent: 12639:3bc68ff68250 >> user: dcubed >> date: Thu Feb 16 10:41:19 2017 -0800 >> summary: 8175086: [BACKOUT] fix for JDK-8166188 >> >> and then I used "hg merge" to merge with current tip. >> This was a "branch" merge with no conflicts. >> >> >> Sanity checks: >> >> - The parent of Kim's changeset is: >> >> changeset: 12638:35db0413819a >> user: roland >> date: Wed Feb 15 17:26:37 2017 -0800 >> summary: 8174164: SafePointNode::_replaced_nodes breaks with irreducible loops >> >> - hg diff -r 35db0413819a -r ee34d6aabf5f >> >> shows no output so the backout is clean. >> >> - $ wc -l files.list >> 37 files.list >> >> - $ hg diff -r 35db0413819a -r tip `cat files.list` >> >> shows no output so the merge with the tip is clean. >> >> Thanks, in advance, for reviews from Jesper and Kim!! >> >> Dan > From daniel.daugherty at oracle.com Thu Feb 16 20:43:53 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 16 Feb 2017 13:43:53 -0700 Subject: URGENT: RFR: [BACKOUT] fix for JDK-8166188 In-Reply-To: <1a37f025-1101-50e5-0171-c7e28a941668@oracle.com> References: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> <1a37f025-1101-50e5-0171-c7e28a941668@oracle.com> Message-ID: <172bd1c5-3785-7bce-00b4-e4eb8d102e89@oracle.com> Thanks for the quick review! Dan On 2/16/17 12:56 PM, Jesper Wilhelmsson wrote: > Looks good! > > /Jesper > > > On 2017-02-16 20:21, Daniel D. Daugherty wrote: >> Greetings, >> >> Sending this backout code review to the same aliases used for: >> >> JDK-8166188 G1 Needs pre barrier on dereference of weak JNI handles >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> more as a heads up that JDK-8166188 might be getting backed out. >> >> I only need reviews from Kim and Jesper. >> >> Doing a contingent backout of Kim's fix for the above due to some >> unexpected test failures with some JDK tests: >> >> JDK-8175085 8166188 causes JDK test failures >> https://bugs.openjdk.java.net/browse/JDK-8175085 >> >> JDK-8175086 [BACKOUT] fix for JDK-8166188 >> https://bugs.openjdk.java.net/browse/JDK-8175086 >> >> >> Webrev URL: http://cr.openjdk.java.net/~dcubed/8175086-webrev/0-jdk9-hs/ >> >> >> I used "hg backout 3bc68ff68250" which created a child >> changeset of Kim's: >> >> changeset: 12639:3bc68ff68250 >> user: kbarrett >> date: Wed Feb 15 22:19:13 2017 -0500 >> summary: 8166188: G1 Needs pre barrier on dereference of weak JNI >> handles >> >> The backout changeset: >> >> changeset: 12644:ee34d6aabf5f >> parent: 12639:3bc68ff68250 >> user: dcubed >> date: Thu Feb 16 10:41:19 2017 -0800 >> summary: 8175086: [BACKOUT] fix for JDK-8166188 >> >> and then I used "hg merge" to merge with current tip. >> This was a "branch" merge with no conflicts. >> >> >> Sanity checks: >> >> - The parent of Kim's changeset is: >> >> changeset: 12638:35db0413819a >> user: roland >> date: Wed Feb 15 17:26:37 2017 -0800 >> summary: 8174164: SafePointNode::_replaced_nodes breaks with >> irreducible loops >> >> - hg diff -r 35db0413819a -r ee34d6aabf5f >> >> shows no output so the backout is clean. >> >> - $ wc -l files.list >> 37 files.list >> >> - $ hg diff -r 35db0413819a -r tip `cat files.list` >> >> shows no output so the merge with the tip is clean. >> >> Thanks, in advance, for reviews from Jesper and Kim!! >> >> Dan >> > From martin.doerr at sap.com Fri Feb 17 12:07:41 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 17 Feb 2017 12:07:41 +0000 Subject: URGENT: RFR: [BACKOUT] fix for JDK-8166188 In-Reply-To: <172bd1c5-3785-7bce-00b4-e4eb8d102e89@oracle.com> References: <66ddf3ea-83e0-ab69-443f-e33dc3be7b8f@oracle.com> <1a37f025-1101-50e5-0171-c7e28a941668@oracle.com> <172bd1c5-3785-7bce-00b4-e4eb8d102e89@oracle.com> Message-ID: <1d2c4abf5b3b48d68093dc49e7400f4f@sap.com> Hi, I should have looked at our test results on SPARC earlier. There are also crashes in e.g. jni_fast_GetBooleanField. PPC64 and s390 don't have FastJNIAccessors at the moment. That's why I haven't seen such problems there. jni_GetBooleanField seems to work fine because it uses JNIHandles::resolve_non_null(obj). Best regards, Martin -----Original Message----- From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Daniel D. Daugherty Sent: Donnerstag, 16. Februar 2017 21:44 To: Jesper Wilhelmsson ; hotspot-dev developers ; s390x-port-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: Re: URGENT: RFR: [BACKOUT] fix for JDK-8166188 Thanks for the quick review! Dan On 2/16/17 12:56 PM, Jesper Wilhelmsson wrote: > Looks good! > > /Jesper > > > On 2017-02-16 20:21, Daniel D. Daugherty wrote: >> Greetings, >> >> Sending this backout code review to the same aliases used for: >> >> JDK-8166188 G1 Needs pre barrier on dereference of weak JNI handles >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> more as a heads up that JDK-8166188 might be getting backed out. >> >> I only need reviews from Kim and Jesper. >> >> Doing a contingent backout of Kim's fix for the above due to some >> unexpected test failures with some JDK tests: >> >> JDK-8175085 8166188 causes JDK test failures >> https://bugs.openjdk.java.net/browse/JDK-8175085 >> >> JDK-8175086 [BACKOUT] fix for JDK-8166188 >> https://bugs.openjdk.java.net/browse/JDK-8175086 >> >> >> Webrev URL: http://cr.openjdk.java.net/~dcubed/8175086-webrev/0-jdk9-hs/ >> >> >> I used "hg backout 3bc68ff68250" which created a child >> changeset of Kim's: >> >> changeset: 12639:3bc68ff68250 >> user: kbarrett >> date: Wed Feb 15 22:19:13 2017 -0500 >> summary: 8166188: G1 Needs pre barrier on dereference of weak JNI >> handles >> >> The backout changeset: >> >> changeset: 12644:ee34d6aabf5f >> parent: 12639:3bc68ff68250 >> user: dcubed >> date: Thu Feb 16 10:41:19 2017 -0800 >> summary: 8175086: [BACKOUT] fix for JDK-8166188 >> >> and then I used "hg merge" to merge with current tip. >> This was a "branch" merge with no conflicts. >> >> >> Sanity checks: >> >> - The parent of Kim's changeset is: >> >> changeset: 12638:35db0413819a >> user: roland >> date: Wed Feb 15 17:26:37 2017 -0800 >> summary: 8174164: SafePointNode::_replaced_nodes breaks with >> irreducible loops >> >> - hg diff -r 35db0413819a -r ee34d6aabf5f >> >> shows no output so the backout is clean. >> >> - $ wc -l files.list >> 37 files.list >> >> - $ hg diff -r 35db0413819a -r tip `cat files.list` >> >> shows no output so the merge with the tip is clean. >> >> Thanks, in advance, for reviews from Jesper and Kim!! >> >> Dan >> > From claes.redestad at oracle.com Sat Feb 18 10:46:59 2017 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 18 Feb 2017 11:46:59 +0100 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> Message-ID: Hi, I've seen Max has run plenty of tests on our internal performance infrastructure and everything I've seen there seems to corroborate the idea that this removal is OK from a performance point of view, the footprint improvements are small but significant and any negative performance impact on throughput benchmarks is at noise levels even with -Xint (it appears many benchmarks time out with this setting both before and after, though; Max, let's discuss offline how to deal with that :-)) I expect this will be tested more thoroughly once adapted to all platforms (which I assume is the intent?), but see no concern from a performance testing point of view: Do it! Thanks! /Claes On 2017-02-16 16:40, Daniel D. Daugherty wrote: > Hi Max, > > Added a note to your bug. Interesting idea, but I think your data is > a bit incomplete at the moment. > > Dan > > > On 2/15/17 3:18 PM, Max Ockner wrote: >> Hello all, >> >> We have filed a bug to remove the interpreter stack caching >> optimization for jdk10. Ideally we can make this change *early* >> during the jdk10 development cycle. See below for justification: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >> >> Stack caching has been around for a long time and is intended to >> replace some of the load/store (pop/push) operations with >> corresponding register operations. The need for this optimization >> arose before caching could adequately lessen the burden of memory >> access. We have reevaluated the JVM stack caching optimization and >> have found that it has a high memory footprint and is very costly to >> maintain, but does not provide significant measurable or theoretical >> benefit for us when used with modern hardware. >> >> Minimal Theoretical Benefit. >> Because modern hardware does not slap us with the same cost for >> accessing memory as it once did, the benefit of replacing memory >> access with register access is far less dramatic now than it once was. >> Additionally, the interpreter runs for a relatively short time before >> relevant code sections are compiled. When the VM starts running >> compiled code instead of interpreted code, performance should begin to >> move asymptotically towards that of compiled code, diluting any >> performance penalties from the interpreter to small performance >> variations. >> >> No Measurable Benefit. >> Please see the results files attached in the bug page. This change >> was adapted for x86 and sparc, and interpreter performance was >> measured with Specjvm98 (run with -Xint). No significant decrease in >> performance was observed. >> >> Memory footprint and code complexity. >> Stack caching in the JVM is implemented by switching the instruction >> look-up table depending on the tos (top-of-stack) state. At any moment >> there are is an active table consisting of one dispatch table for each >> of the 10 tos states. When we enter a safepoint, we copy all 10 >> safepoint dispatch tables into the active table. The additional entry >> code makes this copy less efficient and makes any work in the >> interpreter harder to debug. >> >> If we remove this optimization, we will: >> - decrease memory usage in the interpreter, >> - eliminated wasteful memory transactions during safepoints, >> - decrease code complexity (a lot). >> >> Please let me know what you think. >> Thanks, >> Max >> > From daniel.daugherty at oracle.com Sat Feb 18 15:50:55 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Sat, 18 Feb 2017 08:50:55 -0700 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> Message-ID: <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> If Claes is happy with the perf testing, then I'm happy. :-) Dan On 2/18/17 3:46 AM, Claes Redestad wrote: > Hi, > > I've seen Max has run plenty of tests on our internal performance > infrastructure and everything I've seen there seems to corroborate the > idea that this removal is OK from a performance point of view, the > footprint improvements are small but significant and any negative > performance impact on throughput benchmarks is at noise levels even > with -Xint (it appears many benchmarks time out with this setting > both before and after, though; Max, let's discuss offline how to > deal with that :-)) > > I expect this will be tested more thoroughly once adapted to all > platforms (which I assume is the intent?), but see no concern from > a performance testing point of view: Do it! > > Thanks! > > /Claes > > On 2017-02-16 16:40, Daniel D. Daugherty wrote: >> Hi Max, >> >> Added a note to your bug. Interesting idea, but I think your data is >> a bit incomplete at the moment. >> >> Dan >> >> >> On 2/15/17 3:18 PM, Max Ockner wrote: >>> Hello all, >>> >>> We have filed a bug to remove the interpreter stack caching >>> optimization for jdk10. Ideally we can make this change *early* >>> during the jdk10 development cycle. See below for justification: >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>> >>> Stack caching has been around for a long time and is intended to >>> replace some of the load/store (pop/push) operations with >>> corresponding register operations. The need for this optimization >>> arose before caching could adequately lessen the burden of memory >>> access. We have reevaluated the JVM stack caching optimization and >>> have found that it has a high memory footprint and is very costly to >>> maintain, but does not provide significant measurable or theoretical >>> benefit for us when used with modern hardware. >>> >>> Minimal Theoretical Benefit. >>> Because modern hardware does not slap us with the same cost for >>> accessing memory as it once did, the benefit of replacing memory >>> access with register access is far less dramatic now than it once was. >>> Additionally, the interpreter runs for a relatively short time before >>> relevant code sections are compiled. When the VM starts running >>> compiled code instead of interpreted code, performance should begin to >>> move asymptotically towards that of compiled code, diluting any >>> performance penalties from the interpreter to small performance >>> variations. >>> >>> No Measurable Benefit. >>> Please see the results files attached in the bug page. This change >>> was adapted for x86 and sparc, and interpreter performance was >>> measured with Specjvm98 (run with -Xint). No significant decrease in >>> performance was observed. >>> >>> Memory footprint and code complexity. >>> Stack caching in the JVM is implemented by switching the instruction >>> look-up table depending on the tos (top-of-stack) state. At any moment >>> there are is an active table consisting of one dispatch table for each >>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>> safepoint dispatch tables into the active table. The additional entry >>> code makes this copy less efficient and makes any work in the >>> interpreter harder to debug. >>> >>> If we remove this optimization, we will: >>> - decrease memory usage in the interpreter, >>> - eliminated wasteful memory transactions during safepoints, >>> - decrease code complexity (a lot). >>> >>> Please let me know what you think. >>> Thanks, >>> Max >>> >> From coleen.phillimore at oracle.com Sat Feb 18 16:14:34 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sat, 18 Feb 2017 11:14:34 -0500 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> Message-ID: When Max gets back from the long weekend, he'll post the platforms in your bug. It's amazing that for -Xint there's no significant difference. I've seen -Xint performance of 15% slower cause a 2% slowdown with server but that was before tiered compilation. The reason for this query was to see what developers for the other platform ports think, since this change would affect all of the platforms. Thanks, Coleen On 2/18/17 10:50 AM, Daniel D. Daugherty wrote: > If Claes is happy with the perf testing, then I'm happy. :-) > > Dan > > > On 2/18/17 3:46 AM, Claes Redestad wrote: >> Hi, >> >> I've seen Max has run plenty of tests on our internal performance >> infrastructure and everything I've seen there seems to corroborate the >> idea that this removal is OK from a performance point of view, the >> footprint improvements are small but significant and any negative >> performance impact on throughput benchmarks is at noise levels even >> with -Xint (it appears many benchmarks time out with this setting >> both before and after, though; Max, let's discuss offline how to >> deal with that :-)) >> >> I expect this will be tested more thoroughly once adapted to all >> platforms (which I assume is the intent?), but see no concern from >> a performance testing point of view: Do it! >> >> Thanks! >> >> /Claes >> >> On 2017-02-16 16:40, Daniel D. Daugherty wrote: >>> Hi Max, >>> >>> Added a note to your bug. Interesting idea, but I think your data is >>> a bit incomplete at the moment. >>> >>> Dan >>> >>> >>> On 2/15/17 3:18 PM, Max Ockner wrote: >>>> Hello all, >>>> >>>> We have filed a bug to remove the interpreter stack caching >>>> optimization for jdk10. Ideally we can make this change *early* >>>> during the jdk10 development cycle. See below for justification: >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>>> >>>> Stack caching has been around for a long time and is intended to >>>> replace some of the load/store (pop/push) operations with >>>> corresponding register operations. The need for this optimization >>>> arose before caching could adequately lessen the burden of memory >>>> access. We have reevaluated the JVM stack caching optimization and >>>> have found that it has a high memory footprint and is very costly to >>>> maintain, but does not provide significant measurable or theoretical >>>> benefit for us when used with modern hardware. >>>> >>>> Minimal Theoretical Benefit. >>>> Because modern hardware does not slap us with the same cost for >>>> accessing memory as it once did, the benefit of replacing memory >>>> access with register access is far less dramatic now than it once was. >>>> Additionally, the interpreter runs for a relatively short time before >>>> relevant code sections are compiled. When the VM starts running >>>> compiled code instead of interpreted code, performance should begin to >>>> move asymptotically towards that of compiled code, diluting any >>>> performance penalties from the interpreter to small performance >>>> variations. >>>> >>>> No Measurable Benefit. >>>> Please see the results files attached in the bug page. This change >>>> was adapted for x86 and sparc, and interpreter performance was >>>> measured with Specjvm98 (run with -Xint). No significant decrease in >>>> performance was observed. >>>> >>>> Memory footprint and code complexity. >>>> Stack caching in the JVM is implemented by switching the instruction >>>> look-up table depending on the tos (top-of-stack) state. At any moment >>>> there are is an active table consisting of one dispatch table for each >>>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>>> safepoint dispatch tables into the active table. The additional entry >>>> code makes this copy less efficient and makes any work in the >>>> interpreter harder to debug. >>>> >>>> If we remove this optimization, we will: >>>> - decrease memory usage in the interpreter, >>>> - eliminated wasteful memory transactions during safepoints, >>>> - decrease code complexity (a lot). >>>> >>>> Please let me know what you think. >>>> Thanks, >>>> Max >>>> >>> > From ioi.lam at oracle.com Sat Feb 18 17:55:16 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 18 Feb 2017 09:55:16 -0800 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> Message-ID: <02AA9D0F-21A7-4CEA-95A2-3AA57910F014@oracle.com> I think it's worthwhile to hand craft a micro benchmark with lots of stack operations and see if there's any performance in Xint mode. Also, run this on a wide range of architectures such as 10 year old x86 vs latest x86, ARM, etc. That will give us more insights than the results from large complicated benchmarks. Ioi > On Feb 18, 2017, at 8:14 AM, coleen.phillimore at oracle.com wrote: > > When Max gets back from the long weekend, he'll post the platforms in your bug. > > It's amazing that for -Xint there's no significant difference. I've seen -Xint performance of 15% slower cause a 2% slowdown with server but that was before tiered compilation. > > The reason for this query was to see what developers for the other platform ports think, since this change would affect all of the platforms. > > Thanks, > Coleen > >> On 2/18/17 10:50 AM, Daniel D. Daugherty wrote: >> If Claes is happy with the perf testing, then I'm happy. :-) >> >> Dan >> >> >>> On 2/18/17 3:46 AM, Claes Redestad wrote: >>> Hi, >>> >>> I've seen Max has run plenty of tests on our internal performance >>> infrastructure and everything I've seen there seems to corroborate the >>> idea that this removal is OK from a performance point of view, the >>> footprint improvements are small but significant and any negative >>> performance impact on throughput benchmarks is at noise levels even >>> with -Xint (it appears many benchmarks time out with this setting >>> both before and after, though; Max, let's discuss offline how to >>> deal with that :-)) >>> >>> I expect this will be tested more thoroughly once adapted to all >>> platforms (which I assume is the intent?), but see no concern from >>> a performance testing point of view: Do it! >>> >>> Thanks! >>> >>> /Claes >>> >>>> On 2017-02-16 16:40, Daniel D. Daugherty wrote: >>>> Hi Max, >>>> >>>> Added a note to your bug. Interesting idea, but I think your data is >>>> a bit incomplete at the moment. >>>> >>>> Dan >>>> >>>> >>>>> On 2/15/17 3:18 PM, Max Ockner wrote: >>>>> Hello all, >>>>> >>>>> We have filed a bug to remove the interpreter stack caching >>>>> optimization for jdk10. Ideally we can make this change *early* >>>>> during the jdk10 development cycle. See below for justification: >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>>>> >>>>> Stack caching has been around for a long time and is intended to >>>>> replace some of the load/store (pop/push) operations with >>>>> corresponding register operations. The need for this optimization >>>>> arose before caching could adequately lessen the burden of memory >>>>> access. We have reevaluated the JVM stack caching optimization and >>>>> have found that it has a high memory footprint and is very costly to >>>>> maintain, but does not provide significant measurable or theoretical >>>>> benefit for us when used with modern hardware. >>>>> >>>>> Minimal Theoretical Benefit. >>>>> Because modern hardware does not slap us with the same cost for >>>>> accessing memory as it once did, the benefit of replacing memory >>>>> access with register access is far less dramatic now than it once was. >>>>> Additionally, the interpreter runs for a relatively short time before >>>>> relevant code sections are compiled. When the VM starts running >>>>> compiled code instead of interpreted code, performance should begin to >>>>> move asymptotically towards that of compiled code, diluting any >>>>> performance penalties from the interpreter to small performance >>>>> variations. >>>>> >>>>> No Measurable Benefit. >>>>> Please see the results files attached in the bug page. This change >>>>> was adapted for x86 and sparc, and interpreter performance was >>>>> measured with Specjvm98 (run with -Xint). No significant decrease in >>>>> performance was observed. >>>>> >>>>> Memory footprint and code complexity. >>>>> Stack caching in the JVM is implemented by switching the instruction >>>>> look-up table depending on the tos (top-of-stack) state. At any moment >>>>> there are is an active table consisting of one dispatch table for each >>>>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>>>> safepoint dispatch tables into the active table. The additional entry >>>>> code makes this copy less efficient and makes any work in the >>>>> interpreter harder to debug. >>>>> >>>>> If we remove this optimization, we will: >>>>> - decrease memory usage in the interpreter, >>>>> - eliminated wasteful memory transactions during safepoints, >>>>> - decrease code complexity (a lot). >>>>> >>>>> Please let me know what you think. >>>>> Thanks, >>>>> Max >>>>> >>>> >> > From coleen.phillimore at oracle.com Sun Feb 19 22:11:11 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sun, 19 Feb 2017 17:11:11 -0500 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> Message-ID: <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> On 2/18/17 11:14 AM, coleen.phillimore at oracle.com wrote: > When Max gets back from the long weekend, he'll post the platforms in > your bug. > > It's amazing that for -Xint there's no significant difference. I've > seen -Xint performance of 15% slower cause a 2% slowdown with server > but that was before tiered compilation. I should clarify this. I've seen this slowdown for *different* interpreter optimizations, which *can* affect server performance. I was measuring specjvm98 on linux x64. If there's no significant difference for this TOS optimization, there is no chance of a degredation in overall performance. Coleen > > The reason for this query was to see what developers for the other > platform ports think, since this change would affect all of the > platforms. > > Thanks, > Coleen > > On 2/18/17 10:50 AM, Daniel D. Daugherty wrote: >> If Claes is happy with the perf testing, then I'm happy. :-) >> >> Dan >> >> >> On 2/18/17 3:46 AM, Claes Redestad wrote: >>> Hi, >>> >>> I've seen Max has run plenty of tests on our internal performance >>> infrastructure and everything I've seen there seems to corroborate the >>> idea that this removal is OK from a performance point of view, the >>> footprint improvements are small but significant and any negative >>> performance impact on throughput benchmarks is at noise levels even >>> with -Xint (it appears many benchmarks time out with this setting >>> both before and after, though; Max, let's discuss offline how to >>> deal with that :-)) >>> >>> I expect this will be tested more thoroughly once adapted to all >>> platforms (which I assume is the intent?), but see no concern from >>> a performance testing point of view: Do it! >>> >>> Thanks! >>> >>> /Claes >>> >>> On 2017-02-16 16:40, Daniel D. Daugherty wrote: >>>> Hi Max, >>>> >>>> Added a note to your bug. Interesting idea, but I think your data is >>>> a bit incomplete at the moment. >>>> >>>> Dan >>>> >>>> >>>> On 2/15/17 3:18 PM, Max Ockner wrote: >>>>> Hello all, >>>>> >>>>> We have filed a bug to remove the interpreter stack caching >>>>> optimization for jdk10. Ideally we can make this change *early* >>>>> during the jdk10 development cycle. See below for justification: >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>>>> >>>>> Stack caching has been around for a long time and is intended to >>>>> replace some of the load/store (pop/push) operations with >>>>> corresponding register operations. The need for this optimization >>>>> arose before caching could adequately lessen the burden of memory >>>>> access. We have reevaluated the JVM stack caching optimization and >>>>> have found that it has a high memory footprint and is very costly to >>>>> maintain, but does not provide significant measurable or theoretical >>>>> benefit for us when used with modern hardware. >>>>> >>>>> Minimal Theoretical Benefit. >>>>> Because modern hardware does not slap us with the same cost for >>>>> accessing memory as it once did, the benefit of replacing memory >>>>> access with register access is far less dramatic now than it once >>>>> was. >>>>> Additionally, the interpreter runs for a relatively short time before >>>>> relevant code sections are compiled. When the VM starts running >>>>> compiled code instead of interpreted code, performance should >>>>> begin to >>>>> move asymptotically towards that of compiled code, diluting any >>>>> performance penalties from the interpreter to small performance >>>>> variations. >>>>> >>>>> No Measurable Benefit. >>>>> Please see the results files attached in the bug page. This change >>>>> was adapted for x86 and sparc, and interpreter performance was >>>>> measured with Specjvm98 (run with -Xint). No significant decrease in >>>>> performance was observed. >>>>> >>>>> Memory footprint and code complexity. >>>>> Stack caching in the JVM is implemented by switching the instruction >>>>> look-up table depending on the tos (top-of-stack) state. At any >>>>> moment >>>>> there are is an active table consisting of one dispatch table for >>>>> each >>>>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>>>> safepoint dispatch tables into the active table. The additional >>>>> entry >>>>> code makes this copy less efficient and makes any work in the >>>>> interpreter harder to debug. >>>>> >>>>> If we remove this optimization, we will: >>>>> - decrease memory usage in the interpreter, >>>>> - eliminated wasteful memory transactions during safepoints, >>>>> - decrease code complexity (a lot). >>>>> >>>>> Please let me know what you think. >>>>> Thanks, >>>>> Max >>>>> >>>> >> > From volker.simonis at gmail.com Mon Feb 20 10:45:10 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 20 Feb 2017 11:45:10 +0100 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> Message-ID: Hi, besides the fact that this of course means some work for us :) I currently don't see any problems for our porting platforms (ppc64 and s390x). Are there any webrevs available, so we can see how big they are and maybe do some own benchmarking? Thanks, Volker On Sun, Feb 19, 2017 at 11:11 PM, wrote: > > > On 2/18/17 11:14 AM, coleen.phillimore at oracle.com wrote: >> >> When Max gets back from the long weekend, he'll post the platforms in your >> bug. >> >> It's amazing that for -Xint there's no significant difference. I've seen >> -Xint performance of 15% slower cause a 2% slowdown with server but that was >> before tiered compilation. > > > I should clarify this. I've seen this slowdown for *different* interpreter > optimizations, which *can* affect server performance. I was measuring > specjvm98 on linux x64. If there's no significant difference for this TOS > optimization, there is no chance of a degredation in overall performance. > > Coleen > >> >> The reason for this query was to see what developers for the other >> platform ports think, since this change would affect all of the platforms. >> >> Thanks, >> Coleen >> >> On 2/18/17 10:50 AM, Daniel D. Daugherty wrote: >>> >>> If Claes is happy with the perf testing, then I'm happy. :-) >>> >>> Dan >>> >>> >>> On 2/18/17 3:46 AM, Claes Redestad wrote: >>>> >>>> Hi, >>>> >>>> I've seen Max has run plenty of tests on our internal performance >>>> infrastructure and everything I've seen there seems to corroborate the >>>> idea that this removal is OK from a performance point of view, the >>>> footprint improvements are small but significant and any negative >>>> performance impact on throughput benchmarks is at noise levels even >>>> with -Xint (it appears many benchmarks time out with this setting >>>> both before and after, though; Max, let's discuss offline how to >>>> deal with that :-)) >>>> >>>> I expect this will be tested more thoroughly once adapted to all >>>> platforms (which I assume is the intent?), but see no concern from >>>> a performance testing point of view: Do it! >>>> >>>> Thanks! >>>> >>>> /Claes >>>> >>>> On 2017-02-16 16:40, Daniel D. Daugherty wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> Added a note to your bug. Interesting idea, but I think your data is >>>>> a bit incomplete at the moment. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 2/15/17 3:18 PM, Max Ockner wrote: >>>>>> >>>>>> Hello all, >>>>>> >>>>>> We have filed a bug to remove the interpreter stack caching >>>>>> optimization for jdk10. Ideally we can make this change *early* >>>>>> during the jdk10 development cycle. See below for justification: >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>>>>> >>>>>> Stack caching has been around for a long time and is intended to >>>>>> replace some of the load/store (pop/push) operations with >>>>>> corresponding register operations. The need for this optimization >>>>>> arose before caching could adequately lessen the burden of memory >>>>>> access. We have reevaluated the JVM stack caching optimization and >>>>>> have found that it has a high memory footprint and is very costly to >>>>>> maintain, but does not provide significant measurable or theoretical >>>>>> benefit for us when used with modern hardware. >>>>>> >>>>>> Minimal Theoretical Benefit. >>>>>> Because modern hardware does not slap us with the same cost for >>>>>> accessing memory as it once did, the benefit of replacing memory >>>>>> access with register access is far less dramatic now than it once was. >>>>>> Additionally, the interpreter runs for a relatively short time before >>>>>> relevant code sections are compiled. When the VM starts running >>>>>> compiled code instead of interpreted code, performance should begin to >>>>>> move asymptotically towards that of compiled code, diluting any >>>>>> performance penalties from the interpreter to small performance >>>>>> variations. >>>>>> >>>>>> No Measurable Benefit. >>>>>> Please see the results files attached in the bug page. This change >>>>>> was adapted for x86 and sparc, and interpreter performance was >>>>>> measured with Specjvm98 (run with -Xint). No significant decrease in >>>>>> performance was observed. >>>>>> >>>>>> Memory footprint and code complexity. >>>>>> Stack caching in the JVM is implemented by switching the instruction >>>>>> look-up table depending on the tos (top-of-stack) state. At any moment >>>>>> there are is an active table consisting of one dispatch table for each >>>>>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>>>>> safepoint dispatch tables into the active table. The additional entry >>>>>> code makes this copy less efficient and makes any work in the >>>>>> interpreter harder to debug. >>>>>> >>>>>> If we remove this optimization, we will: >>>>>> - decrease memory usage in the interpreter, >>>>>> - eliminated wasteful memory transactions during safepoints, >>>>>> - decrease code complexity (a lot). >>>>>> >>>>>> Please let me know what you think. >>>>>> Thanks, >>>>>> Max >>>>>> >>>>> >>> >> > From erik.helin at oracle.com Mon Feb 20 14:06:35 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 20 Feb 2017 15:06:35 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <2FC6A0BB-7125-43FC-9E3D-5CB67426AE9B@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> <2FC6A0BB-7125-43FC-9E3D-5CB67426AE9B@oracle.com> Message-ID: On 02/16/2017 07:51 PM, Kim Barrett wrote: > src/share/vm/classfile/classLoaderData.hpp > 179 // Only one thread can add a time, guarded by the Metaspace_lock. > > Only one thread at a time time can add? Yes. See ClassLoaderData::add_handle, all the threads synchronize using ClassLoaderData::metaspace_lock(). Thanks, Erik >> On Feb 16, 2017, at 9:13 AM, Erik Helin wrote: >> >> On 02/15/2017 10:08 PM, coleen.phillimore at oracle.com wrote: >>> Hi Erik, >> >> Hey Coleen, thanks for reviewing! Please see new webrevs at: >> - incremental: http://cr.openjdk.java.net/~ehelin/8168914/01-02/ >> - full: http://cr.openjdk.java.net/~ehelin/8168914/02/ >> > From kim.barrett at oracle.com Mon Feb 20 14:57:44 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 Feb 2017 09:57:44 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> <2FC6A0BB-7125-43FC-9E3D-5CB67426AE9B@oracle.com> Message-ID: > On Feb 20, 2017, at 9:06 AM, Erik Helin wrote: > > On 02/16/2017 07:51 PM, Kim Barrett wrote: >> src/share/vm/classfile/classLoaderData.hpp >> 179 // Only one thread can add a time, guarded by the Metaspace_lock. >> >> Only one thread at a time time can add? > > Yes. See ClassLoaderData::add_handle, all the threads synchronize using > ClassLoaderData::metaspace_lock(). > > Thanks, > Erik Sorry for the confusing review comment. That was a suggested rewording to fix gammer in the quoted text, rather than questioning whether it was really mutex protected. From erik.helin at oracle.com Mon Feb 20 15:25:21 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 20 Feb 2017 16:25:21 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> <2FC6A0BB-7125-43FC-9E3D-5CB67426AE9B@oracle.com> Message-ID: <65e2208d-2d3f-3d89-1537-98cfca4b095c@oracle.com> On 02/20/2017 03:57 PM, Kim Barrett wrote: >> On Feb 20, 2017, at 9:06 AM, Erik Helin wrote: >> >> On 02/16/2017 07:51 PM, Kim Barrett wrote: >>> src/share/vm/classfile/classLoaderData.hpp >>> 179 // Only one thread can add a time, guarded by the Metaspace_lock. >>> >>> Only one thread at a time time can add? >> >> Yes. See ClassLoaderData::add_handle, all the threads synchronize using >> ClassLoaderData::metaspace_lock(). >> >> Thanks, >> Erik > > Sorry for the confusing review comment. That was a suggested rewording to fix gammer in the quoted text, rather than questioning whether it was really mutex protected. Haha :D Thanks for the comment, I'll update it in the next version! Erik From erik.helin at oracle.com Mon Feb 20 15:47:38 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 20 Feb 2017 16:47:38 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> Message-ID: Ok, lets see if we can wrap this up :) I just uploaded a new version, 03: - incremental: http://cr.openjdk.java.net/~ehelin/8168914/02-03/ - full: http://cr.openjdk.java.net/~ehelin/8168914/03/ The following changes have been made to the original patch: 00 -> 01: - Updated copyright year - Fixed typo in comment - Updated stale comment - Removed unnecessary load_aquire when loading c->next 01 -> 02: - ChunkedHandleList now derives ValueObj instead of CHeapObj - Both ChunkedHandleList and Chunk are now classes instead of structs - Changed capacity of chunk to 32 from 64 (same capacity as used by JNIHandleBlock) 02 -> 03: - Only use load_acquire when reading the size for the "head" chunk - Fix grammar in comment - Refer to the correct lock in comment AFAICS (scanning through all the emails) there are now only three remaining comments: - Thomas: can we assert in ChunkedHandeList::add that the correct mutex is locked? - Then we would have to pass the mutex along as a parameter to `add`, which to me seems unnecessary. Then ChunkedHandleList::add might just as well lock the mutex instead of asserting. But thanks for highlight this, I noticed I referred to the wrong lock in the comment :) - Coleen: can we remove the word unsafe from ClassLoaderData::remove_handle_unsafe? - I'd like to keep it until we design a better remove_handle function. There is only one user of remove_handle_unsafe, so changing the name in the future should be easy. - Kim: can we have something like `convert_x_to_y` instead of *((oop*) op) = NULL? - I would like to defer that to another patch, but I agree that it is rather nasty. Mostly because I don't know what the "right" API should look like, since this is the first time in run into this problem. Thanks, Erik On 02/14/2017 05:40 PM, Erik Helin wrote: > Hi all, > > this patch aims to solve the bug [0] by making it safe for a GC to > concurrently traverse the oops in a ClassLoaderData. > > The problem right now is that ClassLoaderData::_handles is a > JNIHandleBlock. It is not safe for one thread to iterate over the oops > in a JNIHandleBlock while another thread concurrently adds a new oop to > the JNIHandleBlock. > > My proposed solution is to replace JNIHandleBlock with another data > structure for ClassLoaderData. ClassLoaderData only needs a place to > store oops that a GC can iterate over, it has no use for the rest of the > methods in the JNIHandleBlock class. I have implemented a minimal > chunked linked list data structure (internal to ClassLoaderData) with > only two public methods: > - add (only executed by one thread at a time) > - oops_do (can be executed by any number of threads, possibly > concurrently with a call to `add`) > > ChunkedHandleList uses load_acquire and release_store to synchronize > access to the underlying chunks (arrays). Since ChunkedHandleList > utilizes the (rather) minimal requirements of its user > (ClassLoaderData), I decided to keep the class internal to > ClassLoaderData for now. If other find it useful elsewhere, the we can > try to create a more generalized version (this is trickier than it > appears at first sight, I tried ;)). > > I also changed ClassLoaderData::remove_handle to write NULL to the oop* > (that is represented by a jobject), instead of using a sentinel oop as > done by JNIHandleBlock. The GC closures should be prepared to handle a > field in a Java object becoming NULL, so this should be fine. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8168914 > > Webrev: > http://cr.openjdk.java.net/~ehelin/8168914/00/ > > Testing: > - Tier 1 (aka JPRT), 2, 3, 4, 5. > > I would appreciate quite a few reviews on this patch, it contains a nice > mixture of: > - me venturing into the runtime code :) > - lock free code > - unreproducible bug (even though I'm sure of the root cause) > > Thanks for everyone participating in the discussions around this bug and > potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and Jiangli. > > Thanks! > Erik > > [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From erik.helin at oracle.com Mon Feb 20 15:48:13 2017 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 20 Feb 2017 16:48:13 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <4b282796-cd3d-5dfa-08ee-e3a92edd88f8@oracle.com> <6271228b-e754-70dd-12b7-aab1f19f7f25@oracle.com> Message-ID: On 02/16/2017 07:14 PM, coleen.phillimore at oracle.com wrote: > On 2/16/17 9:13 AM, Erik Helin wrote: >> On 02/15/2017 10:08 PM, coleen.phillimore at oracle.com wrote: >>> Hi Erik, >> >> Hey Coleen, thanks for reviewing! Please see new webrevs at: >> - incremental: http://cr.openjdk.java.net/~ehelin/8168914/01-02/ >> - full: http://cr.openjdk.java.net/~ehelin/8168914/02/ >> >>> I thought you were going to hide the declaration of class >>> ChunkedHandleList in classLoaderData.cpp since it is only used >>> internally? >>> >>> And have a field in ClassLoaderData: ChunkedHandleList* >>> _chunked_handle_list; >> >> I didn't want to have a pointer to ChunkedHandleList just because I >> want to "hide" the implementation, I prefer to embed an instance (even >> thought that means I have to declare the struct/class in the .hpp file). > > My preference is different but this is fine. >> >>> The class name "Chunk" is so overused! >> >> Haha, I agree that it is quite commonly used :) Fortunately there is >> some context here: Chunk is an inner class/struct to >> ChunkedHandleList. That hopefully gives the reader enough insight to >> distinguish this Chunk from other Chunks in the VM. >> >>> Here ChunkedHandleList should not be a subtype of CHeapObj. >>> It's a ValueObj. >> >> Agree, this is a left-over from an earlier state of the patch. Thanks >> for noticing! >> >>> I don't think it should be a struct either because even though it's >>> internal to ClassLoaderData, the fields could still be accessed outside >>> the class. >> >> Agree, I cleaned that up a bit. Thanks. >> >>> Why is >>> >>> 629 void ClassLoaderData::remove_handle_unsafe(jobject h) { >>> >>> >>> it called unsafe? It is safe to zero the element in the block, isn't >>> it? >> >> Because code using remove_handle_unsafe should be cautious. It is only >> safe to write NULL to the *((oop*) op) if: >> - the jobject originates from a call to ClassLoaderData::add_handle >> (the assert verifies this). >> - *op is the sole remaining pointer to **op (the object). >> - The native memory backing the chunked linked list is not freed. >> >> It is unfortunate that we even need to have the remove_handle_unsafe >> at all. An alternative is to let the only code using >> remove_handle_unsafe (ModuleEntry) do *((oop*) op) = NULL on its own. >> >> Would you prefer that? >> > > I answered this in my last mail. > >>> One other reason to make it internal is that the CAPACITY of 64 is way >>> too big for the ClassLoaderData of anonymous classes which I think will >>> only have presently 1 (or 2 maybe) oops in it. And there are zillions >>> of them. >> >> Oops, thanks for catching this. I intended to use the same capacity as >> JNIHandleBlock, which uses 32 (don't recall where that 64 came from). >> As for using a smaller size, that seems to be a separate patch? For >> now I would prefer to keep the characteristics the same as for >> JNIHandleBlock. > > If the whole ChunkHandleList class was internal, then you could pick 10 > or something. I guess we'll see if anyone notices footprint and fix it > then. JNIHandleBlock was 32 so this won't increase footprint. I don't follow, ChunkedHandleList is private inner class to ClassLoaderData. Why is this less internal than having the declaration in the .cpp file? That ChunkedHandleList is declared in the classLoaderData.hpp shouldn't make it harder to change the field CAPACITY to 10. Or am I missing something? > Thanks - the new webrev looks good. Thanks. I uploaded a new version (see new email). Thanks, Erik > Coleen >> >> >> Thanks, >> Erik >> >>> Thanks, >>> Coleen >>> >>> >>> On 2/14/17 11:40 AM, Erik Helin wrote: >>>> Hi all, >>>> >>>> this patch aims to solve the bug [0] by making it safe for a GC to >>>> concurrently traverse the oops in a ClassLoaderData. >>>> >>>> The problem right now is that ClassLoaderData::_handles is a >>>> JNIHandleBlock. It is not safe for one thread to iterate over the oops >>>> in a JNIHandleBlock while another thread concurrently adds a new oop >>>> to the JNIHandleBlock. >>>> >>>> My proposed solution is to replace JNIHandleBlock with another data >>>> structure for ClassLoaderData. ClassLoaderData only needs a place to >>>> store oops that a GC can iterate over, it has no use for the rest of >>>> the methods in the JNIHandleBlock class. I have implemented a minimal >>>> chunked linked list data structure (internal to ClassLoaderData) with >>>> only two public methods: >>>> - add (only executed by one thread at a time) >>>> - oops_do (can be executed by any number of threads, possibly >>>> concurrently with a call to `add`) >>>> >>>> ChunkedHandleList uses load_acquire and release_store to synchronize >>>> access to the underlying chunks (arrays). Since ChunkedHandleList >>>> utilizes the (rather) minimal requirements of its user >>>> (ClassLoaderData), I decided to keep the class internal to >>>> ClassLoaderData for now. If other find it useful elsewhere, the we can >>>> try to create a more generalized version (this is trickier than it >>>> appears at first sight, I tried ;)). >>>> >>>> I also changed ClassLoaderData::remove_handle to write NULL to the >>>> oop* (that is represented by a jobject), instead of using a sentinel >>>> oop as done by JNIHandleBlock. The GC closures should be prepared to >>>> handle a field in a Java object becoming NULL, so this should be fine. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8168914 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~ehelin/8168914/00/ >>>> >>>> Testing: >>>> - Tier 1 (aka JPRT), 2, 3, 4, 5. >>>> >>>> I would appreciate quite a few reviews on this patch, it contains a >>>> nice mixture of: >>>> - me venturing into the runtime code :) >>>> - lock free code >>>> - unreproducible bug (even though I'm sure of the root cause) >>>> >>>> Thanks for everyone participating in the discussions around this bug >>>> and potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and >>>> Jiangli. >>>> >>>> Thanks! >>>> Erik >>>> >>>> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 >>> > From kim.barrett at oracle.com Mon Feb 20 18:56:57 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 Feb 2017 13:56:57 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> Message-ID: > On Feb 20, 2017, at 10:47 AM, Erik Helin wrote: > > Ok, lets see if we can wrap this up :) I just uploaded a new version, 03: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/02-03/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/03/ ------------------------------------------------------------------------------ src/share/vm/classfile/classLoaderData.cpp 136 Chunk* volatile head = (Chunk*) OrderAccess::load_ptr_acquire((volatile intptr_t*)&_head); The "volatile" in the declaration of head is unnecessary/misplaced. It's not the variable "head" is modifiable by other threads. Correct would be "Chunk volatile * head", but then the volatile would need to be const_casted away when initializing "c". Better to just drop it, as the load_acquire provides all the needed ordering. ------------------------------------------------------------------------------ src/share/vm/classfile/classLoaderData.cpp 136 Chunk* volatile head = (Chunk*) OrderAccess::load_ptr_acquire((volatile intptr_t*)&_head); Why the cast of &_head for the load_ptr_acquire? Similarly, why the casts (both of them) here: 127 OrderAccess::release_store_ptr((volatile intptr_t*) &_head, (intptr_t) next); ------------------------------------------------------------------------------ From coleen.phillimore at oracle.com Mon Feb 20 22:29:59 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 20 Feb 2017 17:29:59 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> Message-ID: On 2/20/17 10:47 AM, Erik Helin wrote: > Ok, lets see if we can wrap this up :) I just uploaded a new version, 03: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/02-03/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/03/ > > The following changes have been made to the original patch: > > 00 -> 01: > - Updated copyright year > - Fixed typo in comment > - Updated stale comment > - Removed unnecessary load_aquire when loading c->next > > 01 -> 02: > - ChunkedHandleList now derives ValueObj instead of CHeapObj > - Both ChunkedHandleList and Chunk are now classes instead of structs > - Changed capacity of chunk to 32 from 64 (same capacity as used by > JNIHandleBlock) > > 02 -> 03: > - Only use load_acquire when reading the size for the "head" chunk > - Fix grammar in comment > - Refer to the correct lock in comment > > AFAICS (scanning through all the emails) there are now only three > remaining comments: > - Thomas: can we assert in ChunkedHandeList::add that the correct mutex > is locked? > - Then we would have to pass the mutex along as a parameter to `add`, > which to me seems unnecessary. Then ChunkedHandleList::add might just > as well lock the mutex instead of asserting. But thanks for > highlight this, I noticed I referred to the wrong lock in the comment > :) > > - Coleen: can we remove the word unsafe from > ClassLoaderData::remove_handle_unsafe? > - I'd like to keep it until we design a better remove_handle function. > There is only one user of remove_handle_unsafe, so changing the name > in the future should be easy. ok. Coleen > > - Kim: can we have something like `convert_x_to_y` instead of > *((oop*) op) = NULL? > - I would like to defer that to another patch, but I agree that it is > rather nasty. Mostly because I don't know what the "right" API should > look like, since this is the first time in run into this problem. > > Thanks, > Erik > > On 02/14/2017 05:40 PM, Erik Helin wrote: >> Hi all, >> >> this patch aims to solve the bug [0] by making it safe for a GC to >> concurrently traverse the oops in a ClassLoaderData. >> >> The problem right now is that ClassLoaderData::_handles is a >> JNIHandleBlock. It is not safe for one thread to iterate over the oops >> in a JNIHandleBlock while another thread concurrently adds a new oop to >> the JNIHandleBlock. >> >> My proposed solution is to replace JNIHandleBlock with another data >> structure for ClassLoaderData. ClassLoaderData only needs a place to >> store oops that a GC can iterate over, it has no use for the rest of the >> methods in the JNIHandleBlock class. I have implemented a minimal >> chunked linked list data structure (internal to ClassLoaderData) with >> only two public methods: >> - add (only executed by one thread at a time) >> - oops_do (can be executed by any number of threads, possibly >> concurrently with a call to `add`) >> >> ChunkedHandleList uses load_acquire and release_store to synchronize >> access to the underlying chunks (arrays). Since ChunkedHandleList >> utilizes the (rather) minimal requirements of its user >> (ClassLoaderData), I decided to keep the class internal to >> ClassLoaderData for now. If other find it useful elsewhere, the we can >> try to create a more generalized version (this is trickier than it >> appears at first sight, I tried ;)). >> >> I also changed ClassLoaderData::remove_handle to write NULL to the oop* >> (that is represented by a jobject), instead of using a sentinel oop as >> done by JNIHandleBlock. The GC closures should be prepared to handle a >> field in a Java object becoming NULL, so this should be fine. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8168914 >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8168914/00/ >> >> Testing: >> - Tier 1 (aka JPRT), 2, 3, 4, 5. >> >> I would appreciate quite a few reviews on this patch, it contains a nice >> mixture of: >> - me venturing into the runtime code :) >> - lock free code >> - unreproducible bug (even though I'm sure of the root cause) >> >> Thanks for everyone participating in the discussions around this bug and >> potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and Jiangli. >> >> Thanks! >> Erik >> >> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From david.holmes at oracle.com Mon Feb 20 22:52:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 21 Feb 2017 08:52:14 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> Message-ID: <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> Hi Erik, On 21/02/2017 1:47 AM, Erik Helin wrote: > Ok, lets see if we can wrap this up :) I just uploaded a new version, 03: > - incremental: http://cr.openjdk.java.net/~ehelin/8168914/02-03/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/03/ > > The following changes have been made to the original patch: > > 00 -> 01: > - Updated copyright year > - Fixed typo in comment > - Updated stale comment > - Removed unnecessary load_aquire when loading c->next > > 01 -> 02: > - ChunkedHandleList now derives ValueObj instead of CHeapObj > - Both ChunkedHandleList and Chunk are now classes instead of structs > - Changed capacity of chunk to 32 from 64 (same capacity as used by > JNIHandleBlock) > > 02 -> 03: > - Only use load_acquire when reading the size for the "head" chunk Sorry but I don't like the way this is done - the conditional may end up being more expensive than the unnecessary load-acquire. Unrolling the first loop iteration, as per the email discussion, is a better way to go IMO. And as Kim indicated you shouldn't need any casts for the OrderAccess ops as you should be using the "void*" versions. > - Fix grammar in comment > - Refer to the correct lock in comment > > AFAICS (scanning through all the emails) there are now only three > remaining comments: > - Thomas: can we assert in ChunkedHandeList::add that the correct mutex > is locked? > - Then we would have to pass the mutex along as a parameter to `add`, Why do you need to pass it in? oop* ClassLoaderData::ChunkedHandleList::add(oop o) { assert(ClassLoaderData::metaspace_lock()->owned_by_self(), "should own lock"); Thanks, David > which to me seems unnecessary. Then ChunkedHandleList::add might just > as well lock the mutex instead of asserting. But thanks for > highlight this, I noticed I referred to the wrong lock in the comment > :) > > - Coleen: can we remove the word unsafe from > ClassLoaderData::remove_handle_unsafe? > - I'd like to keep it until we design a better remove_handle function. > There is only one user of remove_handle_unsafe, so changing the name > in the future should be easy. > > - Kim: can we have something like `convert_x_to_y` instead of > *((oop*) op) = NULL? > - I would like to defer that to another patch, but I agree that it is > rather nasty. Mostly because I don't know what the "right" API should > look like, since this is the first time in run into this problem. > > Thanks, > Erik > > On 02/14/2017 05:40 PM, Erik Helin wrote: >> Hi all, >> >> this patch aims to solve the bug [0] by making it safe for a GC to >> concurrently traverse the oops in a ClassLoaderData. >> >> The problem right now is that ClassLoaderData::_handles is a >> JNIHandleBlock. It is not safe for one thread to iterate over the oops >> in a JNIHandleBlock while another thread concurrently adds a new oop to >> the JNIHandleBlock. >> >> My proposed solution is to replace JNIHandleBlock with another data >> structure for ClassLoaderData. ClassLoaderData only needs a place to >> store oops that a GC can iterate over, it has no use for the rest of the >> methods in the JNIHandleBlock class. I have implemented a minimal >> chunked linked list data structure (internal to ClassLoaderData) with >> only two public methods: >> - add (only executed by one thread at a time) >> - oops_do (can be executed by any number of threads, possibly >> concurrently with a call to `add`) >> >> ChunkedHandleList uses load_acquire and release_store to synchronize >> access to the underlying chunks (arrays). Since ChunkedHandleList >> utilizes the (rather) minimal requirements of its user >> (ClassLoaderData), I decided to keep the class internal to >> ClassLoaderData for now. If other find it useful elsewhere, the we can >> try to create a more generalized version (this is trickier than it >> appears at first sight, I tried ;)). >> >> I also changed ClassLoaderData::remove_handle to write NULL to the oop* >> (that is represented by a jobject), instead of using a sentinel oop as >> done by JNIHandleBlock. The GC closures should be prepared to handle a >> field in a Java object becoming NULL, so this should be fine. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8168914 >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8168914/00/ >> >> Testing: >> - Tier 1 (aka JPRT), 2, 3, 4, 5. >> >> I would appreciate quite a few reviews on this patch, it contains a nice >> mixture of: >> - me venturing into the runtime code :) >> - lock free code >> - unreproducible bug (even though I'm sure of the root cause) >> >> Thanks for everyone participating in the discussions around this bug and >> potential solutions: Volker, Coleen, Mikael G, StefanK, Erik ? and Jiangli. >> >> Thanks! >> Erik >> >> [0]: https://bugs.openjdk.java.net/browse/JDK-8168914 From kim.barrett at oracle.com Tue Feb 21 01:57:01 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 Feb 2017 20:57:01 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> Message-ID: <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> > On Feb 20, 2017, at 5:52 PM, David Holmes wrote: >> 02 -> 03: >> - Only use load_acquire when reading the size for the "head" chunk > > Sorry but I don't like the way this is done - the conditional may end up being more expensive than the unnecessary load-acquire. Unrolling the first loop iteration, as per the email discussion, is a better way to go IMO. +1 > And as Kim indicated you shouldn't need any casts for the OrderAccess ops as you should be using the "void*" versions. > >> - Fix grammar in comment >> - Refer to the correct lock in comment >> >> AFAICS (scanning through all the emails) there are now only three >> remaining comments: >> - Thomas: can we assert in ChunkedHandeList::add that the correct mutex >> is locked? >> - Then we would have to pass the mutex along as a parameter to `add`, > > Why do you need to pass it in? > > oop* ClassLoaderData::ChunkedHandleList::add(oop o) { > assert(ClassLoaderData::metaspace_lock()->owned_by_self(), "should own lock"); The metaspace_lock() is per-CLD, so that doesn?t work. From david.holmes at oracle.com Tue Feb 21 02:05:09 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 21 Feb 2017 12:05:09 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> Message-ID: On 21/02/2017 11:57 AM, Kim Barrett wrote: >> On Feb 20, 2017, at 5:52 PM, David Holmes wrote: >>> 02 -> 03: >>> - Only use load_acquire when reading the size for the "head" chunk >> >> Sorry but I don't like the way this is done - the conditional may end up being more expensive than the unnecessary load-acquire. Unrolling the first loop iteration, as per the email discussion, is a better way to go IMO. > > +1 > >> And as Kim indicated you shouldn't need any casts for the OrderAccess ops as you should be using the "void*" versions. >> >>> - Fix grammar in comment >>> - Refer to the correct lock in comment >>> >>> AFAICS (scanning through all the emails) there are now only three >>> remaining comments: >>> - Thomas: can we assert in ChunkedHandeList::add that the correct mutex >>> is locked? >>> - Then we would have to pass the mutex along as a parameter to `add`, >> >> Why do you need to pass it in? >> >> oop* ClassLoaderData::ChunkedHandleList::add(oop o) { >> assert(ClassLoaderData::metaspace_lock()->owned_by_self(), "should own lock"); > > The metaspace_lock() is per-CLD, so that doesn?t work. Ah! The use of ClassLoaderData::metaspace_lock() in the comment threw me. So we could have the ChunkHandleList store a back-pointer to the CLD that owns it. But for a single method with a single caller this seems like overkill. BTW not sure about nested classes in C++ but does anything in ChunkHandleList need to be public? Thanks, David From kim.barrett at oracle.com Tue Feb 21 02:26:25 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 Feb 2017 21:26:25 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> Message-ID: <19F2A902-2B23-45CC-BC13-029D720F030E@oracle.com> > On Feb 20, 2017, at 9:05 PM, David Holmes wrote: > BTW not sure about nested classes in C++ but does anything in ChunkHandleList need to be public? Yes. The outer class has no special access rights on the inner class. For C++03 the inner class has no special access rights on the outer class either. That was fixed in C++11. From vladimir.x.ivanov at oracle.com Tue Feb 21 14:36:26 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 21 Feb 2017 17:36:26 +0300 Subject: [10] RFC: Relaxing access restrictions for VM annotations Message-ID: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> FYI I've been experimenting with relaxing restrictions on VM annotations - Java annotations which affect how VM work - @ReservedStackAccess, @Contended, @Stable, etc (most of them are located in jdk.internal.vm.annotation in 9). Right now, VM ignores any annotation unless it is declared on a class from privileged context (JDK classes or VM anonymous classes). Some features introduced adhoc flags to relax the restriction (e.g., -XX:RestrictContended & -XX:RestrictReservedStack), but with modules there's a universal way to expose them in controlled manner: rely on annotation class accessibility checks. In order to make some annotation available in non-privileged context, it is enough to add relevant export to java.base module (e.g., --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED). The problem I faced was that accessibility checks are performed on already loaded classes, but annotations are used during class parsing and the checks have to be performed against the class being loaded. One option is to re-implement accessibility checking logic and extract all necessary information from ClassFileParser. Another approach is to collect VM annotations during parsing and after the class is loaded apply them one-by-one doing accessibility checks. It works fine for most of the existing annotations, but problematic for @Contended - it affects instance layout which is computed during parsing. So, after some experiments, I decided to try different approach: optimistically assume all annotations are accessible and re-parse the class if any used annotation is not available: http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.00 The benefit is that parsing & accessibility checking logic stay the same. Also, it enabled sanity checking logic for annotation usages almost for free. Overall, I'm quite satisfied with the result, but stumbled upon some nuisance when implementing re-parsing (e.g., tracing framework needs access to ClassFileParser and it brings unnecessary changes into ClassFileParser::parse_instance_klass_from_stream). Before investing into polishing the change, I'd like to discuss the approach I chose. Any problems with class file re-parsing or annotation resolution in VM I overlooked? Maybe there are strong arguments for other options (#1: accessibity checks in ClassFileParser & #2: apply annotations after the class is loaded)? Thanks! Best regards, Vladimir Ivanov From lutz.schmidt at sap.com Tue Feb 21 15:38:25 2017 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 21 Feb 2017 15:38:25 +0000 Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" Message-ID: Hi all, May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. Bug: https://bugs.openjdk.java.net/browse/JDK-8175267 Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175267/ Description: The "handler_for_unsafe_access? is a relic from former times, having survived only on s390x. The stub is generated, but never used (called). This change removes everything related. Thanks, Lutz From erik.helin at oracle.com Tue Feb 21 15:49:14 2017 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 21 Feb 2017 16:49:14 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> Message-ID: <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> On 02/21/2017 02:57 AM, Kim Barrett wrote: >> On Feb 20, 2017, at 5:52 PM, David Holmes wrote: >>> 02 -> 03: >>> - Only use load_acquire when reading the size for the "head" chunk >> >> Sorry but I don't like the way this is done - the conditional may end up being more expensive than the unnecessary load-acquire. Unrolling the first loop iteration, as per the email discussion, is a better way to go IMO. > > +1 David, Kim, please see new patches at: - inc: http://cr.openjdk.java.net/~ehelin/8168914/03-04/ - full: http://cr.openjdk.java.net/~ehelin/8168914/04/ Thanks, Erik From kim.barrett at oracle.com Tue Feb 21 16:19:10 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 21 Feb 2017 11:19:10 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> Message-ID: > On Feb 21, 2017, at 10:49 AM, Erik Helin wrote: > > David, Kim, please see new patches at: > - inc: http://cr.openjdk.java.net/~ehelin/8168914/03-04/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/04/ > > Thanks, > Erik ------------------------------------------------------------------------------ src/share/vm/classfile/classLoaderData.cpp 144 Chunk* head = (Chunk*) OrderAccess::load_ptr_acquire((volatile intptr_t*)&_head); Still has unnecessary cast of &_head, ------------------------------------------------------------------------------ src/share/vm/classfile/classLoaderData.hpp 186 void oops_do_chunk(OopClosure* f, Chunk* c, const juint size); oops_do_chunk ahould be private. ------------------------------------------------------------------------------ I don't need a new webrev for these. From lutz.schmidt at sap.com Tue Feb 21 16:46:26 2017 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 21 Feb 2017 16:46:26 +0000 Subject: RFR (XS) 8175269: [s390] cleanup calls to vtable_start_offset() and vtable_length_offset() Message-ID: Hi all, May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. Bug: https://bugs.openjdk.java.net/browse/JDK-8175269 Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175269/ Description: The two static functions are declared/defined in class Klass, but accessed via the subclass InstanceKlass. This is true for s390 only. All other architectures access them via the defining class. My change removes the ?detour? via InstanceKlass. Thanks, Lutz From adam.retter at googlemail.com Tue Feb 21 17:06:37 2017 From: adam.retter at googlemail.com (Adam Retter) Date: Tue, 21 Feb 2017 17:06:37 +0000 Subject: JNI Warnings in OpenJDK 8 from JVM Message-ID: Hi there, I have been working on RocksJava (https://github.com/facebook/rocksdb) which is a combination of C++ JNI and Java 7. When running this (make jcheck) with the java argument `-Xcheck:jni` on the Java 8 VM, we noticed lots of warnings along the lines of: WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... We modified our code to call `env->ExceptionCheck()` appropriately (in this branch: https://github.com/adamretter/rocksdb/tree/java8), and the vast majority of these warnings have disappeared. However we do still seem to be getting several warnings (3 distinct warnings repeated), these if I am not mistaken are actually being omitted from code within the JVM itself rather than RocksJava code. The warnings with traces look like: WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod at java.lang.Class.isInstance(Native Method) at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:231) at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255) at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244) at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194) at org.junit.runners.ParentRunner.run(ParentRunner.java:362) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.runner.JUnitCore.run(JUnitCore.java:105) at org.junit.runner.JUnitCore.run(JUnitCore.java:94) at org.rocksdb.test.RocksJunitRunner.main(RocksJunitRunner.java:61) WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod at java.lang.Class.getSuperclass(Native Method) at sun.reflect.Reflection.isSubclassOf(Reflection.java:247) at sun.reflect.ReflectionFactory.newConstructorAccessor(ReflectionFactory.java:194) at java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:460) at java.lang.reflect.Constructor.newInstance(Constructor.java:420) at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217) at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.runner.JUnitCore.run(JUnitCore.java:105) at org.junit.runner.JUnitCore.run(JUnitCore.java:94) at org.rocksdb.test.RocksJunitRunner.main(RocksJunitRunner.java:61) WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod at java.lang.ClassLoader.findBootstrapClass(Native Method) at java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1015) at java.lang.ClassLoader.loadClass(ClassLoader.java:413) - locked <0x000000076ab00380> (a java.lang.Object) at java.lang.ClassLoader.loadClass(ClassLoader.java:411) - locked <0x000000076ab002d0> (a java.lang.Object) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.junit.runner.notification.RunNotifier.fireTestRunFinished(RunNotifier.java:100) at org.junit.runner.JUnitCore.run(JUnitCore.java:138) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.runner.JUnitCore.run(JUnitCore.java:105) at org.junit.runner.JUnitCore.run(JUnitCore.java:94) at org.rocksdb.test.RocksJunitRunner.main(RocksJunitRunner.java:61) It seems I am not the only developer noticing this unexpected behaviour with Java 8, here is a bug report for DropBox - https://github.com/dropbox/djinni/issues/152#issuecomment-157220326 The test cases I have collected that reproduce some of these warnings (weirdly I struggled to reproduce exactly the warnings I see from RocksJava) look like: Test Case 1 ---------------- $ echo 'public class Foo { public static void main(String[] args) throws Exception { } }' > Foo.java $ javac Foo.java $ java -Xcheck:jni Foo WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod Test Case 2 ---------------- $ echo 'public class Foo2 { public static void main(String[] args) throws Exception { System.runFinalization(); System.runFinalization(); } }' > Foo2.java $ javac Foo2.java $ java -Xcheck:jni Foo2 WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod at java.lang.Runtime.runFinalization0(Native Method) at java.lang.Runtime.runFinalization(Runtime.java:712) at java.lang.System.runFinalization(System.java:1015) at Foo.main(Foo.java:1) NOTE - you need the `System.runFinalization();` twice to cause the last warning. Test Case 3 ---------------- $ echo 'import java.net.NetworkInterface; public class Foo3 { public static void main(String[] args) throws Exception { NetworkInterface.getNetworkInterfaces(); } }' > Foo3.java $ javac Foo3.java $ java -Xcheck:jni Foo3 WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING: JNI local refs: 33, exceeds capacity: 32 at java.net.NetworkInterface.getAll(Native Method) at java.net.NetworkInterface.getNetworkInterfaces(NetworkInterface.java:343) at java.net.DefaultInterface.chooseDefaultInterface(DefaultInterface.java:67) at java.net.DefaultInterface.(DefaultInterface.java:46) at java.net.NetworkInterface.(NetworkInterface.java:65) at Foo3.main(Foo3.java:1) WARNING: JNI local refs: 33, exceeds capacity: 32 at java.net.NetworkInterface.getAll(Native Method) at java.net.NetworkInterface.getNetworkInterfaces(NetworkInterface.java:343) at Foo3.main(Foo3.java:1) I have searched the OpenJDK and OracleJDK bug trackers but could not find an issue that quite covers this. For reference I have tested all of the above on Oracle JDK 1.8.0_121-b13 and Azul Zulu OpenJDK 8.20.0.5-jdk8.0.121 on Apple macOS 10.12.3. I would be interested to find out more about what is causing this issue and how to resolve it. Thanks Adam. -- Adam Retter skype: adam.retter tweet: adamretter http://www.adamretter.org.uk From martin.doerr at sap.com Tue Feb 21 17:11:25 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 21 Feb 2017 17:11:25 +0000 Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" In-Reply-To: References: Message-ID: <46d01d26ecc54e00bbc053a1a77b6dee@sap.com> Hi Lutz, thanks for doing this cleanup. Looks good to me. I think I can sponsor it as it only touches s390 files and is targeted for jdk10. However, we need a jdk10 reviewer, too. Best regards, Martin -----Original Message----- From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Schmidt, Lutz Sent: Dienstag, 21. Februar 2017 16:38 To: s390x-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" Hi all, May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. Bug: https://bugs.openjdk.java.net/browse/JDK-8175267 Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175267/ Description: The "handler_for_unsafe_access? is a relic from former times, having survived only on s390x. The stub is generated, but never used (called). This change removes everything related. Thanks, Lutz From martin.doerr at sap.com Tue Feb 21 17:28:36 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 21 Feb 2017 17:28:36 +0000 Subject: RFR (XS) 8175269: [s390] cleanup calls to vtable_start_offset() and vtable_length_offset() In-Reply-To: References: Message-ID: <9aaf1cea09f645589bfe8577bcbd08bd@sap.com> Hi Lutz, this one looks good to me, too. The other platforms also use the Klass version. I can also sponsor this one as it only touches s390 files and targets jdk10, but have to wait for a jdk10 reviewer. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Schmidt, Lutz Sent: Dienstag, 21. Februar 2017 17:46 To: hotspot-dev at openjdk.java.net Subject: RFR (XS) 8175269: [s390] cleanup calls to vtable_start_offset() and vtable_length_offset() Hi all, May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. Bug: https://bugs.openjdk.java.net/browse/JDK-8175269 Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175269/ Description: The two static functions are declared/defined in class Klass, but accessed via the subclass InstanceKlass. This is true for s390 only. All other architectures access them via the defining class. My change removes the ?detour? via InstanceKlass. Thanks, Lutz From bob.vandette at oracle.com Tue Feb 21 18:36:41 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 21 Feb 2017 13:36:41 -0500 Subject: RFR: AOT: jaotc should provide an option to specify the path to the platform linker Message-ID: <393D50F7-24AC-4511-9369-242E44916089@oracle.com> Please review this change to the jaotc tool which allows the user to specify the path to the native toolchain linker required by AOT. RFE: https://bugs.openjdk.java.net/browse/JDK-8174863? WEBREV: http://cr.openjdk.java.net/~bobv/8174863/hotspot/webrev/ Bob. From lutz.schmidt at sap.com Tue Feb 21 19:30:39 2017 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 21 Feb 2017 19:30:39 +0000 Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" In-Reply-To: <46d01d26ecc54e00bbc053a1a77b6dee@sap.com> References: <46d01d26ecc54e00bbc053a1a77b6dee@sap.com> Message-ID: Thank you, Martin, for reviewing this one. Regards, Lutz > On 21 Feb 2017, at 18:11, Doerr, Martin wrote: > > Hi Lutz, > > thanks for doing this cleanup. Looks good to me. > I think I can sponsor it as it only touches s390 files and is targeted for jdk10. > However, we need a jdk10 reviewer, too. > > Best regards, > Martin > > > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On Behalf Of Schmidt, Lutz > Sent: Dienstag, 21. Februar 2017 16:38 > To: s390x-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net > Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" > > Hi all, > > May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8175267 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175267/ > > Description: > The "handler_for_unsafe_access? is a relic from former times, having survived only on s390x. The stub is generated, but never used (called). This change removes everything related. > > Thanks, > Lutz > From lutz.schmidt at sap.com Tue Feb 21 19:31:42 2017 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 21 Feb 2017 19:31:42 +0000 Subject: RFR (XS) 8175269: [s390] cleanup calls to vtable_start_offset() and vtable_length_offset() In-Reply-To: <9aaf1cea09f645589bfe8577bcbd08bd@sap.com> References: <9aaf1cea09f645589bfe8577bcbd08bd@sap.com> Message-ID: Hi Martin, thank you for checking it out. Regards, Lutz Dr. Lutz Schmidt | Reitenweg 11 | 69226 Nu?loch > On 21 Feb 2017, at 18:28, Doerr, Martin wrote: > > Hi Lutz, > > this one looks good to me, too. The other platforms also use the Klass version. > I can also sponsor this one as it only touches s390 files and targets jdk10, but have to wait for a jdk10 reviewer. > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Schmidt, Lutz > Sent: Dienstag, 21. Februar 2017 17:46 > To: hotspot-dev at openjdk.java.net > Subject: RFR (XS) 8175269: [s390] cleanup calls to vtable_start_offset() and vtable_length_offset() > > Hi all, > > May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8175269 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175269/ > > Description: > The two static functions are declared/defined in class Klass, but accessed via the subclass InstanceKlass. This is true for s390 only. All other architectures access them via the defining class. My change removes the ?detour? via InstanceKlass. > > Thanks, > Lutz > From vladimir.kozlov at oracle.com Tue Feb 21 22:13:50 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 21 Feb 2017 14:13:50 -0800 Subject: RFR: AOT: jaotc should provide an option to specify the path to the platform linker In-Reply-To: <393D50F7-24AC-4511-9369-242E44916089@oracle.com> References: <393D50F7-24AC-4511-9369-242E44916089@oracle.com> Message-ID: <74a192ee-d89a-2024-a872-7ea2d5837157@oracle.com> Looks good. Thanks, Vladimir On 2/21/17 10:36 AM, Bob Vandette wrote: > Please review this change to the jaotc tool which allows the user to specify the path to the > native toolchain linker required by AOT. > > RFE: > > https://bugs.openjdk.java.net/browse/JDK-8174863? > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8174863/hotspot/webrev/ > > Bob. > From david.holmes at oracle.com Wed Feb 22 03:09:33 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Feb 2017 13:09:33 +1000 Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" In-Reply-To: References: Message-ID: <60f9e802-ca60-cbdd-e6ec-cc2d4f06b589@oracle.com> Hi Lutz, I concur that this removes all code related to handler_for_unsafe_access in the s390 code. Reviewed. :) Thanks, David On 22/02/2017 1:38 AM, Schmidt, Lutz wrote: > Hi all, > > May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8175267 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175267/ > > Description: > The "handler_for_unsafe_access? is a relic from former times, having survived only on s390x. The stub is generated, but never used (called). This change removes everything related. > > Thanks, > Lutz > From david.holmes at oracle.com Wed Feb 22 03:12:34 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Feb 2017 13:12:34 +1000 Subject: RFR (XS) 8175269: [s390] cleanup calls to vtable_start_offset() and vtable_length_offset() In-Reply-To: References: Message-ID: <504df1a2-a397-b8c8-237b-04116a1e28ab@oracle.com> Hi Lutz, This looks fine as well. Reviewed. Thanks, David On 22/02/2017 2:46 AM, Schmidt, Lutz wrote: > Hi all, > > May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8175269 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175269/ > > Description: > The two static functions are declared/defined in class Klass, but accessed via the subclass InstanceKlass. This is true for s390 only. All other architectures access them via the defining class. My change removes the ?detour? via InstanceKlass. > > Thanks, > Lutz > From david.holmes at oracle.com Wed Feb 22 04:26:04 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Feb 2017 14:26:04 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> Message-ID: On 22/02/2017 1:49 AM, Erik Helin wrote: > > > On 02/21/2017 02:57 AM, Kim Barrett wrote: >>> On Feb 20, 2017, at 5:52 PM, David Holmes >>> wrote: >>>> 02 -> 03: >>>> - Only use load_acquire when reading the size for the "head" chunk >>> >>> Sorry but I don't like the way this is done - the conditional may end >>> up being more expensive than the unnecessary load-acquire. Unrolling >>> the first loop iteration, as per the email discussion, is a better >>> way to go IMO. >> >> +1 > > David, Kim, please see new patches at: > - inc: http://cr.openjdk.java.net/~ehelin/8168914/03-04/ Didn't need to add a new function to the API just to unroll the first loop iteration! :( Ditto Kim's comments. Thanks, David ----- > - full: http://cr.openjdk.java.net/~ehelin/8168914/04/ > > Thanks, > Erik From david.holmes at oracle.com Wed Feb 22 04:47:40 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Feb 2017 14:47:40 +1000 Subject: JNI Warnings in OpenJDK 8 from JVM In-Reply-To: References: Message-ID: Hi Adam, On 22/02/2017 3:06 AM, Adam Retter wrote: > Hi there, > > I have been working on RocksJava (https://github.com/facebook/rocksdb) > which is a combination of C++ JNI and Java 7. > > When running this (make jcheck) with the java argument `-Xcheck:jni` > on the Java 8 VM, we noticed lots of warnings along the lines of: > > WARNING in native method: JNI call made without checking exceptions > when required to from CallObjectMethod > WARNING in native method: JNI call made without checking exceptions > when required to from CallVoidMethod > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticVoidMethod > ... > > We modified our code to call `env->ExceptionCheck()` appropriately (in > this branch: https://github.com/adamretter/rocksdb/tree/java8), and > the vast majority of these warnings have disappeared. However we do > still seem to be getting several warnings (3 distinct warnings > repeated), these if I am not mistaken are actually being omitted from > code within the JVM itself rather than RocksJava code. The warnings > with traces look like: > > > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticVoidMethod > at java.lang.Class.isInstance(Native Method) > at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:231) > at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255) > at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244) > at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194) > at org.junit.runners.ParentRunner.run(ParentRunner.java:362) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:27) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at org.junit.runner.JUnitCore.run(JUnitCore.java:115) > at org.junit.runner.JUnitCore.run(JUnitCore.java:105) > at org.junit.runner.JUnitCore.run(JUnitCore.java:94) > at org.rocksdb.test.RocksJunitRunner.main(RocksJunitRunner.java:61) I think I need to see how main gets invoked in a thread. I suspect it is being done by reflection. Even so the flag to say there is a pending-"exception check" is enabled when the method returns, so we should not be in a position where that flag should be on when we eventually call Class.isInstance(). Unless the flag was already on, due to the way in "main" actually gets invoked by the junit code! David ----- > > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticVoidMethod > at java.lang.Class.getSuperclass(Native Method) > at sun.reflect.Reflection.isSubclassOf(Reflection.java:247) > at sun.reflect.ReflectionFactory.newConstructorAccessor(ReflectionFactory.java:194) > at java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:460) > at java.lang.reflect.Constructor.newInstance(Constructor.java:420) > at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217) > at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266) > at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263) > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:27) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at org.junit.runner.JUnitCore.run(JUnitCore.java:115) > at org.junit.runner.JUnitCore.run(JUnitCore.java:105) > at org.junit.runner.JUnitCore.run(JUnitCore.java:94) > at org.rocksdb.test.RocksJunitRunner.main(RocksJunitRunner.java:61) > > > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticVoidMethod > at java.lang.ClassLoader.findBootstrapClass(Native Method) > at java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1015) > at java.lang.ClassLoader.loadClass(ClassLoader.java:413) > - locked <0x000000076ab00380> (a java.lang.Object) > at java.lang.ClassLoader.loadClass(ClassLoader.java:411) > - locked <0x000000076ab002d0> (a java.lang.Object) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at org.junit.runner.notification.RunNotifier.fireTestRunFinished(RunNotifier.java:100) > at org.junit.runner.JUnitCore.run(JUnitCore.java:138) > at org.junit.runner.JUnitCore.run(JUnitCore.java:115) > at org.junit.runner.JUnitCore.run(JUnitCore.java:105) > at org.junit.runner.JUnitCore.run(JUnitCore.java:94) > at org.rocksdb.test.RocksJunitRunner.main(RocksJunitRunner.java:61) > > > It seems I am not the only developer noticing this unexpected > behaviour with Java 8, here is a bug report for DropBox - > https://github.com/dropbox/djinni/issues/152#issuecomment-157220326 > > The test cases I have collected that reproduce some of these warnings > (weirdly I struggled to reproduce exactly the warnings I see from > RocksJava) look like: > > Test Case 1 > ---------------- > $ echo 'public class Foo { public static void main(String[] args) > throws Exception { } }' > Foo.java > $ javac Foo.java > $ java -Xcheck:jni Foo > > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticObjectMethod > WARNING in native method: JNI call made without checking exceptions > when required to from CallObjectMethod > > Test Case 2 > ---------------- > $ echo 'public class Foo2 { public static void main(String[] args) > throws Exception { > System.runFinalization(); System.runFinalization(); } }' > Foo2.java > $ javac Foo2.java > $ java -Xcheck:jni Foo2 > > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticObjectMethod > WARNING in native method: JNI call made without checking exceptions > when required to from CallObjectMethod > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticVoidMethod > at java.lang.Runtime.runFinalization0(Native Method) > at java.lang.Runtime.runFinalization(Runtime.java:712) > at java.lang.System.runFinalization(System.java:1015) > at Foo.main(Foo.java:1) > > NOTE - you need the `System.runFinalization();` twice to cause the last warning. > > > Test Case 3 > ---------------- > $ echo 'import java.net.NetworkInterface; public class Foo3 { public > static void main(String[] args) throws Exception { > NetworkInterface.getNetworkInterfaces(); } }' > Foo3.java > $ javac Foo3.java > $ java -Xcheck:jni Foo3 > > WARNING in native method: JNI call made without checking exceptions > when required to from CallStaticObjectMethod > WARNING in native method: JNI call made without checking exceptions > when required to from CallObjectMethod > WARNING: JNI local refs: 33, exceeds capacity: 32 > at java.net.NetworkInterface.getAll(Native Method) > at java.net.NetworkInterface.getNetworkInterfaces(NetworkInterface.java:343) > at java.net.DefaultInterface.chooseDefaultInterface(DefaultInterface.java:67) > at java.net.DefaultInterface.(DefaultInterface.java:46) > at java.net.NetworkInterface.(NetworkInterface.java:65) > at Foo3.main(Foo3.java:1) > WARNING: JNI local refs: 33, exceeds capacity: 32 > at java.net.NetworkInterface.getAll(Native Method) > at java.net.NetworkInterface.getNetworkInterfaces(NetworkInterface.java:343) > at Foo3.main(Foo3.java:1) > > > I have searched the OpenJDK and OracleJDK bug trackers but could not > find an issue that quite covers this. For reference I have tested all > of the above on Oracle JDK 1.8.0_121-b13 and Azul Zulu OpenJDK > 8.20.0.5-jdk8.0.121 on Apple macOS 10.12.3. > > I would be interested to find out more about what is causing this > issue and how to resolve it. > > Thanks Adam. > > From john.r.rose at oracle.com Wed Feb 22 05:49:43 2017 From: john.r.rose at oracle.com (John Rose) Date: Tue, 21 Feb 2017 21:49:43 -0800 Subject: [10] RFC: Relaxing access restrictions for VM annotations In-Reply-To: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> References: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> Message-ID: <8BE0EB55-C32C-4C88-82DA-B6F545A4468D@oracle.com> Gating annotation effectiveness on the module system is brilliant. But I recommend not making too many compromises for the sake of java.base. Bootstrapping that particular code will always be hard and messy. (Yes, "always". No magic bullets on any horizon, at least until we have full AOT with no backoff to bootstrapping from bytecode. That's pretty much "always".) So to cut bootstrap loops you need a temporary first opinion about whatever it is that is getting bootstrapped, until you can get the permanent good facts from the system after it is bootstrapped. (Getting good facts from the module system requires significant parts of java.base to be booted up.) Thus, expect to have some temporary oracle that can give answers on behalf of the module system, for exactly those annotations which are used in java.base. You can record the answers of the oracle and assertion-check them much later, when the module system can be queried. I don't think delaying from phase to phase within one class loading operation buys you anything but trouble. Does that help? ? John On Feb 21, 2017, at 6:36 AM, Vladimir Ivanov wrote: > > FYI I've been experimenting with relaxing restrictions on VM annotations - Java annotations which affect how VM work - @ReservedStackAccess, @Contended, @Stable, etc (most of them are located in jdk.internal.vm.annotation in 9). > > Right now, VM ignores any annotation unless it is declared on a class from privileged context (JDK classes or VM anonymous classes). > > Some features introduced adhoc flags to relax the restriction (e.g., -XX:RestrictContended & -XX:RestrictReservedStack), but with modules there's a universal way to expose them in controlled manner: rely on annotation class accessibility checks. In order to make some annotation available in non-privileged context, it is enough to add relevant export to java.base module (e.g., --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED). > > The problem I faced was that accessibility checks are performed on already loaded classes, but annotations are used during class parsing and the checks have to be performed against the class being loaded. > > One option is to re-implement accessibility checking logic and extract all necessary information from ClassFileParser. > > Another approach is to collect VM annotations during parsing and after the class is loaded apply them one-by-one doing accessibility checks. It works fine for most of the existing annotations, but problematic for @Contended - it affects instance layout which is computed during parsing. > > So, after some experiments, I decided to try different approach: optimistically assume all annotations are accessible and re-parse the class if any used annotation is not available: > > http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.00 > > The benefit is that parsing & accessibility checking logic stay the same. Also, it enabled sanity checking logic for annotation usages almost for free. > > Overall, I'm quite satisfied with the result, but stumbled upon some nuisance when implementing re-parsing (e.g., tracing framework needs access to ClassFileParser and it brings unnecessary changes into ClassFileParser::parse_instance_klass_from_stream). > > Before investing into polishing the change, I'd like to discuss the approach I chose. > > Any problems with class file re-parsing or annotation resolution in VM I overlooked? > > Maybe there are strong arguments for other options (#1: accessibity checks in ClassFileParser & #2: apply annotations after the class is loaded)? > > Thanks! > > Best regards, > Vladimir Ivanov From lutz.schmidt at sap.com Wed Feb 22 06:59:16 2017 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Wed, 22 Feb 2017 06:59:16 +0000 Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" In-Reply-To: <60f9e802-ca60-cbdd-e6ec-cc2d4f06b589@oracle.com> References: <60f9e802-ca60-cbdd-e6ec-cc2d4f06b589@oracle.com> Message-ID: <3C50656D-2D4C-4C2F-9B29-CBEDF19B5320@sap.com> Hi David, thanks for reviewing! Lutz On 22.02.17, 04:09, "David Holmes" wrote: Hi Lutz, I concur that this removes all code related to handler_for_unsafe_access in the s390 code. Reviewed. :) Thanks, David On 22/02/2017 1:38 AM, Schmidt, Lutz wrote: > Hi all, > > May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8175267 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175267/ > > Description: > The "handler_for_unsafe_access? is a relic from former times, having survived only on s390x. The stub is generated, but never used (called). This change removes everything related. > > Thanks, > Lutz > From lutz.schmidt at sap.com Wed Feb 22 06:59:49 2017 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Wed, 22 Feb 2017 06:59:49 +0000 Subject: RFR (XS) 8175269: [s390] cleanup calls to vtable_start_offset() and vtable_length_offset() In-Reply-To: <504df1a2-a397-b8c8-237b-04116a1e28ab@oracle.com> References: <504df1a2-a397-b8c8-237b-04116a1e28ab@oracle.com> Message-ID: <8AB994AB-35BB-432A-831C-FDE994BD8324@sap.com> Thanks again, David! On 22.02.17, 04:12, "David Holmes" wrote: Hi Lutz, This looks fine as well. Reviewed. Thanks, David On 22/02/2017 2:46 AM, Schmidt, Lutz wrote: > Hi all, > > May I kindly request reviews for my very small cleanup change? Further down the road, I would need a sponsor, too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8175269 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175269/ > > Description: > The two static functions are declared/defined in class Klass, but accessed via the subclass InstanceKlass. This is true for s390 only. All other architectures access them via the defining class. My change removes the ?detour? via InstanceKlass. > > Thanks, > Lutz > From thomas.schatzl at oracle.com Wed Feb 22 08:35:16 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 22 Feb 2017 09:35:16 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> Message-ID: <1487752516.4206.4.camel@oracle.com> Hi Erik, On Tue, 2017-02-21 at 11:19 -0500, Kim Barrett wrote: > > > > On Feb 21, 2017, at 10:49 AM, Erik Helin > > wrote: > > > > David, Kim, please see new patches at: > > - inc: http://cr.openjdk.java.net/~ehelin/8168914/03-04/ > > - full: http://cr.openjdk.java.net/~ehelin/8168914/04/ > > > > Thanks, > > Erik > ------------------------------------------------------------------- > ----------- > src/share/vm/classfile/classLoaderData.cpp > ?144???Chunk* head = (Chunk*) OrderAccess::load_ptr_acquire((volatile > intptr_t*)&_head); > > Still has unnecessary cast of &_head, > > ------------------------------------------------------------------- > ----------- > src/share/vm/classfile/classLoaderData.hpp > ?186?????void oops_do_chunk(OopClosure* f, Chunk* c, const juint > size); > > oops_do_chunk ahould be private. > > ------------------------------------------------------------------- > ----------- > > I don't need a new webrev for these. > ? same here. Thanks, ? Thomas From goetz.lindenmaier at sap.com Wed Feb 22 08:53:31 2017 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 22 Feb 2017 08:53:31 +0000 Subject: RFR (XS) 8175267: [s390] cleanup stub code "handler_for_unsafe_access" In-Reply-To: <46d01d26ecc54e00bbc053a1a77b6dee@sap.com> References: <46d01d26ecc54e00bbc053a1a77b6dee@sap.com> Message-ID: <25b90d1e03ee45459a3e97cf375bccc2@sap.com> Hi Lutz, the change looks good. Best regards, Goetz. > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On > Behalf Of Doerr, Martin > Sent: Dienstag, 21. Februar 2017 19:11 > To: Schmidt, Lutz ; s390x-port- > dev at openjdk.java.net; hotspot-dev at openjdk.java.net > Subject: RE: RFR (XS) 8175267: [s390] cleanup stub code > "handler_for_unsafe_access" > > Hi Lutz, > > thanks for doing this cleanup. Looks good to me. > I think I can sponsor it as it only touches s390 files and is targeted for jdk10. > However, we need a jdk10 reviewer, too. > > Best regards, > Martin > > > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] On > Behalf Of Schmidt, Lutz > Sent: Dienstag, 21. Februar 2017 16:38 > To: s390x-port-dev at openjdk.java.net; hotspot-dev at openjdk.java.net > Subject: RFR (XS) 8175267: [s390] cleanup stub code > "handler_for_unsafe_access" > > Hi all, > > May I kindly request reviews for my very small cleanup change? Further > down the road, I would need a sponsor, too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8175267 > Webrev: http://cr.openjdk.java.net/~lucy/webrevs/8175267/ > > Description: > The "handler_for_unsafe_access? is a relic from former times, having > survived only on s390x. The stub is generated, but never used (called). This > change removes everything related. > > Thanks, > Lutz From mikael.gerdin at oracle.com Wed Feb 22 16:07:06 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 22 Feb 2017 17:07:06 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles Message-ID: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> Hi all, Please review this revised change to jweak to support G1. I've copied Kim's original description of the fix below for reference: > > The problem being addressed is that, when using G1, dereferencing a > jweak must ensure the obtained referent is ensured live, just as for > other weak reference types. This has always been a requirement, and > failure to do so appears to be a since-day1 G1 bug. > > There are two categories of places that need to address this issue: > JNIHandles::resolve and friends, and returning a jobject from native > code to Java. In both of these places the jobject whose contained oop > is being extracted might really be a jweak. jweak is > representationally indistinguishable from jobject; jweak is just a > typedef for jobject. The documentation says a jweak can be used > anywhere a jobject is permitted, making that type equivalence > necessary for a C API. (A C++ API might be able to have distinct > types via inheritance, though would still need a method for > distinguishing them at runtime. But since JNI is a C API...). > > The basic approach being taken is to "tag" jweaks by setting the low > order bit (recall that jweak == jobject == _jobject*), in order to > distinguish them from non-jweak jobjects. This tagging is only being > done when the selected GC needs the distinction, e.g. G1. This gives > us a representational difference between jobject and jweak on which to > base the different behavior, hiding that distinction behind the > existing API. > > JNIHandles::resolve and friends are changed to unconditionally strip > off any potential weak tag when translating from jobject to to oop*. > That's cheaper than testing for the conditional use of tagging and > then stripping. Also stripped when deleting JNI handles. > > TemplateInterpreterGenerator::generate_native_entry and > SharedRuntime::generate_native_wrapper are changed to conditionally > emit code to apply the G1 pre-barrier to the value obtained from a > jobject result, when the result is tagged as a jweak, which only > occurs when using G1. > > For arm32/arm64, this required moving the g1_write_barrier_pre > definitions from InterpreterMacroAssembler to MacroAssembler. Also > moved g1_write_barrier_post, for consistency. > In addition to Kim's final version of the change (which was backed out) this updated version adds code to mask out the weak tag bit in the fast JNI field getters on all platforms where fast JNI getters are used. Since the fast JNI getters only read primitive values from objects me and Kim decided that there was no need to invoke the G1 pre-barrier if a jweak was used as the "receiver" for the getters, this simplifies the change to the fast getters and I was able to come up with a seemingly efficient instruction encoding on all platforms. A couple of comments about the implementations: My goal was to encode clearing of the weak_tag_mask (which at the moment is 1). On SPARC and 32 bit arm there is an instruction that can be used straight off to perform an and with a value that is first negated, ANDN on SPARC and BTC on ARM. On 64 bit ARM the encoding for immediates allows encoding of the bitwise inverse of 1 (0xfffffffffffffffe) but if weak_tag_mask is changed the encoding may fail. On x86 I pass the mask as a signed 32 bit immediate but note that the assembler encodes it as an 8 bit immediate and sets a sign-extension bit in the opcode. Testing: Standard JPRT and tier2-3 HotSpot tests. I've modified the CallWithJNIWeak test to call all the primitive getters and some other interesting cases I came up with. I've also run the CallWithJNIWeak test on all platforms on both product and fastdebug builds (since fast JNI getters are implicitly disabled in debug builds due to VerifyJNIFields being enabled by default in debug builds. I've not built or tested the aarch64 port but I think it's correct and I hope someone can test it for me. Webrevs: Kim's initial change: http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ My additions: http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ Full webrev: http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ Bugs: https://bugs.openjdk.java.net/browse/JDK-8175085 https://bugs.openjdk.java.net/browse/JDK-8166188 Thanks /Mikael From kim.barrett at oracle.com Wed Feb 22 19:40:11 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 22 Feb 2017 14:40:11 -0500 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> Message-ID: > On Feb 22, 2017, at 11:07 AM, Mikael Gerdin wrote: > > Hi all, > Please review this revised change to jweak to support G1. > > I've copied Kim's original description of the fix below for reference: > >> >> The problem being addressed is that, when using G1, dereferencing a >> jweak must ensure the obtained referent is ensured live, just as for >> other weak reference types. This has always been a requirement, and >> failure to do so appears to be a since-day1 G1 bug. >> >> There are two categories of places that need to address this issue: >> JNIHandles::resolve and friends, and returning a jobject from native >> code to Java. In both of these places the jobject whose contained oop >> is being extracted might really be a jweak. jweak is >> representationally indistinguishable from jobject; jweak is just a >> typedef for jobject. The documentation says a jweak can be used >> anywhere a jobject is permitted, making that type equivalence >> necessary for a C API. (A C++ API might be able to have distinct >> types via inheritance, though would still need a method for >> distinguishing them at runtime. But since JNI is a C API...). >> >> The basic approach being taken is to "tag" jweaks by setting the low >> order bit (recall that jweak == jobject == _jobject*), in order to >> distinguish them from non-jweak jobjects. This tagging is only being >> done when the selected GC needs the distinction, e.g. G1. This gives >> us a representational difference between jobject and jweak on which to >> base the different behavior, hiding that distinction behind the >> existing API. >> >> JNIHandles::resolve and friends are changed to unconditionally strip >> off any potential weak tag when translating from jobject to to oop*. >> That's cheaper than testing for the conditional use of tagging and >> then stripping. Also stripped when deleting JNI handles. >> >> TemplateInterpreterGenerator::generate_native_entry and >> SharedRuntime::generate_native_wrapper are changed to conditionally >> emit code to apply the G1 pre-barrier to the value obtained from a >> jobject result, when the result is tagged as a jweak, which only >> occurs when using G1. >> >> For arm32/arm64, this required moving the g1_write_barrier_pre >> definitions from InterpreterMacroAssembler to MacroAssembler. Also >> moved g1_write_barrier_post, for consistency. >> > > In addition to Kim's final version of the change (which was backed out) this updated version adds code to mask out the weak tag bit in the fast JNI field getters on all platforms where fast JNI getters are used. > > [?] > > Webrevs: > Kim's initial change: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ > My additions: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ > Full webrev: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ > > Bugs: > https://bugs.openjdk.java.net/browse/JDK-8175085 > https://bugs.openjdk.java.net/browse/JDK-8166188 > > Thanks > /Mikael Mikael - Thanks for taking this up for me. Just a few minor comments. ------------------------------------------------------------------------------ The usual copyright reminder. ------------------------------------------------------------------------------ src/cpu/arm/vm/jniFastGetField_arm.cpp 122 #ifndef AARCH64 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); 124 #else 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); 126 #endif Usual style when selecting between AARCH64 and not seems to be to put the AARCH64 code first, e.g. #ifdef AARCH64 ... #else ... #endif ------------------------------------------------------------------------------ src/cpu/x86/vm/jniFastGetField_x86_32.cpp 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); Use STATIC_ASSERT rather than assert. ------------------------------------------------------------------------------ src/cpu/x86/vm/jniFastGetField_x86_32.cpp 89 const intptr_t inverted_jweak_mask = ~static_cast(JNIHandles::weak_tag_mask); 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to sign-extension With three occurrences of this snippet in this file and two more similar ones in the 64-bit file, a macroAssembler helper seems called for here. ------------------------------------------------------------------------------ src/cpu/x86/vm/jniFastGetField_x86_64.cpp 84 const intptr_t inverted_jweak_mask = ~static_cast(JNIHandles::weak_tag_mask); 85 const int32_t truncated_mask = static_cast(inverted_jweak_mask); Why the two-step conversion here? Why not just const int32_t truncated_mask = static_cast(JNIHandles::weak_tag_mask); That would also make the 32 and 64-bit code more similar in the suggested macroAssembler helper. ------------------------------------------------------------------------------ test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c 51 #define CHECK(variable, expected) \ 52 do { if ((variable) != (expected)) { \ 53 (*env)->ThrowNew(env, exception, #variable" != " #expected); \ 54 return; \ 55 } \ 56 } while(0); The formatting of the code in the macro body is strange and confusing. Also the do-while-zero idiom that I'm used to leaves off the final semicolon in the macro. Instead of the uses are expected to be terminated with semicolons, as you've done. ------------------------------------------------------------------------------ From vladimir.x.ivanov at oracle.com Wed Feb 22 22:20:44 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 23 Feb 2017 01:20:44 +0300 Subject: [10] RFC: Relaxing access restrictions for VM annotations In-Reply-To: <8BE0EB55-C32C-4C88-82DA-B6F545A4468D@oracle.com> References: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> <8BE0EB55-C32C-4C88-82DA-B6F545A4468D@oracle.com> Message-ID: <3c6c6423-173c-4a91-5bcc-854bc06dc9e3@oracle.com> Thanks for the feedback, John. The problem here is not related to bootstrapping (the checks for privileged classes can be just skipped), but because we need to make checks in the context of the class being loaded (in particular, determine the module the class belongs to). I did a quick prototype of accessibility checks during parsing and ended up with the following: http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.01/ Diff between versions: http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.01_vs_00/ It required to duplicate code from IK::set_package(), IK::module(), and Reflection::verify_class_access(). Also, I had to change the loading sequence and compute (possibly, create) package early in the parsing phase. Overall, it looked much messier and more brittle than wrapping class file paring into re-parsing logic. Best regards, Vladimir Ivanov On 2/22/17 8:49 AM, John Rose wrote: > Gating annotation effectiveness on the module system is brilliant. > > But I recommend not making too many compromises for the sake of java.base. > Bootstrapping that particular code will always be hard and messy. > (Yes, "always". No magic bullets on any horizon, at least until we > have full AOT with no backoff to bootstrapping from bytecode. > That's pretty much "always".) So to cut bootstrap loops you need > a temporary first opinion about whatever it is that is getting bootstrapped, > until you can get the permanent good facts from the system after it > is bootstrapped. (Getting good facts from the module system requires > significant parts of java.base to be booted up.) > > Thus, expect to have some temporary oracle that can give answers > on behalf of the module system, for exactly those annotations which > are used in java.base. You can record the answers of the oracle > and assertion-check them much later, when the module system can > be queried. I don't think delaying from phase to phase within one > class loading operation buys you anything but trouble. > > Does that help? > > ? John > > On Feb 21, 2017, at 6:36 AM, Vladimir Ivanov wrote: >> >> FYI I've been experimenting with relaxing restrictions on VM annotations - Java annotations which affect how VM work - @ReservedStackAccess, @Contended, @Stable, etc (most of them are located in jdk.internal.vm.annotation in 9). >> >> Right now, VM ignores any annotation unless it is declared on a class from privileged context (JDK classes or VM anonymous classes). >> >> Some features introduced adhoc flags to relax the restriction (e.g., -XX:RestrictContended & -XX:RestrictReservedStack), but with modules there's a universal way to expose them in controlled manner: rely on annotation class accessibility checks. In order to make some annotation available in non-privileged context, it is enough to add relevant export to java.base module (e.g., --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED). >> >> The problem I faced was that accessibility checks are performed on already loaded classes, but annotations are used during class parsing and the checks have to be performed against the class being loaded. >> >> One option is to re-implement accessibility checking logic and extract all necessary information from ClassFileParser. >> >> Another approach is to collect VM annotations during parsing and after the class is loaded apply them one-by-one doing accessibility checks. It works fine for most of the existing annotations, but problematic for @Contended - it affects instance layout which is computed during parsing. >> >> So, after some experiments, I decided to try different approach: optimistically assume all annotations are accessible and re-parse the class if any used annotation is not available: >> >> http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.00 >> >> The benefit is that parsing & accessibility checking logic stay the same. Also, it enabled sanity checking logic for annotation usages almost for free. >> >> Overall, I'm quite satisfied with the result, but stumbled upon some nuisance when implementing re-parsing (e.g., tracing framework needs access to ClassFileParser and it brings unnecessary changes into ClassFileParser::parse_instance_klass_from_stream). >> >> Before investing into polishing the change, I'd like to discuss the approach I chose. >> >> Any problems with class file re-parsing or annotation resolution in VM I overlooked? >> >> Maybe there are strong arguments for other options (#1: accessibity checks in ClassFileParser & #2: apply annotations after the class is loaded)? >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov > From forax at univ-mlv.fr Wed Feb 22 22:30:20 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 22 Feb 2017 23:30:20 +0100 (CET) Subject: [10] RFC: Relaxing access restrictions for VM annotations In-Reply-To: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> References: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> Message-ID: <1014535310.1420709.1487802620885.JavaMail.zimbra@u-pem.fr> Hi Vladimir, I do not know about the VM side, but you do not have to load the annotation to do the accessibility check, from the annotation name you can find the package and from the package you can find the module, so you do not have to delay until the class is loaded to find its module (if i've understand your issue). R?mi ----- Mail original ----- > De: "Vladimir Ivanov" > ?: "hotspot-dev at openjdk.java.net developers" > Envoy?: Mardi 21 F?vrier 2017 15:36:26 > Objet: [10] RFC: Relaxing access restrictions for VM annotations > FYI I've been experimenting with relaxing restrictions on VM annotations > - Java annotations which affect how VM work - @ReservedStackAccess, > @Contended, @Stable, etc (most of them are located in > jdk.internal.vm.annotation in 9). > > Right now, VM ignores any annotation unless it is declared on a class > from privileged context (JDK classes or VM anonymous classes). > > Some features introduced adhoc flags to relax the restriction (e.g., > -XX:RestrictContended & -XX:RestrictReservedStack), but with modules > there's a universal way to expose them in controlled manner: rely on > annotation class accessibility checks. In order to make some annotation > available in non-privileged context, it is enough to add relevant export > to java.base module (e.g., --add-exports > java.base/jdk.internal.vm.annotation=ALL-UNNAMED). > > The problem I faced was that accessibility checks are performed on > already loaded classes, but annotations are used during class parsing > and the checks have to be performed against the class being loaded. > > One option is to re-implement accessibility checking logic and extract > all necessary information from ClassFileParser. > > Another approach is to collect VM annotations during parsing and after > the class is loaded apply them one-by-one doing accessibility checks. It > works fine for most of the existing annotations, but problematic for > @Contended - it affects instance layout which is computed during parsing. > > So, after some experiments, I decided to try different approach: > optimistically assume all annotations are accessible and re-parse the > class if any used annotation is not available: > > http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.00 > > The benefit is that parsing & accessibility checking logic stay the > same. Also, it enabled sanity checking logic for annotation usages > almost for free. > > Overall, I'm quite satisfied with the result, but stumbled upon some > nuisance when implementing re-parsing (e.g., tracing framework needs > access to ClassFileParser and it brings unnecessary changes into > ClassFileParser::parse_instance_klass_from_stream). > > Before investing into polishing the change, I'd like to discuss the > approach I chose. > > Any problems with class file re-parsing or annotation resolution in VM I > overlooked? > > Maybe there are strong arguments for other options (#1: accessibity > checks in ClassFileParser & #2: apply annotations after the class is > loaded)? > > Thanks! > > Best regards, > Vladimir Ivanov From vladimir.x.ivanov at oracle.com Wed Feb 22 22:47:17 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 23 Feb 2017 01:47:17 +0300 Subject: [10] RFC: Relaxing access restrictions for VM annotations In-Reply-To: <1014535310.1420709.1487802620885.JavaMail.zimbra@u-pem.fr> References: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> <1014535310.1420709.1487802620885.JavaMail.zimbra@u-pem.fr> Message-ID: > I do not know about the VM side, but you do not have to load the annotation to do the accessibility check, > from the annotation name you can find the package and from the package you can find the module, > so you do not have to delay until the class is loaded to find its module (if i've understand your issue). Yes, that's what I did in webrev.01 [1]. The downsides are: (1) it uses info from the class file which haven't been validated yet; (2) some sensitive code is duplicated. That's why I'm uncomfortable with that approach. There's no problem with getting to the class representing the annotation. The problem is with the class which _uses_ that annotation and it is being loaded when decisions are made. E.g., @Contended affects instance layout of the class being loaded and its effects should be applied before the loading is finished. Best regards, Vladimir Ivanov [1] http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.01/ ClassFileParser::precompute_package_and_module() & ClassFileParser::is_annotation_accessible() > ----- Mail original ----- >> De: "Vladimir Ivanov" >> ?: "hotspot-dev at openjdk.java.net developers" >> Envoy?: Mardi 21 F?vrier 2017 15:36:26 >> Objet: [10] RFC: Relaxing access restrictions for VM annotations > >> FYI I've been experimenting with relaxing restrictions on VM annotations >> - Java annotations which affect how VM work - @ReservedStackAccess, >> @Contended, @Stable, etc (most of them are located in >> jdk.internal.vm.annotation in 9). >> >> Right now, VM ignores any annotation unless it is declared on a class >> from privileged context (JDK classes or VM anonymous classes). >> >> Some features introduced adhoc flags to relax the restriction (e.g., >> -XX:RestrictContended & -XX:RestrictReservedStack), but with modules >> there's a universal way to expose them in controlled manner: rely on >> annotation class accessibility checks. In order to make some annotation >> available in non-privileged context, it is enough to add relevant export >> to java.base module (e.g., --add-exports >> java.base/jdk.internal.vm.annotation=ALL-UNNAMED). >> >> The problem I faced was that accessibility checks are performed on >> already loaded classes, but annotations are used during class parsing >> and the checks have to be performed against the class being loaded. >> >> One option is to re-implement accessibility checking logic and extract >> all necessary information from ClassFileParser. >> >> Another approach is to collect VM annotations during parsing and after >> the class is loaded apply them one-by-one doing accessibility checks. It >> works fine for most of the existing annotations, but problematic for >> @Contended - it affects instance layout which is computed during parsing. >> >> So, after some experiments, I decided to try different approach: >> optimistically assume all annotations are accessible and re-parse the >> class if any used annotation is not available: >> >> http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.00 >> >> The benefit is that parsing & accessibility checking logic stay the >> same. Also, it enabled sanity checking logic for annotation usages >> almost for free. >> >> Overall, I'm quite satisfied with the result, but stumbled upon some >> nuisance when implementing re-parsing (e.g., tracing framework needs >> access to ClassFileParser and it brings unnecessary changes into >> ClassFileParser::parse_instance_klass_from_stream). >> >> Before investing into polishing the change, I'd like to discuss the >> approach I chose. >> >> Any problems with class file re-parsing or annotation resolution in VM I >> overlooked? >> >> Maybe there are strong arguments for other options (#1: accessibity >> checks in ClassFileParser & #2: apply annotations after the class is >> loaded)? >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov From john.r.rose at oracle.com Wed Feb 22 23:46:14 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 22 Feb 2017 15:46:14 -0800 Subject: [10] RFC: Relaxing access restrictions for VM annotations In-Reply-To: <3c6c6423-173c-4a91-5bcc-854bc06dc9e3@oracle.com> References: <427a137a-b9b7-c547-b9a8-3c654ae3ec84@oracle.com> <8BE0EB55-C32C-4C88-82DA-B6F545A4468D@oracle.com> <3c6c6423-173c-4a91-5bcc-854bc06dc9e3@oracle.com> Message-ID: <0401E022-1A46-4C92-A249-4FFCE8CB26FB@oracle.com> On Feb 22, 2017, at 2:20 PM, Vladimir Ivanov wrote: > > Thanks for the feedback, John. (You are kind; it wasn't very on-point.) > The problem here is not related to bootstrapping (the checks for privileged classes can be just skipped), but because we need to make checks in the context of the class being loaded (in particular, determine the module the class belongs to). > > I did a quick prototype of accessibility checks during parsing and ended up with the following: > http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.01/ > > Diff between versions: > http://cr.openjdk.java.net/~vlivanov/vm_annotations/webrev.01_vs_00/ Yes, that's ugly. The IK is lying in parts on the operating table and you need to ask it an urgent question. So you make a note to ask the question later on when it is awake. (And then if it gives the wrong answer, it's back to the table!) OK, so the retry is not so ugly. I would prefer if the retry logic were the *only* thing happening in the function that manages it. In particular, the soundness of the technique (replaying the second time with a more precise annotation mask) will be easier to verify if the function doesn't have any other distractions in it. I see you have some "FIXME" items to refactor stray code elsewhere, so +1 to that. > It required to duplicate code from IK::set_package(), IK::module(), and Reflection::verify_class_access(). Also, I had to change the loading sequence and compute (possibly, create) package early in the parsing phase. > > Overall, it looked much messier and more brittle than wrapping class file paring into re-parsing logic. As we discussed on IM you can do more with those nifty little annotation masks, including location validity testing. The stress test strategy looks good. If there are residual bugs due to reparsing they will only affect those rare class files which attempt unsuccessfully to use the scoped annotations. Reviewed, with suggestions. ? John P.S. Tentative suggestion: Extend the logic to optionally track *all* annotations, with a flag -XX:+CheckInaccessibleAnnotations or some such. Would require an extra growable array off to the side. The benefit would be a debugging tool that would detect a general superset of the visibility anomalies that the JVM needs to detect. This touches more on the language's binary compatibility story for annotations, not so much the JVM. It's maybe worth a tracking bug so folks can vote for it. From shafi.s.ahmad at oracle.com Thu Feb 23 09:28:49 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 23 Feb 2017 01:28:49 -0800 (PST) Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <56a78bd9-bac8-4fff-a1bf-1417e709426f@default> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> <56a78bd9-bac8-4fff-a1bf-1417e709426f@default> Message-ID: Hi Coleen, David, This is reviewed for jdk10 but when I sent for push to one of my colleague he has suggested me to push is jdk9 and this will automatically pushed to jdk10. So can this be pushed this to jdk9? If yes should I sent a separate review request or current review is sufficient? Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Thursday, February 16, 2017 1:56 PM > To: Coleen Phillimore ; hotspot- > dev at openjdk.java.net > Subject: RE: jdk10 RFR for JDK-8167423: Incorrect implementation of > JDK_Version::to_string OR proper return statement is missing OR proper > comment is needed > > Hi Coleen, > > Thank you for the review. > > Regards, > Shafi > > > -----Original Message----- > > From: Coleen Phillimore > > Sent: Thursday, February 16, 2017 2:09 AM > > To: hotspot-dev at openjdk.java.net > > Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of > > JDK_Version::to_string OR proper return statement is missing OR proper > > comment is needed > > > > Shafi, > > This looks good to me also. Thank you for fixing this. > > Coleen > > > > On 2/15/17 6:35 AM, Shafi Ahmad wrote: > > > Hi All, > > > > > > Summary: Adding return value check and update index variable. It's > > > a very > > small change to single file. > > > > > > Webrev link: > http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ > > > bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 > > > > > > Testing: jprt and jtreg test. > > > > > > Regards, > > > Shafi > > From david.holmes at oracle.com Thu Feb 23 11:03:51 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Feb 2017 21:03:51 +1000 Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> <56a78bd9-bac8-4fff-a1bf-1417e709426f@default> Message-ID: <299f582b-693f-ddaa-3ce1-3c838cad7d1d@oracle.com> Hi Shafi, On 23/02/2017 7:28 PM, Shafi Ahmad wrote: > Hi Coleen, David, > > This is reviewed for jdk10 but when I sent for push to one of my colleague he has suggested me to push is jdk9 and this will automatically pushed to jdk10. > > So can this be pushed this to jdk9? If yes should I sent a separate review request or current review is sufficient? No - this is a P4 bug and we are in RDP1 for JDK 9 so this can not be pushed to 9 unless it goes through a specific critical approval process. David ----- > Regards, > Shafi > >> -----Original Message----- >> From: Shafi Ahmad >> Sent: Thursday, February 16, 2017 1:56 PM >> To: Coleen Phillimore ; hotspot- >> dev at openjdk.java.net >> Subject: RE: jdk10 RFR for JDK-8167423: Incorrect implementation of >> JDK_Version::to_string OR proper return statement is missing OR proper >> comment is needed >> >> Hi Coleen, >> >> Thank you for the review. >> >> Regards, >> Shafi >> >>> -----Original Message----- >>> From: Coleen Phillimore >>> Sent: Thursday, February 16, 2017 2:09 AM >>> To: hotspot-dev at openjdk.java.net >>> Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of >>> JDK_Version::to_string OR proper return statement is missing OR proper >>> comment is needed >>> >>> Shafi, >>> This looks good to me also. Thank you for fixing this. >>> Coleen >>> >>> On 2/15/17 6:35 AM, Shafi Ahmad wrote: >>>> Hi All, >>>> >>>> Summary: Adding return value check and update index variable. It's >>>> a very >>> small change to single file. >>>> >>>> Webrev link: >> http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ >>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 >>>> >>>> Testing: jprt and jtreg test. >>>> >>>> Regards, >>>> Shafi >>> From shafi.s.ahmad at oracle.com Thu Feb 23 11:46:06 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 23 Feb 2017 03:46:06 -0800 (PST) Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <299f582b-693f-ddaa-3ce1-3c838cad7d1d@oracle.com> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> <56a78bd9-bac8-4fff-a1bf-1417e709426f@default> <299f582b-693f-ddaa-3ce1-3c838cad7d1d@oracle.com> Message-ID: <01839b03-fe02-4df1-b4e9-7f18c178fcf2@default> Hi David, Thanks for the input. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Thursday, February 23, 2017 4:34 PM > To: Shafi Ahmad ; Coleen Phillimore > > Cc: hotspot-dev at openjdk.java.net > Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of > JDK_Version::to_string OR proper return statement is missing OR proper > comment is needed > > Hi Shafi, > > On 23/02/2017 7:28 PM, Shafi Ahmad wrote: > > Hi Coleen, David, > > > > This is reviewed for jdk10 but when I sent for push to one of my colleague > he has suggested me to push is jdk9 and this will automatically pushed to > jdk10. > > > > So can this be pushed this to jdk9? If yes should I sent a separate review > request or current review is sufficient? > > No - this is a P4 bug and we are in RDP1 for JDK 9 so this can not be pushed to > 9 unless it goes through a specific critical approval process. > > David > ----- > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: Shafi Ahmad > >> Sent: Thursday, February 16, 2017 1:56 PM > >> To: Coleen Phillimore ; hotspot- > >> dev at openjdk.java.net > >> Subject: RE: jdk10 RFR for JDK-8167423: Incorrect implementation of > >> JDK_Version::to_string OR proper return statement is missing OR > >> proper comment is needed > >> > >> Hi Coleen, > >> > >> Thank you for the review. > >> > >> Regards, > >> Shafi > >> > >>> -----Original Message----- > >>> From: Coleen Phillimore > >>> Sent: Thursday, February 16, 2017 2:09 AM > >>> To: hotspot-dev at openjdk.java.net > >>> Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of > >>> JDK_Version::to_string OR proper return statement is missing OR > >>> proper comment is needed > >>> > >>> Shafi, > >>> This looks good to me also. Thank you for fixing this. > >>> Coleen > >>> > >>> On 2/15/17 6:35 AM, Shafi Ahmad wrote: > >>>> Hi All, > >>>> > >>>> Summary: Adding return value check and update index variable. It's > >>>> a very > >>> small change to single file. > >>>> > >>>> Webrev link: > >> http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ > >>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 > >>>> > >>>> Testing: jprt and jtreg test. > >>>> > >>>> Regards, > >>>> Shafi > >>> From mikael.gerdin at oracle.com Thu Feb 23 13:29:29 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 23 Feb 2017 14:29:29 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> Message-ID: <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> Hi Kim, Thanks for the prompt review! On 2017-02-22 20:40, Kim Barrett wrote: >> On Feb 22, 2017, at 11:07 AM, Mikael Gerdin wrote: >> >> Hi all, >> Please review this revised change to jweak to support G1. >> >> I've copied Kim's original description of the fix below for reference: >> >>> >>> The problem being addressed is that, when using G1, dereferencing a >>> jweak must ensure the obtained referent is ensured live, just as for >>> other weak reference types. This has always been a requirement, and >>> failure to do so appears to be a since-day1 G1 bug. >>> >>> There are two categories of places that need to address this issue: >>> JNIHandles::resolve and friends, and returning a jobject from native >>> code to Java. In both of these places the jobject whose contained oop >>> is being extracted might really be a jweak. jweak is >>> representationally indistinguishable from jobject; jweak is just a >>> typedef for jobject. The documentation says a jweak can be used >>> anywhere a jobject is permitted, making that type equivalence >>> necessary for a C API. (A C++ API might be able to have distinct >>> types via inheritance, though would still need a method for >>> distinguishing them at runtime. But since JNI is a C API...). >>> >>> The basic approach being taken is to "tag" jweaks by setting the low >>> order bit (recall that jweak == jobject == _jobject*), in order to >>> distinguish them from non-jweak jobjects. This tagging is only being >>> done when the selected GC needs the distinction, e.g. G1. This gives >>> us a representational difference between jobject and jweak on which to >>> base the different behavior, hiding that distinction behind the >>> existing API. >>> >>> JNIHandles::resolve and friends are changed to unconditionally strip >>> off any potential weak tag when translating from jobject to to oop*. >>> That's cheaper than testing for the conditional use of tagging and >>> then stripping. Also stripped when deleting JNI handles. >>> >>> TemplateInterpreterGenerator::generate_native_entry and >>> SharedRuntime::generate_native_wrapper are changed to conditionally >>> emit code to apply the G1 pre-barrier to the value obtained from a >>> jobject result, when the result is tagged as a jweak, which only >>> occurs when using G1. >>> >>> For arm32/arm64, this required moving the g1_write_barrier_pre >>> definitions from InterpreterMacroAssembler to MacroAssembler. Also >>> moved g1_write_barrier_post, for consistency. >>> >> >> In addition to Kim's final version of the change (which was backed out) this updated version adds code to mask out the weak tag bit in the fast JNI field getters on all platforms where fast JNI getters are used. >> >> [?] >> >> Webrevs: >> Kim's initial change: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ >> My additions: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ >> Full webrev: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ >> >> Bugs: >> https://bugs.openjdk.java.net/browse/JDK-8175085 >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> Thanks >> /Mikael > > Mikael - Thanks for taking this up for me. Just a few minor comments. > > ------------------------------------------------------------------------------ > The usual copyright reminder. Updated copyrights. > > ------------------------------------------------------------------------------ > src/cpu/arm/vm/jniFastGetField_arm.cpp > 122 #ifndef AARCH64 > 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); > 124 #else > 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); > 126 #endif > > Usual style when selecting between AARCH64 and not seems to be to put > the AARCH64 code first, e.g. > > #ifdef AARCH64 > ... > #else > ... > #endif Fixed. > > ------------------------------------------------------------------------------ > src/cpu/x86/vm/jniFastGetField_x86_32.cpp > 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); > > Use STATIC_ASSERT rather than assert. > > ------------------------------------------------------------------------------ > src/cpu/x86/vm/jniFastGetField_x86_32.cpp > 89 const intptr_t inverted_jweak_mask = ~static_cast(JNIHandles::weak_tag_mask); > 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); > 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to sign-extension > > > With three occurrences of this snippet in this file and two more > similar ones in the 64-bit file, a macroAssembler helper seems called > for here. > > ------------------------------------------------------------------------------ > src/cpu/x86/vm/jniFastGetField_x86_64.cpp > 84 const intptr_t inverted_jweak_mask = ~static_cast(JNIHandles::weak_tag_mask); > 85 const int32_t truncated_mask = static_cast(inverted_jweak_mask); > > Why the two-step conversion here? Why not just > > const int32_t truncated_mask = static_cast(JNIHandles::weak_tag_mask); I had changed this a bunch of times since I was a bit unsure about how to best do this but when I decided to do the bitwise invert after the cast to signed I forgot to clean up the casts. > > That would also make the 32 and 64-bit code more similar in the > suggested macroAssembler helper. Indeed, I even found MacroAssembler::andptr which makes this a nice and small helper. > > ------------------------------------------------------------------------------ > test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c > 51 #define CHECK(variable, expected) \ > 52 do { if ((variable) != (expected)) { \ > 53 (*env)->ThrowNew(env, exception, #variable" != " #expected); \ > 54 return; \ > 55 } \ > 56 } while(0); > > The formatting of the code in the macro body is strange and confusing. Cleaned up formatting. > > Also the do-while-zero idiom that I'm used to leaves off the final > semicolon in the macro. Instead of the uses are expected to be > terminated with semicolons, as you've done. Oh, oops. I also took the liberty to add STATIC_ASSERTS on 64 bit arm since the encoding of the immediate in the "andr" instruction is a bit "funny". The assembler code only verifies that immediate encoding works in debug builds but in debug builds the fast JNI getters aren't generated. New webrevs: Incremental: http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ Full: http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ Thanks /Mikael > > ------------------------------------------------------------------------------ > From kim.barrett at oracle.com Thu Feb 23 14:50:59 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 23 Feb 2017 09:50:59 -0500 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> Message-ID: > On Feb 23, 2017, at 8:29 AM, Mikael Gerdin wrote:[?] > > New webrevs: > Incremental: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ > Full: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ looks good From mikael.gerdin at oracle.com Thu Feb 23 15:46:48 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 23 Feb 2017 16:46:48 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> Message-ID: On 2017-02-23 15:50, Kim Barrett wrote: >> On Feb 23, 2017, at 8:29 AM, Mikael Gerdin wrote:[?] >> >> New webrevs: >> Incremental: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ >> Full: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ > > looks good > Thanks for the review, Kim! /Mikael From daniel.daugherty at oracle.com Thu Feb 23 16:18:33 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 23 Feb 2017 09:18:33 -0700 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> Message-ID: <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> On 2/23/17 6:29 AM, Mikael Gerdin wrote: > Hi Kim, > > Thanks for the prompt review! > > On 2017-02-22 20:40, Kim Barrett wrote: >>> On Feb 22, 2017, at 11:07 AM, Mikael Gerdin >>> wrote: >>> >>> Hi all, >>> Please review this revised change to jweak to support G1. >>> >>> I've copied Kim's original description of the fix below for reference: >>> >>>> >>>> The problem being addressed is that, when using G1, dereferencing a >>>> jweak must ensure the obtained referent is ensured live, just as for >>>> other weak reference types. This has always been a requirement, and >>>> failure to do so appears to be a since-day1 G1 bug. >>>> >>>> There are two categories of places that need to address this issue: >>>> JNIHandles::resolve and friends, and returning a jobject from native >>>> code to Java. In both of these places the jobject whose contained oop >>>> is being extracted might really be a jweak. jweak is >>>> representationally indistinguishable from jobject; jweak is just a >>>> typedef for jobject. The documentation says a jweak can be used >>>> anywhere a jobject is permitted, making that type equivalence >>>> necessary for a C API. (A C++ API might be able to have distinct >>>> types via inheritance, though would still need a method for >>>> distinguishing them at runtime. But since JNI is a C API...). >>>> >>>> The basic approach being taken is to "tag" jweaks by setting the low >>>> order bit (recall that jweak == jobject == _jobject*), in order to >>>> distinguish them from non-jweak jobjects. This tagging is only being >>>> done when the selected GC needs the distinction, e.g. G1. This gives >>>> us a representational difference between jobject and jweak on which to >>>> base the different behavior, hiding that distinction behind the >>>> existing API. >>>> >>>> JNIHandles::resolve and friends are changed to unconditionally strip >>>> off any potential weak tag when translating from jobject to to oop*. >>>> That's cheaper than testing for the conditional use of tagging and >>>> then stripping. Also stripped when deleting JNI handles. >>>> >>>> TemplateInterpreterGenerator::generate_native_entry and >>>> SharedRuntime::generate_native_wrapper are changed to conditionally >>>> emit code to apply the G1 pre-barrier to the value obtained from a >>>> jobject result, when the result is tagged as a jweak, which only >>>> occurs when using G1. >>>> >>>> For arm32/arm64, this required moving the g1_write_barrier_pre >>>> definitions from InterpreterMacroAssembler to MacroAssembler. Also >>>> moved g1_write_barrier_post, for consistency. >>>> >>> >>> In addition to Kim's final version of the change (which was backed >>> out) this updated version adds code to mask out the weak tag bit in >>> the fast JNI field getters on all platforms where fast JNI getters >>> are used. >>> >>> [?] >>> >>> Webrevs: >>> Kim's initial change: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ >>> My additions: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ >>> Full webrev: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ >>> >>> Bugs: >>> https://bugs.openjdk.java.net/browse/JDK-8175085 >>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>> >>> Thanks >>> /Mikael >> >> Mikael - Thanks for taking this up for me. Just a few minor comments. >> >> ------------------------------------------------------------------------------ >> >> The usual copyright reminder. > > Updated copyrights. > >> >> ------------------------------------------------------------------------------ >> >> src/cpu/arm/vm/jniFastGetField_arm.cpp >> 122 #ifndef AARCH64 >> 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); >> 124 #else >> 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); >> 126 #endif >> >> Usual style when selecting between AARCH64 and not seems to be to put >> the AARCH64 code first, e.g. >> >> #ifdef AARCH64 >> ... >> #else >> ... >> #endif > > Fixed. > >> >> ------------------------------------------------------------------------------ >> >> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >> >> Use STATIC_ASSERT rather than assert. >> >> ------------------------------------------------------------------------------ >> >> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >> 89 const intptr_t inverted_jweak_mask = >> ~static_cast(JNIHandles::weak_tag_mask); >> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >> 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to >> sign-extension >> >> >> With three occurrences of this snippet in this file and two more >> similar ones in the 64-bit file, a macroAssembler helper seems called >> for here. >> >> ------------------------------------------------------------------------------ >> >> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >> 84 const intptr_t inverted_jweak_mask = >> ~static_cast(JNIHandles::weak_tag_mask); >> 85 const int32_t truncated_mask = >> static_cast(inverted_jweak_mask); >> >> Why the two-step conversion here? Why not just >> >> const int32_t truncated_mask = >> static_cast(JNIHandles::weak_tag_mask); > > I had changed this a bunch of times since I was a bit unsure about how > to best do this but when I decided to do the bitwise invert after the > cast to signed I forgot to clean up the casts. > >> >> That would also make the 32 and 64-bit code more similar in the >> suggested macroAssembler helper. > > Indeed, I even found MacroAssembler::andptr which makes this a nice > and small helper. > >> >> ------------------------------------------------------------------------------ >> >> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >> 51 #define CHECK(variable, expected) \ >> 52 do { if ((variable) != (expected)) { \ >> 53 (*env)->ThrowNew(env, exception, #variable" != " #expected); \ >> 54 return; \ >> 55 } \ >> 56 } while(0); >> >> The formatting of the code in the macro body is strange and confusing. > > Cleaned up formatting. > >> >> Also the do-while-zero idiom that I'm used to leaves off the final >> semicolon in the macro. Instead of the uses are expected to be >> terminated with semicolons, as you've done. > > Oh, oops. > > I also took the liberty to add STATIC_ASSERTS on 64 bit arm since the > encoding of the immediate in the "andr" instruction is a bit "funny". > The assembler code only verifies that immediate encoding works in > debug builds but in debug builds the fast JNI getters aren't generated. > > New webrevs: > Incremental: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ > Full: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ I only took a look at these files from the full webrev. These are the files touch in the two incremental webrevs: src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp No comments. src/cpu/arm/vm/jniFastGetField_arm.cpp No comments. src/cpu/sparc/vm/jniFastGetField_sparc.cpp No comments. src/cpu/x86/vm/jniFastGetField_x86_32.cpp No comments. src/cpu/x86/vm/jniFastGetField_x86_64.cpp No comments. src/cpu/x86/vm/macroAssembler_x86.cpp I like the new helper! src/cpu/x86/vm/macroAssembler_x86.hpp No comments. test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java L50 - any reason for the blank line? test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c L32 : Java_CallWithJNIWeak_testJNIFieldAccessors (JNIEnv *env, jclass clazz, jobject this) { L102: Java_CallWithJNIWeak_runTests (JNIEnv *env, jclass clazz, jobject this) { L137: Java_CallWithJNIWeak_weakReceiverTest0 (JNIEnv *env, jobject obj) { Please delete the space before the first '('. You may want to add a comment between the two halves of the test for the literal values that you're setting in the .java file and testing for in the .c file. In your (Mikael G) original invite: > Testing: > Standard JPRT and tier2-3 HotSpot tests. Does tier2 and/or tier3 cover the tests that failed in Mach5? > I've modified the CallWithJNIWeak test to call all the primitive > getters and some other interesting cases I came up with. > I've also run the CallWithJNIWeak test on all platforms on both > product and fastdebug builds (since fast JNI getters are implicitly > disabled in debug builds due to VerifyJNIFields being enabled by > default in debug builds. Great coverage with the new tests! > I've not built or tested the aarch64 port but I think it's correct > and I hope someone can test it for me. Have you heard back from anyone on aarch64? I haven't seen any e-mail on this OpenJDK thread... Thumbs up! Dan > > Thanks > /Mikael > >> >> ------------------------------------------------------------------------------ >> >> From erik.helin at oracle.com Thu Feb 23 18:09:37 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 23 Feb 2017 19:09:37 +0100 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> Message-ID: <81bf6cf4-18ad-1da6-3399-2260da7de812@oracle.com> On 02/21/2017 04:49 PM, Erik Helin wrote: > On 02/21/2017 02:57 AM, Kim Barrett wrote: >>> On Feb 20, 2017, at 5:52 PM, David Holmes >>> wrote: >>>> 02 -> 03: >>>> - Only use load_acquire when reading the size for the "head" chunk >>> >>> Sorry but I don't like the way this is done - the conditional may end >>> up being more expensive than the unnecessary load-acquire. Unrolling >>> the first loop iteration, as per the email discussion, is a better >>> way to go IMO. >> >> +1 > > David, Kim, please see new patches at: > - inc: http://cr.openjdk.java.net/~ehelin/8168914/03-04/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/04/ I got a few comments from StefanK offline so I figured that I might as well create new patch including other comments as well. Please see new patches at: - inc: http://cr.openjdk.java.net/~ehelin/8168914/04-05/ - full: http://cr.openjdk.java.net/~ehelin/8168914/05/ The changes made to version 04 includes: - Prefix all fields with underscore (StefanK) - Rename all arguments of type oop* to `p` (StefanK) - Group fields together in Chunk (StefanK) - Make oops_do_chunk private (Kim, David, Thomas) - Remove casts to intptr_t (Kim, David, Thomas) Thanks everyone for the thorough reviewing! Erik > Thanks, > Erik From vladimir.kozlov at oracle.com Thu Feb 23 18:16:45 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 23 Feb 2017 10:16:45 -0800 Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <01839b03-fe02-4df1-b4e9-7f18c178fcf2@default> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> <56a78bd9-bac8-4fff-a1bf-1417e709426f@default> <299f582b-693f-ddaa-3ce1-3c838cad7d1d@oracle.com> <01839b03-fe02-4df1-b4e9-7f18c178fcf2@default> Message-ID: <70ad3ad6-a60b-6107-7b90-74ed15df045c@oracle.com> Shafi If you think that this bug has to be fixed in jdk 9 you can rise its priority (P4 now) with explanation in the bug report and adding 9-critical-watch label. We can still push P1-P3 bugs without RT approval until RDP2 starts - March 16. Thanks, Vladimir On 2/23/17 3:46 AM, Shafi Ahmad wrote: > Hi David, > > Thanks for the input. > > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Thursday, February 23, 2017 4:34 PM >> To: Shafi Ahmad ; Coleen Phillimore >> >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of >> JDK_Version::to_string OR proper return statement is missing OR proper >> comment is needed >> >> Hi Shafi, >> >> On 23/02/2017 7:28 PM, Shafi Ahmad wrote: >>> Hi Coleen, David, >>> >>> This is reviewed for jdk10 but when I sent for push to one of my colleague >> he has suggested me to push is jdk9 and this will automatically pushed to >> jdk10. >>> >>> So can this be pushed this to jdk9? If yes should I sent a separate review >> request or current review is sufficient? >> >> No - this is a P4 bug and we are in RDP1 for JDK 9 so this can not be pushed to >> 9 unless it goes through a specific critical approval process. >> >> David >> ----- >> >>> Regards, >>> Shafi >>> >>>> -----Original Message----- >>>> From: Shafi Ahmad >>>> Sent: Thursday, February 16, 2017 1:56 PM >>>> To: Coleen Phillimore ; hotspot- >>>> dev at openjdk.java.net >>>> Subject: RE: jdk10 RFR for JDK-8167423: Incorrect implementation of >>>> JDK_Version::to_string OR proper return statement is missing OR >>>> proper comment is needed >>>> >>>> Hi Coleen, >>>> >>>> Thank you for the review. >>>> >>>> Regards, >>>> Shafi >>>> >>>>> -----Original Message----- >>>>> From: Coleen Phillimore >>>>> Sent: Thursday, February 16, 2017 2:09 AM >>>>> To: hotspot-dev at openjdk.java.net >>>>> Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of >>>>> JDK_Version::to_string OR proper return statement is missing OR >>>>> proper comment is needed >>>>> >>>>> Shafi, >>>>> This looks good to me also. Thank you for fixing this. >>>>> Coleen >>>>> >>>>> On 2/15/17 6:35 AM, Shafi Ahmad wrote: >>>>>> Hi All, >>>>>> >>>>>> Summary: Adding return value check and update index variable. It's >>>>>> a very >>>>> small change to single file. >>>>>> >>>>>> Webrev link: >>>> http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ >>>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 >>>>>> >>>>>> Testing: jprt and jtreg test. >>>>>> >>>>>> Regards, >>>>>> Shafi >>>>> From coleen.phillimore at oracle.com Thu Feb 23 20:14:09 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 23 Feb 2017 15:14:09 -0500 Subject: jdk10 RFR for JDK-8167423: Incorrect implementation of JDK_Version::to_string OR proper return statement is missing OR proper comment is needed In-Reply-To: <01839b03-fe02-4df1-b4e9-7f18c178fcf2@default> References: <542ce8a3-b975-4a17-bcfc-bb5f17203c26@default> <1edf99e0-f4c8-4643-4b5f-78f23b5f5b08@oracle.com> <56a78bd9-bac8-4fff-a1bf-1417e709426f@default> <299f582b-693f-ddaa-3ce1-3c838cad7d1d@oracle.com> <01839b03-fe02-4df1-b4e9-7f18c178fcf2@default> Message-ID: Yes, agree, this should be in JDK10. thanks, Coleen On 2/23/17 6:46 AM, Shafi Ahmad wrote: > Hi David, > > Thanks for the input. > > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Thursday, February 23, 2017 4:34 PM >> To: Shafi Ahmad ; Coleen Phillimore >> >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of >> JDK_Version::to_string OR proper return statement is missing OR proper >> comment is needed >> >> Hi Shafi, >> >> On 23/02/2017 7:28 PM, Shafi Ahmad wrote: >>> Hi Coleen, David, >>> >>> This is reviewed for jdk10 but when I sent for push to one of my colleague >> he has suggested me to push is jdk9 and this will automatically pushed to >> jdk10. >>> So can this be pushed this to jdk9? If yes should I sent a separate review >> request or current review is sufficient? >> >> No - this is a P4 bug and we are in RDP1 for JDK 9 so this can not be pushed to >> 9 unless it goes through a specific critical approval process. >> >> David >> ----- >> >>> Regards, >>> Shafi >>> >>>> -----Original Message----- >>>> From: Shafi Ahmad >>>> Sent: Thursday, February 16, 2017 1:56 PM >>>> To: Coleen Phillimore ; hotspot- >>>> dev at openjdk.java.net >>>> Subject: RE: jdk10 RFR for JDK-8167423: Incorrect implementation of >>>> JDK_Version::to_string OR proper return statement is missing OR >>>> proper comment is needed >>>> >>>> Hi Coleen, >>>> >>>> Thank you for the review. >>>> >>>> Regards, >>>> Shafi >>>> >>>>> -----Original Message----- >>>>> From: Coleen Phillimore >>>>> Sent: Thursday, February 16, 2017 2:09 AM >>>>> To: hotspot-dev at openjdk.java.net >>>>> Subject: Re: jdk10 RFR for JDK-8167423: Incorrect implementation of >>>>> JDK_Version::to_string OR proper return statement is missing OR >>>>> proper comment is needed >>>>> >>>>> Shafi, >>>>> This looks good to me also. Thank you for fixing this. >>>>> Coleen >>>>> >>>>> On 2/15/17 6:35 AM, Shafi Ahmad wrote: >>>>>> Hi All, >>>>>> >>>>>> Summary: Adding return value check and update index variable. It's >>>>>> a very >>>>> small change to single file. >>>>>> Webrev link: >>>> http://cr.openjdk.java.net/~shshahma/8167423/webrev.00/ >>>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8167423 >>>>>> >>>>>> Testing: jprt and jtreg test. >>>>>> >>>>>> Regards, >>>>>> Shafi From max.ockner at oracle.com Thu Feb 23 21:21:23 2017 From: max.ockner at oracle.com (Max Ockner) Date: Thu, 23 Feb 2017 16:21:23 -0500 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> Message-ID: <58AF5253.7020005@oracle.com> Hi Volker, I have attached the patch that I have been testing. Thanks, Max On 2/20/2017 5:45 AM, Volker Simonis wrote: > Hi, > > besides the fact that this of course means some work for us :) I > currently don't see any problems for our porting platforms (ppc64 and > s390x). > > Are there any webrevs available, so we can see how big they are and > maybe do some own benchmarking? > > Thanks, > Volker > > > On Sun, Feb 19, 2017 at 11:11 PM, wrote: >> >> On 2/18/17 11:14 AM, coleen.phillimore at oracle.com wrote: >>> When Max gets back from the long weekend, he'll post the platforms in your >>> bug. >>> >>> It's amazing that for -Xint there's no significant difference. I've seen >>> -Xint performance of 15% slower cause a 2% slowdown with server but that was >>> before tiered compilation. >> >> I should clarify this. I've seen this slowdown for *different* interpreter >> optimizations, which *can* affect server performance. I was measuring >> specjvm98 on linux x64. If there's no significant difference for this TOS >> optimization, there is no chance of a degredation in overall performance. >> >> Coleen >> >>> The reason for this query was to see what developers for the other >>> platform ports think, since this change would affect all of the platforms. >>> >>> Thanks, >>> Coleen >>> >>> On 2/18/17 10:50 AM, Daniel D. Daugherty wrote: >>>> If Claes is happy with the perf testing, then I'm happy. :-) >>>> >>>> Dan >>>> >>>> >>>> On 2/18/17 3:46 AM, Claes Redestad wrote: >>>>> Hi, >>>>> >>>>> I've seen Max has run plenty of tests on our internal performance >>>>> infrastructure and everything I've seen there seems to corroborate the >>>>> idea that this removal is OK from a performance point of view, the >>>>> footprint improvements are small but significant and any negative >>>>> performance impact on throughput benchmarks is at noise levels even >>>>> with -Xint (it appears many benchmarks time out with this setting >>>>> both before and after, though; Max, let's discuss offline how to >>>>> deal with that :-)) >>>>> >>>>> I expect this will be tested more thoroughly once adapted to all >>>>> platforms (which I assume is the intent?), but see no concern from >>>>> a performance testing point of view: Do it! >>>>> >>>>> Thanks! >>>>> >>>>> /Claes >>>>> >>>>> On 2017-02-16 16:40, Daniel D. Daugherty wrote: >>>>>> Hi Max, >>>>>> >>>>>> Added a note to your bug. Interesting idea, but I think your data is >>>>>> a bit incomplete at the moment. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 2/15/17 3:18 PM, Max Ockner wrote: >>>>>>> Hello all, >>>>>>> >>>>>>> We have filed a bug to remove the interpreter stack caching >>>>>>> optimization for jdk10. Ideally we can make this change *early* >>>>>>> during the jdk10 development cycle. See below for justification: >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>>>>>> >>>>>>> Stack caching has been around for a long time and is intended to >>>>>>> replace some of the load/store (pop/push) operations with >>>>>>> corresponding register operations. The need for this optimization >>>>>>> arose before caching could adequately lessen the burden of memory >>>>>>> access. We have reevaluated the JVM stack caching optimization and >>>>>>> have found that it has a high memory footprint and is very costly to >>>>>>> maintain, but does not provide significant measurable or theoretical >>>>>>> benefit for us when used with modern hardware. >>>>>>> >>>>>>> Minimal Theoretical Benefit. >>>>>>> Because modern hardware does not slap us with the same cost for >>>>>>> accessing memory as it once did, the benefit of replacing memory >>>>>>> access with register access is far less dramatic now than it once was. >>>>>>> Additionally, the interpreter runs for a relatively short time before >>>>>>> relevant code sections are compiled. When the VM starts running >>>>>>> compiled code instead of interpreted code, performance should begin to >>>>>>> move asymptotically towards that of compiled code, diluting any >>>>>>> performance penalties from the interpreter to small performance >>>>>>> variations. >>>>>>> >>>>>>> No Measurable Benefit. >>>>>>> Please see the results files attached in the bug page. This change >>>>>>> was adapted for x86 and sparc, and interpreter performance was >>>>>>> measured with Specjvm98 (run with -Xint). No significant decrease in >>>>>>> performance was observed. >>>>>>> >>>>>>> Memory footprint and code complexity. >>>>>>> Stack caching in the JVM is implemented by switching the instruction >>>>>>> look-up table depending on the tos (top-of-stack) state. At any moment >>>>>>> there are is an active table consisting of one dispatch table for each >>>>>>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>>>>>> safepoint dispatch tables into the active table. The additional entry >>>>>>> code makes this copy less efficient and makes any work in the >>>>>>> interpreter harder to debug. >>>>>>> >>>>>>> If we remove this optimization, we will: >>>>>>> - decrease memory usage in the interpreter, >>>>>>> - eliminated wasteful memory transactions during safepoints, >>>>>>> - decrease code complexity (a lot). >>>>>>> >>>>>>> Please let me know what you think. >>>>>>> Thanks, >>>>>>> Max >>>>>>> -------------- next part -------------- diff --git a/src/cpu/sparc/vm/interp_masm_sparc.cpp b/src/cpu/sparc/vm/interp_masm_sparc.cpp --- a/src/cpu/sparc/vm/interp_masm_sparc.cpp +++ b/src/cpu/sparc/vm/interp_masm_sparc.cpp @@ -90,9 +90,16 @@ #else ldub( Lbcp, bcp_incr, Lbyte_code); // load next bytecode // dispatch table to use + + if (!EnableTosCache){ + AddressLiteral tbl(Interpreter::dispatch_table(vtos)); + sll(Lbyte_code, LogBytesPerWord, Lbyte_code); // multiply by wordSize + set(tbl, G3_scratch); // compute addr of table + } else { AddressLiteral tbl(Interpreter::dispatch_table(state)); sll(Lbyte_code, LogBytesPerWord, Lbyte_code); // multiply by wordSize set(tbl, G3_scratch); // compute addr of table + } ld_ptr(G3_scratch, Lbyte_code, IdispatchAddress); // get entry addr #endif } @@ -105,6 +112,10 @@ assert_not_delayed(); verify_FPU(1, state); interp_verify_oop(Otos_i, state, __FILE__, __LINE__); + + if (!EnableTosCache){ + push(state); + } jmp( IdispatchAddress, 0 ); if (bcp_incr != 0) delayed()->inc(Lbcp, bcp_incr); else delayed()->nop(); diff --git a/src/cpu/x86/vm/interp_masm_x86.cpp b/src/cpu/x86/vm/interp_masm_x86.cpp --- a/src/cpu/x86/vm/interp_masm_x86.cpp +++ b/src/cpu/x86/vm/interp_masm_x86.cpp @@ -796,7 +796,25 @@ } void InterpreterMacroAssembler::dispatch_epilog(TosState state, int step) { + if (EnableTosCache) { dispatch_next(state, step); + } else { + switch (state) { + case btos: + case ctos: + case stos: + ShouldNotReachHere(); // btos/ctos/stos should use itos. + break; + case atos: push(atos); break; + case itos: push(itos); break; + case ltos: push(ltos); break; + case ftos: push(ftos); break; + case dtos: push(dtos); break; + case vtos: break; + default : ShouldNotReachHere(); break; + } + dispatch_next(vtos, step); + } } void InterpreterMacroAssembler::dispatch_base(TosState state, diff --git a/src/share/vm/runtime/globals.hpp b/src/share/vm/runtime/globals.hpp --- a/src/share/vm/runtime/globals.hpp +++ b/src/share/vm/runtime/globals.hpp @@ -2675,6 +2675,8 @@ "Produce histogram of IC misses") \ \ /* interpreter */ \ + product(bool, EnableTosCache, true, \ + "Enable Top of Stack Cache optimization") \ product_pd(bool, RewriteBytecodes, \ "Allow rewriting of bytecodes (bytecodes are not immutable)") \ \ @@ -2776,7 +2778,7 @@ "Test only") \ \ /* compilation */ \ - product(bool, UseCompiler, true, \ + product(bool, UseCompiler, false, \ "Use Just-In-Time compilation") \ \ develop(bool, TraceCompilationPolicy, false, \ From david.holmes at oracle.com Fri Feb 24 00:16:43 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 24 Feb 2017 10:16:43 +1000 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <81bf6cf4-18ad-1da6-3399-2260da7de812@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> <81bf6cf4-18ad-1da6-3399-2260da7de812@oracle.com> Message-ID: Hi Erik, No further comments from me. Thanks, David On 24/02/2017 4:09 AM, Erik Helin wrote: > On 02/21/2017 04:49 PM, Erik Helin wrote: >> On 02/21/2017 02:57 AM, Kim Barrett wrote: >>>> On Feb 20, 2017, at 5:52 PM, David Holmes >>>> wrote: >>>>> 02 -> 03: >>>>> - Only use load_acquire when reading the size for the "head" chunk >>>> >>>> Sorry but I don't like the way this is done - the conditional may end >>>> up being more expensive than the unnecessary load-acquire. Unrolling >>>> the first loop iteration, as per the email discussion, is a better >>>> way to go IMO. >>> >>> +1 >> >> David, Kim, please see new patches at: >> - inc: http://cr.openjdk.java.net/~ehelin/8168914/03-04/ >> - full: http://cr.openjdk.java.net/~ehelin/8168914/04/ > > I got a few comments from StefanK offline so I figured that I might as > well create new patch including other comments as well. > > Please see new patches at: > - inc: http://cr.openjdk.java.net/~ehelin/8168914/04-05/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/05/ > > The changes made to version 04 includes: > - Prefix all fields with underscore (StefanK) > - Rename all arguments of type oop* to `p` (StefanK) > - Group fields together in Chunk (StefanK) > - Make oops_do_chunk private (Kim, David, Thomas) > - Remove casts to intptr_t (Kim, David, Thomas) > > Thanks everyone for the thorough reviewing! > Erik > >> Thanks, >> Erik From ioi.lam at oracle.com Fri Feb 24 03:47:03 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 23 Feb 2017 19:47:03 -0800 Subject: Determining the size of C++ vtables Message-ID: <58AFACB7.7020203@oracle.com> Hi, I am working on https://bugs.openjdk.java.net/browse/JDK-8005165 (Remove CPU-dependent code in self-patching vtables), I need a way find out the size of a C++ vtable. I ended up doing this: // Objects of the Metadata types (such as Klass and ConstantPool) have C++ vtables. // (In GCC this is the field ::_vptr, i.e., first word in the object.) // // Addresses of the vtables and the methods may be different across JVM runs, // if libjvm.so is dynamically loaded at a different base address. // // To ensure that the Metadata objects in the CDS archive always have the correct vtable: // // + at dump time: we redirect the _vptr to point to our own vtables inside // the CDS image // + at run time: we clone the actual contents of the vtables from libjvm.so // into our own tables. // // To determine the size of the vtable for each type, we use the following // trick by declaring 2 subclasses: // // class CppVtabTesterA: public InstanceKlass { // virtual int last_virtual_method() {return 1;} // }; // class CppVtabTesterB: public InstanceKlass { // virtual void* last_virtual_method() {return NULL}; // }; // // CppVtabTesterA and CppVtabTesterB's vtables have the following properties: // - Their size (N+1) is exactly one more than the size of InstanceKlass's vtable (N) // - The first N entries have are exactly the same as in InstanceKlass's vtable. // - Their last entry is different. // // So to determine the value of N, we just walk CppVtabTesterA and CppVtabTesterB's tables // and find the first entry that's different Could anyone comment if this is acceptable? I know it's not 100% portable (C++ doesn't specify where to find the vtable, or what's inside), but my assumptions is the same as the existing code. I.e., _vptr is a pointer located at offset 0 of the object, and it points to a one-dimensional array. So at least it's not any worse than before? Thanks - Ioi From ioi.lam at oracle.com Fri Feb 24 03:55:26 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 23 Feb 2017 19:55:26 -0800 Subject: Determining the size of C++ vtables In-Reply-To: <58AFACB7.7020203@oracle.com> References: <58AFACB7.7020203@oracle.com> Message-ID: <58AFAEAE.2080205@oracle.com> On 2/23/17 7:47 PM, Ioi Lam wrote: > Hi, > > I am working on https://bugs.openjdk.java.net/browse/JDK-8005165 (Remove > CPU-dependent code in self-patching vtables), I need a way find out > the size > of a C++ vtable. I ended up doing this: > > > // Objects of the Metadata types (such as Klass and ConstantPool) have > C++ vtables. > // (In GCC this is the field ::_vptr, i.e., first word in the > object.) > // > // Addresses of the vtables and the methods may be different across > JVM runs, > // if libjvm.so is dynamically loaded at a different base address. > // > // To ensure that the Metadata objects in the CDS archive always have > the correct vtable: > // > // + at dump time: we redirect the _vptr to point to our own vtables > inside > // the CDS image > // + at run time: we clone the actual contents of the vtables from > libjvm.so > // into our own tables. > // > // To determine the size of the vtable for each type, we use the > following > // trick by declaring 2 subclasses: > // > // class CppVtabTesterA: public InstanceKlass { > // virtual int last_virtual_method() {return 1;} > // }; > // class CppVtabTesterB: public InstanceKlass { > // virtual void* last_virtual_method() {return NULL}; > // }; > // > // CppVtabTesterA and CppVtabTesterB's vtables have the following > properties: > // - Their size (N+1) is exactly one more than the size of > InstanceKlass's vtable (N) > // - The first N entries have are exactly the same as in > InstanceKlass's vtable. > // - Their last entry is different. > // > // So to determine the value of N, we just walk CppVtabTesterA and > CppVtabTesterB's tables > // and find the first entry that's different > > > Could anyone comment if this is acceptable? I know it's not 100% > portable (C++ doesn't > specify where to find the vtable, or what's inside), but my > assumptions is the same as > the existing code. I.e., _vptr is a pointer located at offset 0 of the > object, and it > points to a one-dimensional array. > > So at least it's not any worse than before? > > Thanks > - Ioi > By the way, I first tried having only a single "tester" class and walk the vtable to look for &last_virtual_method, but the C++ compiler told me that taking the address of a non-static function is not allowed ..... so I ended up creating two tester classes and checking their differences. From kim.barrett at oracle.com Fri Feb 24 04:07:44 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 23 Feb 2017 23:07:44 -0500 Subject: RFR: 8168914: Crash in ClassLoaderData/JNIHandleBlock::oops_do during concurrent marking In-Reply-To: <81bf6cf4-18ad-1da6-3399-2260da7de812@oracle.com> References: <2c401213-c69c-df61-2020-5583fb36bd87@oracle.com> <44592cae-ae85-6468-5af7-18cd61ca7537@oracle.com> <77BE1289-B1F8-4077-9A3B-1F484E6413E4@oracle.com> <4ebc153c-88f4-1501-6b9c-0d927336fcbe@oracle.com> <81bf6cf4-18ad-1da6-3399-2260da7de812@oracle.com> Message-ID: > On Feb 23, 2017, at 1:09 PM, Erik Helin wrote: > > On 02/21/2017 04:49 PM, Erik Helin wrote: >> On 02/21/2017 02:57 AM, Kim Barrett wrote: >>>> On Feb 20, 2017, at 5:52 PM, David Holmes >>>> wrote: >>>>> 02 -> 03: >>>>> - Only use load_acquire when reading the size for the "head" chunk >>>> >>>> Sorry but I don't like the way this is done - the conditional may end >>>> up being more expensive than the unnecessary load-acquire. Unrolling >>>> the first loop iteration, as per the email discussion, is a better >>>> way to go IMO. >>> >>> +1 >> >> David, Kim, please see new patches at: >> - inc: http://cr.openjdk.java.net/~ehelin/8168914/03-04/ >> - full: http://cr.openjdk.java.net/~ehelin/8168914/04/ > > I got a few comments from StefanK offline so I figured that I might as > well create new patch including other comments as well. > > Please see new patches at: > - inc: http://cr.openjdk.java.net/~ehelin/8168914/04-05/ > - full: http://cr.openjdk.java.net/~ehelin/8168914/05/ > > The changes made to version 04 includes: > - Prefix all fields with underscore (StefanK) hotspot seems relatively evenly split on underscore prefixes for public data members. other codebases I?ve seen tend toward unqualified. style guide does say underscore prefix without mentioning exceptions. > - Rename all arguments of type oop* to `p` (StefanK) > - Group fields together in Chunk (StefanK) > - Make oops_do_chunk private (Kim, David, Thomas) > - Remove casts to intptr_t (Kim, David, Thomas) > > Thanks everyone for the thorough reviewing! > Erik > >> Thanks, >> Erik Still looks good. From thomas.stuefe at gmail.com Fri Feb 24 07:17:28 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 24 Feb 2017 08:17:28 +0100 Subject: Determining the size of C++ vtables In-Reply-To: <58AFAEAE.2080205@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> Message-ID: Hi Ioi, After reading through the original mailthread referenced in the bug report, the only reason the "just copy enough (200) slots from the vtable in libjvm.so to the vtable in the image" approach does not work for you is that you are afraid of hitting unmapped memory? If I understand it correctly. If that is the case, why not copy 200 slots with SafeFetch and stop at the first error? That should not incur much additional costs as long as you do not hit an unmapped area, in which case you'd stop anyway. That way you make yourself a bit less dependent on compiler internals. I agree that your approach is more elegant, though. Kind Regards, Thomas On Fri, Feb 24, 2017 at 4:55 AM, Ioi Lam wrote: > > > On 2/23/17 7:47 PM, Ioi Lam wrote: > >> Hi, >> >> I am working on https://bugs.openjdk.java.net/browse/JDK-8005165 (Remove >> CPU-dependent code in self-patching vtables), I need a way find out the >> size >> of a C++ vtable. I ended up doing this: >> >> >> // Objects of the Metadata types (such as Klass and ConstantPool) have >> C++ vtables. >> // (In GCC this is the field ::_vptr, i.e., first word in the >> object.) >> // >> // Addresses of the vtables and the methods may be different across JVM >> runs, >> // if libjvm.so is dynamically loaded at a different base address. >> // >> // To ensure that the Metadata objects in the CDS archive always have the >> correct vtable: >> // >> // + at dump time: we redirect the _vptr to point to our own vtables >> inside >> // the CDS image >> // + at run time: we clone the actual contents of the vtables from >> libjvm.so >> // into our own tables. >> // >> // To determine the size of the vtable for each type, we use the following >> // trick by declaring 2 subclasses: >> // >> // class CppVtabTesterA: public InstanceKlass { >> // virtual int last_virtual_method() {return 1;} >> // }; >> // class CppVtabTesterB: public InstanceKlass { >> // virtual void* last_virtual_method() {return NULL}; >> // }; >> // >> // CppVtabTesterA and CppVtabTesterB's vtables have the following >> properties: >> // - Their size (N+1) is exactly one more than the size of >> InstanceKlass's vtable (N) >> // - The first N entries have are exactly the same as in InstanceKlass's >> vtable. >> // - Their last entry is different. >> // >> // So to determine the value of N, we just walk CppVtabTesterA and >> CppVtabTesterB's tables >> // and find the first entry that's different >> >> >> Could anyone comment if this is acceptable? I know it's not 100% portable >> (C++ doesn't >> specify where to find the vtable, or what's inside), but my assumptions >> is the same as >> the existing code. I.e., _vptr is a pointer located at offset 0 of the >> object, and it >> points to a one-dimensional array. >> >> So at least it's not any worse than before? >> >> Thanks >> - Ioi >> >> By the way, I first tried having only a single "tester" class and walk > the vtable to look for &last_virtual_method, but the C++ compiler told me > that taking the address of a non-static function is not allowed ..... so I > ended up creating two tester classes and checking their differences. > > > > From mikael.gerdin at oracle.com Fri Feb 24 09:55:32 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 24 Feb 2017 10:55:32 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> Message-ID: <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> Hi Dan, Thanks for looking at this review! On 2017-02-23 17:18, Daniel D. Daugherty wrote: > On 2/23/17 6:29 AM, Mikael Gerdin wrote: >> Hi Kim, >> >> Thanks for the prompt review! >> >> On 2017-02-22 20:40, Kim Barrett wrote: >>>> On Feb 22, 2017, at 11:07 AM, Mikael Gerdin >>>> wrote: >>>> >>>> Hi all, >>>> Please review this revised change to jweak to support G1. >>>> >>>> I've copied Kim's original description of the fix below for reference: >>>> >>>>> >>>>> The problem being addressed is that, when using G1, dereferencing a >>>>> jweak must ensure the obtained referent is ensured live, just as for >>>>> other weak reference types. This has always been a requirement, and >>>>> failure to do so appears to be a since-day1 G1 bug. >>>>> >>>>> There are two categories of places that need to address this issue: >>>>> JNIHandles::resolve and friends, and returning a jobject from native >>>>> code to Java. In both of these places the jobject whose contained oop >>>>> is being extracted might really be a jweak. jweak is >>>>> representationally indistinguishable from jobject; jweak is just a >>>>> typedef for jobject. The documentation says a jweak can be used >>>>> anywhere a jobject is permitted, making that type equivalence >>>>> necessary for a C API. (A C++ API might be able to have distinct >>>>> types via inheritance, though would still need a method for >>>>> distinguishing them at runtime. But since JNI is a C API...). >>>>> >>>>> The basic approach being taken is to "tag" jweaks by setting the low >>>>> order bit (recall that jweak == jobject == _jobject*), in order to >>>>> distinguish them from non-jweak jobjects. This tagging is only being >>>>> done when the selected GC needs the distinction, e.g. G1. This gives >>>>> us a representational difference between jobject and jweak on which to >>>>> base the different behavior, hiding that distinction behind the >>>>> existing API. >>>>> >>>>> JNIHandles::resolve and friends are changed to unconditionally strip >>>>> off any potential weak tag when translating from jobject to to oop*. >>>>> That's cheaper than testing for the conditional use of tagging and >>>>> then stripping. Also stripped when deleting JNI handles. >>>>> >>>>> TemplateInterpreterGenerator::generate_native_entry and >>>>> SharedRuntime::generate_native_wrapper are changed to conditionally >>>>> emit code to apply the G1 pre-barrier to the value obtained from a >>>>> jobject result, when the result is tagged as a jweak, which only >>>>> occurs when using G1. >>>>> >>>>> For arm32/arm64, this required moving the g1_write_barrier_pre >>>>> definitions from InterpreterMacroAssembler to MacroAssembler. Also >>>>> moved g1_write_barrier_post, for consistency. >>>>> >>>> >>>> In addition to Kim's final version of the change (which was backed >>>> out) this updated version adds code to mask out the weak tag bit in >>>> the fast JNI field getters on all platforms where fast JNI getters >>>> are used. >>>> >>>> [?] >>>> >>>> Webrevs: >>>> Kim's initial change: >>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ >>>> My additions: >>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ >>>> Full webrev: >>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ >>>> >>>> Bugs: >>>> https://bugs.openjdk.java.net/browse/JDK-8175085 >>>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>>> >>>> Thanks >>>> /Mikael >>> >>> Mikael - Thanks for taking this up for me. Just a few minor comments. >>> >>> ------------------------------------------------------------------------------ >>> >>> The usual copyright reminder. >> >> Updated copyrights. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/cpu/arm/vm/jniFastGetField_arm.cpp >>> 122 #ifndef AARCH64 >>> 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); >>> 124 #else >>> 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); >>> 126 #endif >>> >>> Usual style when selecting between AARCH64 and not seems to be to put >>> the AARCH64 code first, e.g. >>> >>> #ifdef AARCH64 >>> ... >>> #else >>> ... >>> #endif >> >> Fixed. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>> >>> Use STATIC_ASSERT rather than assert. >>> >>> ------------------------------------------------------------------------------ >>> >>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>> 89 const intptr_t inverted_jweak_mask = >>> ~static_cast(JNIHandles::weak_tag_mask); >>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>> 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to >>> sign-extension >>> >>> >>> With three occurrences of this snippet in this file and two more >>> similar ones in the 64-bit file, a macroAssembler helper seems called >>> for here. >>> >>> ------------------------------------------------------------------------------ >>> >>> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >>> 84 const intptr_t inverted_jweak_mask = >>> ~static_cast(JNIHandles::weak_tag_mask); >>> 85 const int32_t truncated_mask = >>> static_cast(inverted_jweak_mask); >>> >>> Why the two-step conversion here? Why not just >>> >>> const int32_t truncated_mask = >>> static_cast(JNIHandles::weak_tag_mask); >> >> I had changed this a bunch of times since I was a bit unsure about how >> to best do this but when I decided to do the bitwise invert after the >> cast to signed I forgot to clean up the casts. >> >>> >>> That would also make the 32 and 64-bit code more similar in the >>> suggested macroAssembler helper. >> >> Indeed, I even found MacroAssembler::andptr which makes this a nice >> and small helper. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >>> 51 #define CHECK(variable, expected) \ >>> 52 do { if ((variable) != (expected)) { \ >>> 53 (*env)->ThrowNew(env, exception, #variable" != " #expected); \ >>> 54 return; \ >>> 55 } \ >>> 56 } while(0); >>> >>> The formatting of the code in the macro body is strange and confusing. >> >> Cleaned up formatting. >> >>> >>> Also the do-while-zero idiom that I'm used to leaves off the final >>> semicolon in the macro. Instead of the uses are expected to be >>> terminated with semicolons, as you've done. >> >> Oh, oops. >> >> I also took the liberty to add STATIC_ASSERTS on 64 bit arm since the >> encoding of the immediate in the "andr" instruction is a bit "funny". >> The assembler code only verifies that immediate encoding works in >> debug builds but in debug builds the fast JNI getters aren't generated. >> >> New webrevs: >> Incremental: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ >> Full: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ > > I only took a look at these files from the full webrev. These are the > files touch in the two incremental webrevs: > > src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp > No comments. > > src/cpu/arm/vm/jniFastGetField_arm.cpp > No comments. > > src/cpu/sparc/vm/jniFastGetField_sparc.cpp > No comments. > > src/cpu/x86/vm/jniFastGetField_x86_32.cpp > No comments. > > src/cpu/x86/vm/jniFastGetField_x86_64.cpp > No comments. > > src/cpu/x86/vm/macroAssembler_x86.cpp > I like the new helper! > > src/cpu/x86/vm/macroAssembler_x86.hpp > No comments. > > test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java > L50 - any reason for the blank line? No reason whatsoever, I'll remove it. > > test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c > L32 : Java_CallWithJNIWeak_testJNIFieldAccessors (JNIEnv *env, > jclass clazz, jobject this) { > L102: Java_CallWithJNIWeak_runTests (JNIEnv *env, jclass clazz, > jobject this) { > L137: Java_CallWithJNIWeak_weakReceiverTest0 (JNIEnv *env, jobject > obj) { > Please delete the space before the first '('. Oh, oops. I'm going to blame javah for that :) > > You may want to add a comment between the two halves of > the test for the literal values that you're setting in > the .java file and testing for in the .c file. Added short comments in the .java and .c files. > > In your (Mikael G) original invite: > >> Testing: >> Standard JPRT and tier2-3 HotSpot tests. > > Does tier2 and/or tier3 cover the tests that failed in Mach5? Somehow I forgot to mention that I also ran jdk_tier3 and the tests that failed in Mach5 did pass on all platforms. > > >> I've modified the CallWithJNIWeak test to call all the primitive >> getters and some other interesting cases I came up with. >> I've also run the CallWithJNIWeak test on all platforms on both >> product and fastdebug builds (since fast JNI getters are implicitly >> disabled in debug builds due to VerifyJNIFields being enabled by >> default in debug builds. > > Great coverage with the new tests! > > >> I've not built or tested the aarch64 port but I think it's correct >> and I hope someone can test it for me. > > Have you heard back from anyone on aarch64? I haven't > seen any e-mail on this OpenJDK thread... I've cc:ed Andrew in this thread to see if I can catch his attention. New webrevs at: http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full3/webrev/ http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental2/webrev/ Thanks /Mikael > > Thumbs up! > > Dan > > >> >> Thanks >> /Mikael >> >>> >>> ------------------------------------------------------------------------------ >>> >>> > From volker.simonis at gmail.com Fri Feb 24 11:37:24 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 24 Feb 2017 12:37:24 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> Message-ID: Hi Mikael, I had a little problems to apply the patch from your webrev because the file actually contains two patches [1] but after fixing that manually, I could still build and run the test on ppc64 and s390x. Thanks, Volker [1] http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/webrev/hotspot.changeset On Fri, Feb 24, 2017 at 10:55 AM, Mikael Gerdin wrote: > Hi Dan, > > Thanks for looking at this review! > > > On 2017-02-23 17:18, Daniel D. Daugherty wrote: >> >> On 2/23/17 6:29 AM, Mikael Gerdin wrote: >>> >>> Hi Kim, >>> >>> Thanks for the prompt review! >>> >>> On 2017-02-22 20:40, Kim Barrett wrote: >>>>> >>>>> On Feb 22, 2017, at 11:07 AM, Mikael Gerdin >>>>> wrote: >>>>> >>>>> Hi all, >>>>> Please review this revised change to jweak to support G1. >>>>> >>>>> I've copied Kim's original description of the fix below for reference: >>>>> >>>>>> >>>>>> The problem being addressed is that, when using G1, dereferencing a >>>>>> jweak must ensure the obtained referent is ensured live, just as for >>>>>> other weak reference types. This has always been a requirement, and >>>>>> failure to do so appears to be a since-day1 G1 bug. >>>>>> >>>>>> There are two categories of places that need to address this issue: >>>>>> JNIHandles::resolve and friends, and returning a jobject from native >>>>>> code to Java. In both of these places the jobject whose contained oop >>>>>> is being extracted might really be a jweak. jweak is >>>>>> representationally indistinguishable from jobject; jweak is just a >>>>>> typedef for jobject. The documentation says a jweak can be used >>>>>> anywhere a jobject is permitted, making that type equivalence >>>>>> necessary for a C API. (A C++ API might be able to have distinct >>>>>> types via inheritance, though would still need a method for >>>>>> distinguishing them at runtime. But since JNI is a C API...). >>>>>> >>>>>> The basic approach being taken is to "tag" jweaks by setting the low >>>>>> order bit (recall that jweak == jobject == _jobject*), in order to >>>>>> distinguish them from non-jweak jobjects. This tagging is only being >>>>>> done when the selected GC needs the distinction, e.g. G1. This gives >>>>>> us a representational difference between jobject and jweak on which to >>>>>> base the different behavior, hiding that distinction behind the >>>>>> existing API. >>>>>> >>>>>> JNIHandles::resolve and friends are changed to unconditionally strip >>>>>> off any potential weak tag when translating from jobject to to oop*. >>>>>> That's cheaper than testing for the conditional use of tagging and >>>>>> then stripping. Also stripped when deleting JNI handles. >>>>>> >>>>>> TemplateInterpreterGenerator::generate_native_entry and >>>>>> SharedRuntime::generate_native_wrapper are changed to conditionally >>>>>> emit code to apply the G1 pre-barrier to the value obtained from a >>>>>> jobject result, when the result is tagged as a jweak, which only >>>>>> occurs when using G1. >>>>>> >>>>>> For arm32/arm64, this required moving the g1_write_barrier_pre >>>>>> definitions from InterpreterMacroAssembler to MacroAssembler. Also >>>>>> moved g1_write_barrier_post, for consistency. >>>>>> >>>>> >>>>> In addition to Kim's final version of the change (which was backed >>>>> out) this updated version adds code to mask out the weak tag bit in >>>>> the fast JNI field getters on all platforms where fast JNI getters >>>>> are used. >>>>> >>>>> [?] >>>>> >>>>> Webrevs: >>>>> Kim's initial change: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ >>>>> My additions: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ >>>>> Full webrev: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ >>>>> >>>>> Bugs: >>>>> https://bugs.openjdk.java.net/browse/JDK-8175085 >>>>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>>>> >>>>> Thanks >>>>> /Mikael >>>> >>>> >>>> Mikael - Thanks for taking this up for me. Just a few minor comments. >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> The usual copyright reminder. >>> >>> >>> Updated copyrights. >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/arm/vm/jniFastGetField_arm.cpp >>>> 122 #ifndef AARCH64 >>>> 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); >>>> 124 #else >>>> 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); >>>> 126 #endif >>>> >>>> Usual style when selecting between AARCH64 and not seems to be to put >>>> the AARCH64 code first, e.g. >>>> >>>> #ifdef AARCH64 >>>> ... >>>> #else >>>> ... >>>> #endif >>> >>> >>> Fixed. >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>> >>>> Use STATIC_ASSERT rather than assert. >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>> 89 const intptr_t inverted_jweak_mask = >>>> ~static_cast(JNIHandles::weak_tag_mask); >>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>> 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to >>>> sign-extension >>>> >>>> >>>> With three occurrences of this snippet in this file and two more >>>> similar ones in the 64-bit file, a macroAssembler helper seems called >>>> for here. >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >>>> 84 const intptr_t inverted_jweak_mask = >>>> ~static_cast(JNIHandles::weak_tag_mask); >>>> 85 const int32_t truncated_mask = >>>> static_cast(inverted_jweak_mask); >>>> >>>> Why the two-step conversion here? Why not just >>>> >>>> const int32_t truncated_mask = >>>> static_cast(JNIHandles::weak_tag_mask); >>> >>> >>> I had changed this a bunch of times since I was a bit unsure about how >>> to best do this but when I decided to do the bitwise invert after the >>> cast to signed I forgot to clean up the casts. >>> >>>> >>>> That would also make the 32 and 64-bit code more similar in the >>>> suggested macroAssembler helper. >>> >>> >>> Indeed, I even found MacroAssembler::andptr which makes this a nice >>> and small helper. >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >>>> 51 #define CHECK(variable, expected) \ >>>> 52 do { if ((variable) != (expected)) { \ >>>> 53 (*env)->ThrowNew(env, exception, #variable" != " #expected); \ >>>> 54 return; \ >>>> 55 } \ >>>> 56 } while(0); >>>> >>>> The formatting of the code in the macro body is strange and confusing. >>> >>> >>> Cleaned up formatting. >>> >>>> >>>> Also the do-while-zero idiom that I'm used to leaves off the final >>>> semicolon in the macro. Instead of the uses are expected to be >>>> terminated with semicolons, as you've done. >>> >>> >>> Oh, oops. >>> >>> I also took the liberty to add STATIC_ASSERTS on 64 bit arm since the >>> encoding of the immediate in the "andr" instruction is a bit "funny". >>> The assembler code only verifies that immediate encoding works in >>> debug builds but in debug builds the fast JNI getters aren't generated. >>> >>> New webrevs: >>> Incremental: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ >>> Full: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ >> >> >> I only took a look at these files from the full webrev. These are the >> files touch in the two incremental webrevs: >> >> src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp >> No comments. >> >> src/cpu/arm/vm/jniFastGetField_arm.cpp >> No comments. >> >> src/cpu/sparc/vm/jniFastGetField_sparc.cpp >> No comments. >> >> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >> No comments. >> >> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >> No comments. >> >> src/cpu/x86/vm/macroAssembler_x86.cpp >> I like the new helper! >> >> src/cpu/x86/vm/macroAssembler_x86.hpp >> No comments. >> >> test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java >> L50 - any reason for the blank line? > > > No reason whatsoever, I'll remove it. > >> >> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >> L32 : Java_CallWithJNIWeak_testJNIFieldAccessors (JNIEnv *env, >> jclass clazz, jobject this) { >> L102: Java_CallWithJNIWeak_runTests (JNIEnv *env, jclass clazz, >> jobject this) { >> L137: Java_CallWithJNIWeak_weakReceiverTest0 (JNIEnv *env, jobject >> obj) { >> Please delete the space before the first '('. > > > Oh, oops. I'm going to blame javah for that :) > >> >> You may want to add a comment between the two halves of >> the test for the literal values that you're setting in >> the .java file and testing for in the .c file. > > > Added short comments in the .java and .c files. > >> >> In your (Mikael G) original invite: >> >>> Testing: >>> Standard JPRT and tier2-3 HotSpot tests. >> >> >> Does tier2 and/or tier3 cover the tests that failed in Mach5? > > > Somehow I forgot to mention that I also ran jdk_tier3 and the tests that > failed in Mach5 did pass on all platforms. > >> >> >>> I've modified the CallWithJNIWeak test to call all the primitive >>> getters and some other interesting cases I came up with. >>> I've also run the CallWithJNIWeak test on all platforms on both >>> product and fastdebug builds (since fast JNI getters are implicitly >>> disabled in debug builds due to VerifyJNIFields being enabled by >>> default in debug builds. >> >> >> Great coverage with the new tests! >> >> >>> I've not built or tested the aarch64 port but I think it's correct >>> and I hope someone can test it for me. >> >> >> Have you heard back from anyone on aarch64? I haven't >> seen any e-mail on this OpenJDK thread... > > > I've cc:ed Andrew in this thread to see if I can catch his attention. > > New webrevs at: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full3/webrev/ > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental2/webrev/ > > Thanks > /Mikael > > >> >> Thumbs up! >> >> Dan >> >> >>> >>> Thanks >>> /Mikael >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >> > From mikael.gerdin at oracle.com Fri Feb 24 11:54:29 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 24 Feb 2017 12:54:29 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> Message-ID: Hi Volker, On 2017-02-24 12:37, Volker Simonis wrote: > Hi Mikael, > > I had a little problems to apply the patch from your webrev because > the file actually contains two patches [1] but after fixing that > manually, I could still build and run the test on ppc64 and s390x. I seem to recall that that's a known limitation in webrev. Thanks for verifying the fix on your side. /Mikael > > Thanks, > Volker > > [1] http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/webrev/hotspot.changeset > > On Fri, Feb 24, 2017 at 10:55 AM, Mikael Gerdin > wrote: >> Hi Dan, >> >> Thanks for looking at this review! >> >> >> On 2017-02-23 17:18, Daniel D. Daugherty wrote: >>> >>> On 2/23/17 6:29 AM, Mikael Gerdin wrote: >>>> >>>> Hi Kim, >>>> >>>> Thanks for the prompt review! >>>> >>>> On 2017-02-22 20:40, Kim Barrett wrote: >>>>>> >>>>>> On Feb 22, 2017, at 11:07 AM, Mikael Gerdin >>>>>> wrote: >>>>>> >>>>>> Hi all, >>>>>> Please review this revised change to jweak to support G1. >>>>>> >>>>>> I've copied Kim's original description of the fix below for reference: >>>>>> >>>>>>> >>>>>>> The problem being addressed is that, when using G1, dereferencing a >>>>>>> jweak must ensure the obtained referent is ensured live, just as for >>>>>>> other weak reference types. This has always been a requirement, and >>>>>>> failure to do so appears to be a since-day1 G1 bug. >>>>>>> >>>>>>> There are two categories of places that need to address this issue: >>>>>>> JNIHandles::resolve and friends, and returning a jobject from native >>>>>>> code to Java. In both of these places the jobject whose contained oop >>>>>>> is being extracted might really be a jweak. jweak is >>>>>>> representationally indistinguishable from jobject; jweak is just a >>>>>>> typedef for jobject. The documentation says a jweak can be used >>>>>>> anywhere a jobject is permitted, making that type equivalence >>>>>>> necessary for a C API. (A C++ API might be able to have distinct >>>>>>> types via inheritance, though would still need a method for >>>>>>> distinguishing them at runtime. But since JNI is a C API...). >>>>>>> >>>>>>> The basic approach being taken is to "tag" jweaks by setting the low >>>>>>> order bit (recall that jweak == jobject == _jobject*), in order to >>>>>>> distinguish them from non-jweak jobjects. This tagging is only being >>>>>>> done when the selected GC needs the distinction, e.g. G1. This gives >>>>>>> us a representational difference between jobject and jweak on which to >>>>>>> base the different behavior, hiding that distinction behind the >>>>>>> existing API. >>>>>>> >>>>>>> JNIHandles::resolve and friends are changed to unconditionally strip >>>>>>> off any potential weak tag when translating from jobject to to oop*. >>>>>>> That's cheaper than testing for the conditional use of tagging and >>>>>>> then stripping. Also stripped when deleting JNI handles. >>>>>>> >>>>>>> TemplateInterpreterGenerator::generate_native_entry and >>>>>>> SharedRuntime::generate_native_wrapper are changed to conditionally >>>>>>> emit code to apply the G1 pre-barrier to the value obtained from a >>>>>>> jobject result, when the result is tagged as a jweak, which only >>>>>>> occurs when using G1. >>>>>>> >>>>>>> For arm32/arm64, this required moving the g1_write_barrier_pre >>>>>>> definitions from InterpreterMacroAssembler to MacroAssembler. Also >>>>>>> moved g1_write_barrier_post, for consistency. >>>>>>> >>>>>> >>>>>> In addition to Kim's final version of the change (which was backed >>>>>> out) this updated version adds code to mask out the weak tag bit in >>>>>> the fast JNI field getters on all platforms where fast JNI getters >>>>>> are used. >>>>>> >>>>>> [?] >>>>>> >>>>>> Webrevs: >>>>>> Kim's initial change: >>>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ >>>>>> My additions: >>>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ >>>>>> Full webrev: >>>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ >>>>>> >>>>>> Bugs: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8175085 >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>>>>> >>>>>> Thanks >>>>>> /Mikael >>>>> >>>>> >>>>> Mikael - Thanks for taking this up for me. Just a few minor comments. >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> The usual copyright reminder. >>>> >>>> >>>> Updated copyrights. >>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/cpu/arm/vm/jniFastGetField_arm.cpp >>>>> 122 #ifndef AARCH64 >>>>> 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); >>>>> 124 #else >>>>> 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); >>>>> 126 #endif >>>>> >>>>> Usual style when selecting between AARCH64 and not seems to be to put >>>>> the AARCH64 code first, e.g. >>>>> >>>>> #ifdef AARCH64 >>>>> ... >>>>> #else >>>>> ... >>>>> #endif >>>> >>>> >>>> Fixed. >>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>>> >>>>> Use STATIC_ASSERT rather than assert. >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>>> 89 const intptr_t inverted_jweak_mask = >>>>> ~static_cast(JNIHandles::weak_tag_mask); >>>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>>> 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to >>>>> sign-extension >>>>> >>>>> >>>>> With three occurrences of this snippet in this file and two more >>>>> similar ones in the 64-bit file, a macroAssembler helper seems called >>>>> for here. >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >>>>> 84 const intptr_t inverted_jweak_mask = >>>>> ~static_cast(JNIHandles::weak_tag_mask); >>>>> 85 const int32_t truncated_mask = >>>>> static_cast(inverted_jweak_mask); >>>>> >>>>> Why the two-step conversion here? Why not just >>>>> >>>>> const int32_t truncated_mask = >>>>> static_cast(JNIHandles::weak_tag_mask); >>>> >>>> >>>> I had changed this a bunch of times since I was a bit unsure about how >>>> to best do this but when I decided to do the bitwise invert after the >>>> cast to signed I forgot to clean up the casts. >>>> >>>>> >>>>> That would also make the 32 and 64-bit code more similar in the >>>>> suggested macroAssembler helper. >>>> >>>> >>>> Indeed, I even found MacroAssembler::andptr which makes this a nice >>>> and small helper. >>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >>>>> 51 #define CHECK(variable, expected) \ >>>>> 52 do { if ((variable) != (expected)) { \ >>>>> 53 (*env)->ThrowNew(env, exception, #variable" != " #expected); \ >>>>> 54 return; \ >>>>> 55 } \ >>>>> 56 } while(0); >>>>> >>>>> The formatting of the code in the macro body is strange and confusing. >>>> >>>> >>>> Cleaned up formatting. >>>> >>>>> >>>>> Also the do-while-zero idiom that I'm used to leaves off the final >>>>> semicolon in the macro. Instead of the uses are expected to be >>>>> terminated with semicolons, as you've done. >>>> >>>> >>>> Oh, oops. >>>> >>>> I also took the liberty to add STATIC_ASSERTS on 64 bit arm since the >>>> encoding of the immediate in the "andr" instruction is a bit "funny". >>>> The assembler code only verifies that immediate encoding works in >>>> debug builds but in debug builds the fast JNI getters aren't generated. >>>> >>>> New webrevs: >>>> Incremental: >>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ >>>> Full: >>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ >>> >>> >>> I only took a look at these files from the full webrev. These are the >>> files touch in the two incremental webrevs: >>> >>> src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp >>> No comments. >>> >>> src/cpu/arm/vm/jniFastGetField_arm.cpp >>> No comments. >>> >>> src/cpu/sparc/vm/jniFastGetField_sparc.cpp >>> No comments. >>> >>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>> No comments. >>> >>> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >>> No comments. >>> >>> src/cpu/x86/vm/macroAssembler_x86.cpp >>> I like the new helper! >>> >>> src/cpu/x86/vm/macroAssembler_x86.hpp >>> No comments. >>> >>> test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java >>> L50 - any reason for the blank line? >> >> >> No reason whatsoever, I'll remove it. >> >>> >>> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >>> L32 : Java_CallWithJNIWeak_testJNIFieldAccessors (JNIEnv *env, >>> jclass clazz, jobject this) { >>> L102: Java_CallWithJNIWeak_runTests (JNIEnv *env, jclass clazz, >>> jobject this) { >>> L137: Java_CallWithJNIWeak_weakReceiverTest0 (JNIEnv *env, jobject >>> obj) { >>> Please delete the space before the first '('. >> >> >> Oh, oops. I'm going to blame javah for that :) >> >>> >>> You may want to add a comment between the two halves of >>> the test for the literal values that you're setting in >>> the .java file and testing for in the .c file. >> >> >> Added short comments in the .java and .c files. >> >>> >>> In your (Mikael G) original invite: >>> >>>> Testing: >>>> Standard JPRT and tier2-3 HotSpot tests. >>> >>> >>> Does tier2 and/or tier3 cover the tests that failed in Mach5? >> >> >> Somehow I forgot to mention that I also ran jdk_tier3 and the tests that >> failed in Mach5 did pass on all platforms. >> >>> >>> >>>> I've modified the CallWithJNIWeak test to call all the primitive >>>> getters and some other interesting cases I came up with. >>>> I've also run the CallWithJNIWeak test on all platforms on both >>>> product and fastdebug builds (since fast JNI getters are implicitly >>>> disabled in debug builds due to VerifyJNIFields being enabled by >>>> default in debug builds. >>> >>> >>> Great coverage with the new tests! >>> >>> >>>> I've not built or tested the aarch64 port but I think it's correct >>>> and I hope someone can test it for me. >>> >>> >>> Have you heard back from anyone on aarch64? I haven't >>> seen any e-mail on this OpenJDK thread... >> >> >> I've cc:ed Andrew in this thread to see if I can catch his attention. >> >> New webrevs at: >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full3/webrev/ >> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental2/webrev/ >> >> Thanks >> /Mikael >> >> >>> >>> Thumbs up! >>> >>> Dan >>> >>> >>>> >>>> Thanks >>>> /Mikael >>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>> >> From thomas.schatzl at oracle.com Fri Feb 24 11:58:45 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 24 Feb 2017 12:58:45 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> Message-ID: <1487937525.3471.1.camel@oracle.com> Hi, On Fri, 2017-02-24 at 10:55 +0100, Mikael Gerdin wrote: > Hi Dan, > > [...] > > > > > > I've not built or tested the aarch64 port but I think it's > > > correct > > > and I hope someone can test it for me. > > Have you heard back from anyone on aarch64? I haven't > > seen any e-mail on this OpenJDK thread... > I've cc:ed Andrew in this thread to see if I can catch his attention. > > New webrevs at: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full3/webrev/ > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental2/webre > v/ ? looks good. Thomas From mikael.gerdin at oracle.com Fri Feb 24 12:02:19 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 24 Feb 2017 13:02:19 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> Message-ID: <664fc325-3779-aadb-c6ae-eaba41648468@oracle.com> Actually CC:ing Andrew. On 2017-02-24 10:55, Mikael Gerdin wrote: > Hi Dan, > > Thanks for looking at this review! > > On 2017-02-23 17:18, Daniel D. Daugherty wrote: >> On 2/23/17 6:29 AM, Mikael Gerdin wrote: >>> Hi Kim, >>> >>> Thanks for the prompt review! >>> >>> On 2017-02-22 20:40, Kim Barrett wrote: >>>>> On Feb 22, 2017, at 11:07 AM, Mikael Gerdin >>>>> wrote: >>>>> >>>>> Hi all, >>>>> Please review this revised change to jweak to support G1. >>>>> >>>>> I've copied Kim's original description of the fix below for reference: >>>>> >>>>>> >>>>>> The problem being addressed is that, when using G1, dereferencing a >>>>>> jweak must ensure the obtained referent is ensured live, just as for >>>>>> other weak reference types. This has always been a requirement, and >>>>>> failure to do so appears to be a since-day1 G1 bug. >>>>>> >>>>>> There are two categories of places that need to address this issue: >>>>>> JNIHandles::resolve and friends, and returning a jobject from native >>>>>> code to Java. In both of these places the jobject whose contained >>>>>> oop >>>>>> is being extracted might really be a jweak. jweak is >>>>>> representationally indistinguishable from jobject; jweak is just a >>>>>> typedef for jobject. The documentation says a jweak can be used >>>>>> anywhere a jobject is permitted, making that type equivalence >>>>>> necessary for a C API. (A C++ API might be able to have distinct >>>>>> types via inheritance, though would still need a method for >>>>>> distinguishing them at runtime. But since JNI is a C API...). >>>>>> >>>>>> The basic approach being taken is to "tag" jweaks by setting the low >>>>>> order bit (recall that jweak == jobject == _jobject*), in order to >>>>>> distinguish them from non-jweak jobjects. This tagging is only being >>>>>> done when the selected GC needs the distinction, e.g. G1. This gives >>>>>> us a representational difference between jobject and jweak on >>>>>> which to >>>>>> base the different behavior, hiding that distinction behind the >>>>>> existing API. >>>>>> >>>>>> JNIHandles::resolve and friends are changed to unconditionally strip >>>>>> off any potential weak tag when translating from jobject to to oop*. >>>>>> That's cheaper than testing for the conditional use of tagging and >>>>>> then stripping. Also stripped when deleting JNI handles. >>>>>> >>>>>> TemplateInterpreterGenerator::generate_native_entry and >>>>>> SharedRuntime::generate_native_wrapper are changed to conditionally >>>>>> emit code to apply the G1 pre-barrier to the value obtained from a >>>>>> jobject result, when the result is tagged as a jweak, which only >>>>>> occurs when using G1. >>>>>> >>>>>> For arm32/arm64, this required moving the g1_write_barrier_pre >>>>>> definitions from InterpreterMacroAssembler to MacroAssembler. Also >>>>>> moved g1_write_barrier_post, for consistency. >>>>>> >>>>> >>>>> In addition to Kim's final version of the change (which was backed >>>>> out) this updated version adds code to mask out the weak tag bit in >>>>> the fast JNI field getters on all platforms where fast JNI getters >>>>> are used. >>>>> >>>>> [?] >>>>> >>>>> Webrevs: >>>>> Kim's initial change: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ >>>>> My additions: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ >>>>> Full webrev: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ >>>>> >>>>> Bugs: >>>>> https://bugs.openjdk.java.net/browse/JDK-8175085 >>>>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>>>> >>>>> Thanks >>>>> /Mikael >>>> >>>> Mikael - Thanks for taking this up for me. Just a few minor comments. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> The usual copyright reminder. >>> >>> Updated copyrights. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/cpu/arm/vm/jniFastGetField_arm.cpp >>>> 122 #ifndef AARCH64 >>>> 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); >>>> 124 #else >>>> 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); >>>> 126 #endif >>>> >>>> Usual style when selecting between AARCH64 and not seems to be to put >>>> the AARCH64 code first, e.g. >>>> >>>> #ifdef AARCH64 >>>> ... >>>> #else >>>> ... >>>> #endif >>> >>> Fixed. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>> >>>> Use STATIC_ASSERT rather than assert. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>> 89 const intptr_t inverted_jweak_mask = >>>> ~static_cast(JNIHandles::weak_tag_mask); >>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>> 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to >>>> sign-extension >>>> >>>> >>>> With three occurrences of this snippet in this file and two more >>>> similar ones in the 64-bit file, a macroAssembler helper seems called >>>> for here. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >>>> 84 const intptr_t inverted_jweak_mask = >>>> ~static_cast(JNIHandles::weak_tag_mask); >>>> 85 const int32_t truncated_mask = >>>> static_cast(inverted_jweak_mask); >>>> >>>> Why the two-step conversion here? Why not just >>>> >>>> const int32_t truncated_mask = >>>> static_cast(JNIHandles::weak_tag_mask); >>> >>> I had changed this a bunch of times since I was a bit unsure about how >>> to best do this but when I decided to do the bitwise invert after the >>> cast to signed I forgot to clean up the casts. >>> >>>> >>>> That would also make the 32 and 64-bit code more similar in the >>>> suggested macroAssembler helper. >>> >>> Indeed, I even found MacroAssembler::andptr which makes this a nice >>> and small helper. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >>>> 51 #define CHECK(variable, >>>> expected) \ >>>> 52 do { if ((variable) != (expected)) >>>> { \ >>>> 53 (*env)->ThrowNew(env, exception, #variable" != " >>>> #expected); \ >>>> 54 return; \ >>>> 55 } \ >>>> 56 } while(0); >>>> >>>> The formatting of the code in the macro body is strange and confusing. >>> >>> Cleaned up formatting. >>> >>>> >>>> Also the do-while-zero idiom that I'm used to leaves off the final >>>> semicolon in the macro. Instead of the uses are expected to be >>>> terminated with semicolons, as you've done. >>> >>> Oh, oops. >>> >>> I also took the liberty to add STATIC_ASSERTS on 64 bit arm since the >>> encoding of the immediate in the "andr" instruction is a bit "funny". >>> The assembler code only verifies that immediate encoding works in >>> debug builds but in debug builds the fast JNI getters aren't generated. >>> >>> New webrevs: >>> Incremental: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ >>> Full: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ >> >> I only took a look at these files from the full webrev. These are the >> files touch in the two incremental webrevs: >> >> src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp >> No comments. >> >> src/cpu/arm/vm/jniFastGetField_arm.cpp >> No comments. >> >> src/cpu/sparc/vm/jniFastGetField_sparc.cpp >> No comments. >> >> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >> No comments. >> >> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >> No comments. >> >> src/cpu/x86/vm/macroAssembler_x86.cpp >> I like the new helper! >> >> src/cpu/x86/vm/macroAssembler_x86.hpp >> No comments. >> >> test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java >> L50 - any reason for the blank line? > > No reason whatsoever, I'll remove it. > >> >> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >> L32 : Java_CallWithJNIWeak_testJNIFieldAccessors (JNIEnv *env, >> jclass clazz, jobject this) { >> L102: Java_CallWithJNIWeak_runTests (JNIEnv *env, jclass clazz, >> jobject this) { >> L137: Java_CallWithJNIWeak_weakReceiverTest0 (JNIEnv *env, jobject >> obj) { >> Please delete the space before the first '('. > > Oh, oops. I'm going to blame javah for that :) > >> >> You may want to add a comment between the two halves of >> the test for the literal values that you're setting in >> the .java file and testing for in the .c file. > > Added short comments in the .java and .c files. > >> >> In your (Mikael G) original invite: >> >>> Testing: >>> Standard JPRT and tier2-3 HotSpot tests. >> >> Does tier2 and/or tier3 cover the tests that failed in Mach5? > > Somehow I forgot to mention that I also ran jdk_tier3 and the tests that > failed in Mach5 did pass on all platforms. > >> >> >>> I've modified the CallWithJNIWeak test to call all the primitive >>> getters and some other interesting cases I came up with. >>> I've also run the CallWithJNIWeak test on all platforms on both >>> product and fastdebug builds (since fast JNI getters are implicitly >>> disabled in debug builds due to VerifyJNIFields being enabled by >>> default in debug builds. >> >> Great coverage with the new tests! >> >> >>> I've not built or tested the aarch64 port but I think it's correct >>> and I hope someone can test it for me. >> >> Have you heard back from anyone on aarch64? I haven't >> seen any e-mail on this OpenJDK thread... > > I've cc:ed Andrew in this thread to see if I can catch his attention. > > New webrevs at: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full3/webrev/ > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental2/webrev/ > > Thanks > /Mikael > >> >> Thumbs up! >> >> Dan >> >> >>> >>> Thanks >>> /Mikael >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> >> From gromero at linux.vnet.ibm.com Fri Feb 24 12:02:31 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Fri, 24 Feb 2017 09:02:31 -0300 Subject: Linux/PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: <8a0d9151-8ed7-9409-20a1-10258023ae7d@oracle.com> References: <5898EF94.5060904@linux.vnet.ibm.com> <8a0d9151-8ed7-9409-20a1-10258023ae7d@oracle.com> Message-ID: <58B020D7.3020508@linux.vnet.ibm.com> Hi Sangheon, Please find my comments inline. On 06-02-2017 20:23, sangheon wrote: > Hi Gustavo, > > On 02/06/2017 01:50 PM, Gustavo Romero wrote: >> Hi, >> >> On Linux/PPC64 I'm getting a series of "mbind: Invalid argument" that seems >> exactly the same as reported for x64 [1]: >> >> [root at spocfire3 ~]# java -XX:+UseNUMA -version >> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument >> openjdk version "1.8.0_121" >> OpenJDK Runtime Environment (build 1.8.0_121-b13) >> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) >> >> [root at spocfire3 ~]# uname -a >> Linux spocfire3.aus.stglabs.ibm.com 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux >> >> [root at spocfire3 ~]# lscpu >> Architecture: ppc64le >> Byte Order: Little Endian >> CPU(s): 160 >> On-line CPU(s) list: 0-159 >> Thread(s) per core: 8 >> Core(s) per socket: 10 >> Socket(s): 2 >> NUMA node(s): 2 >> Model: 2.0 (pvr 004d 0200) >> Model name: POWER8 (raw), altivec supported >> L1d cache: 64K >> L1i cache: 32K >> L2 cache: 512K >> L3 cache: 8192K >> NUMA node0 CPU(s): 0-79 >> NUMA node8 CPU(s): 80-159 >> >> On chasing down it, looks like it comes from PSYoungGen::initialize() in >> src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp that calls >> initialize_work(), that calls the MutableNUMASpace() constructor if >> UseNUMA is set: >> http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/567e410935e5/src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp#l77 >> >> MutableNUMASpace() then calls os::numa_make_local(), that in the end calls >> numa_set_bind_policy() in libnuma.so.1 [2]. >> >> I've traced some values for which mbind() syscall fails: >> http://termbin.com/ztfs (search for "Invalid argument" in the log). >> >> Assuming it's the same bug as reported in [1] and so it's not fixed on 9 and 10: >> >> - Is there any WIP or known workaround? > There's no progress on JDK-8163796 and no workaround found yet. > And unfortunately, I'm not planning to fix it soon. Hive, a critical component of Hadoop ecosystem, comes with a shell and uses java (with UseNUMA flag) in the background to run mysql-kind of queries. On PPC64 the mbind() messages in question make the shell pretty cumbersome. For instance: hive> show databases; mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument (repeat message more 28 times...) ... OK mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument default tpcds_bin_partitioned_orc_10 tpcds_text_10 Time taken: 1.036 seconds, Fetched: 3 row(s) hive> mbind: Invalid argument mbind: Invalid argument mbind: Invalid argument Also, on PPC64 a simple "java -XX:+UseParallelGC -XX:+UseNUMA -version" will trigger the problem, without any additional flags. So I'd like to correct that behavior (please see my next comment on that). >> - Should I append this output in [1] description or open a new one and make it >> related to" [1]? > I think your problem seems same as JDK-8163796, so adding your output on the CR seems good. > And please add logs as well. I recommend to enabling something like "-Xlog:gc*,gc+heap*=trace". > IIRC, the problem was only occurred when the -Xmx was small in my case. JVM code used to discover which numa nodes it can bind assumes that nodes are consecutive and tries to bind from 0 to numa_max_node() [1, 2, 3, 4], i.e. from 0 to the highest node number available on the system. However, at least on PPC64 that assumption is not always true. For instance, consider the following numa topology: available: 4 nodes (0-1,16-17) node 0 cpus: 0 8 16 24 32 node 0 size: 130706 MB node 0 free: 145 MB node 1 cpus: 40 48 56 64 72 node 1 size: 0 MB node 1 free: 0 MB node 16 cpus: 80 88 96 104 112 node 16 size: 130630 MB node 16 free: 529 MB node 17 cpus: 120 128 136 144 152 node 17 size: 0 MB node 17 free: 0 MB node distances: node 0 1 16 17 0: 10 20 40 40 1: 20 10 40 40 16: 40 40 10 20 17: 40 40 20 10 In that case we have four nodes, 2 without memory (1 and 17), where the highest node id is 17. Hence if the JVM tries to bind from 0 to 17, mbind() will fail except for nodes 0 and 16, which are configured and have memory. mbind() failures will generate the "mbind: Invalid argument" messages. A solution would be to use in os::numa_get_group_num() not numa_max_node() but instead numa_num_configured_nodes() which returns the total number of nodes with memory in the system (so in our example above it will return exactly 2 nodes) and then inspect numa_all_node_ptr in os::numa_get_leaf_groups() to find the correct node ids to append (in our case, map ids[0] = 0 [node 0] and ids[1] = 16 [node 16]). One thing is that os::numa_get_leaf_groups() argument "size" will not be required anymore and will be loose, so the interface will have to be adapted on other OSs besides Linux I guess [5]. It would be necessary to adapt os::Linux::rebuild_cpu_to_node_map() since not all numa nodes are suitable to be returned by a call to os::numa_get_group_id() as some cpus would be in a node without memory. In that case we can return the closest numa node instead. A new way to translate indices to nodes is also useful since nodes are not always consecutive. Finally, although "numa_nodes_ptr" is not present in libnuma's manual it's what is used in numactl to find out the total number of nodes in the system [6]. I could not find a function that would return that number readily. I asked to libnuma ML if a better solution exists [7]. The following webrev implements the proposed changes on jdk9 (backport to 8 is simple): webrev: http://cr.openjdk.java.net/~gromero/8175813/ bug: https://bugs.openjdk.java.net/browse/JDK-8175813 Here are the logs with "-Xlog:gc*,gc+heap*=trace": http://cr.openjdk.java.net/~gromero/logs/pristine.log (current state) http://cr.openjdk.java.net/~gromero/logs/numa_patched.log (proposed change) I've tested on 8 against SPECjvm2008 on the aforementioned machine and performance improved ~5% in comparison to the same version packaged by the distro, but I don't expect any difference on machines where nodes are always consecutive and where nodes always have memory. After a due community review, could you sponsor that change? Thank you. Best regards, Gustavo [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l241 [2] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2745 [3] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l243 [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2761 [5] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/runtime/os.hpp#l356 [6] https://github.com/numactl/numactl/blob/master/numactl.c#L251 [7] http://www.spinics.net/lists/linux-numa/msg01173.html > > Thanks, > Sangheon > > >> >> Thank you. >> >> >> Best regards, >> Gustavo >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8163796 >> [2] https://da.gd/4vXF >> > From stuart.monteith at linaro.org Fri Feb 24 16:31:19 2017 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Fri, 24 Feb 2017 16:31:19 +0000 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> Message-ID: Hi, I can build and test the patch on AArch64. I can post the results here. BR, Stuart On 24 February 2017 at 09:55, Mikael Gerdin wrote: > Hi Dan, > > Thanks for looking at this review! > > > On 2017-02-23 17:18, Daniel D. Daugherty wrote: >> >> On 2/23/17 6:29 AM, Mikael Gerdin wrote: >>> >>> Hi Kim, >>> >>> Thanks for the prompt review! >>> >>> On 2017-02-22 20:40, Kim Barrett wrote: >>>>> >>>>> On Feb 22, 2017, at 11:07 AM, Mikael Gerdin >>>>> wrote: >>>>> >>>>> Hi all, >>>>> Please review this revised change to jweak to support G1. >>>>> >>>>> I've copied Kim's original description of the fix below for reference: >>>>> >>>>>> >>>>>> The problem being addressed is that, when using G1, dereferencing a >>>>>> jweak must ensure the obtained referent is ensured live, just as for >>>>>> other weak reference types. This has always been a requirement, and >>>>>> failure to do so appears to be a since-day1 G1 bug. >>>>>> >>>>>> There are two categories of places that need to address this issue: >>>>>> JNIHandles::resolve and friends, and returning a jobject from native >>>>>> code to Java. In both of these places the jobject whose contained oop >>>>>> is being extracted might really be a jweak. jweak is >>>>>> representationally indistinguishable from jobject; jweak is just a >>>>>> typedef for jobject. The documentation says a jweak can be used >>>>>> anywhere a jobject is permitted, making that type equivalence >>>>>> necessary for a C API. (A C++ API might be able to have distinct >>>>>> types via inheritance, though would still need a method for >>>>>> distinguishing them at runtime. But since JNI is a C API...). >>>>>> >>>>>> The basic approach being taken is to "tag" jweaks by setting the low >>>>>> order bit (recall that jweak == jobject == _jobject*), in order to >>>>>> distinguish them from non-jweak jobjects. This tagging is only being >>>>>> done when the selected GC needs the distinction, e.g. G1. This gives >>>>>> us a representational difference between jobject and jweak on which to >>>>>> base the different behavior, hiding that distinction behind the >>>>>> existing API. >>>>>> >>>>>> JNIHandles::resolve and friends are changed to unconditionally strip >>>>>> off any potential weak tag when translating from jobject to to oop*. >>>>>> That's cheaper than testing for the conditional use of tagging and >>>>>> then stripping. Also stripped when deleting JNI handles. >>>>>> >>>>>> TemplateInterpreterGenerator::generate_native_entry and >>>>>> SharedRuntime::generate_native_wrapper are changed to conditionally >>>>>> emit code to apply the G1 pre-barrier to the value obtained from a >>>>>> jobject result, when the result is tagged as a jweak, which only >>>>>> occurs when using G1. >>>>>> >>>>>> For arm32/arm64, this required moving the g1_write_barrier_pre >>>>>> definitions from InterpreterMacroAssembler to MacroAssembler. Also >>>>>> moved g1_write_barrier_post, for consistency. >>>>>> >>>>> >>>>> In addition to Kim's final version of the change (which was backed >>>>> out) this updated version adds code to mask out the weak tag bit in >>>>> the fast JNI field getters on all platforms where fast JNI getters >>>>> are used. >>>>> >>>>> [?] >>>>> >>>>> Webrevs: >>>>> Kim's initial change: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.kim/webrev/ >>>>> My additions: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.mg/webrev/ >>>>> Full webrev: >>>>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full/ >>>>> >>>>> Bugs: >>>>> https://bugs.openjdk.java.net/browse/JDK-8175085 >>>>> https://bugs.openjdk.java.net/browse/JDK-8166188 >>>>> >>>>> Thanks >>>>> /Mikael >>>> >>>> >>>> Mikael - Thanks for taking this up for me. Just a few minor comments. >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> The usual copyright reminder. >>> >>> >>> Updated copyrights. >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/arm/vm/jniFastGetField_arm.cpp >>>> 122 #ifndef AARCH64 >>>> 123 __ bic(R1, R1, JNIHandles::weak_tag_mask); >>>> 124 #else >>>> 125 __ andr(R1, R1, ~JNIHandles::weak_tag_mask); >>>> 126 #endif >>>> >>>> Usual style when selecting between AARCH64 and not seems to be to put >>>> the AARCH64 code first, e.g. >>>> >>>> #ifdef AARCH64 >>>> ... >>>> #else >>>> ... >>>> #endif >>> >>> >>> Fixed. >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>> >>>> Use STATIC_ASSERT rather than assert. >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >>>> 89 const intptr_t inverted_jweak_mask = >>>> ~static_cast(JNIHandles::weak_tag_mask); >>>> 90 assert(inverted_jweak_mask == -2, "Otherwise check this code"); >>>> 91 __ andl(rdx, inverted_jweak_mask); // mask is subject to >>>> sign-extension >>>> >>>> >>>> With three occurrences of this snippet in this file and two more >>>> similar ones in the 64-bit file, a macroAssembler helper seems called >>>> for here. >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >>>> 84 const intptr_t inverted_jweak_mask = >>>> ~static_cast(JNIHandles::weak_tag_mask); >>>> 85 const int32_t truncated_mask = >>>> static_cast(inverted_jweak_mask); >>>> >>>> Why the two-step conversion here? Why not just >>>> >>>> const int32_t truncated_mask = >>>> static_cast(JNIHandles::weak_tag_mask); >>> >>> >>> I had changed this a bunch of times since I was a bit unsure about how >>> to best do this but when I decided to do the bitwise invert after the >>> cast to signed I forgot to clean up the casts. >>> >>>> >>>> That would also make the 32 and 64-bit code more similar in the >>>> suggested macroAssembler helper. >>> >>> >>> Indeed, I even found MacroAssembler::andptr which makes this a nice >>> and small helper. >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >>>> 51 #define CHECK(variable, expected) \ >>>> 52 do { if ((variable) != (expected)) { \ >>>> 53 (*env)->ThrowNew(env, exception, #variable" != " #expected); \ >>>> 54 return; \ >>>> 55 } \ >>>> 56 } while(0); >>>> >>>> The formatting of the code in the macro body is strange and confusing. >>> >>> >>> Cleaned up formatting. >>> >>>> >>>> Also the do-while-zero idiom that I'm used to leaves off the final >>>> semicolon in the macro. Instead of the uses are expected to be >>>> terminated with semicolons, as you've done. >>> >>> >>> Oh, oops. >>> >>> I also took the liberty to add STATIC_ASSERTS on 64 bit arm since the >>> encoding of the immediate in the "andr" instruction is a bit "funny". >>> The assembler code only verifies that immediate encoding works in >>> debug builds but in debug builds the fast JNI getters aren't generated. >>> >>> New webrevs: >>> Incremental: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental/webrev/ >>> Full: >>> http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full2/webrev/ >> >> >> I only took a look at these files from the full webrev. These are the >> files touch in the two incremental webrevs: >> >> src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp >> No comments. >> >> src/cpu/arm/vm/jniFastGetField_arm.cpp >> No comments. >> >> src/cpu/sparc/vm/jniFastGetField_sparc.cpp >> No comments. >> >> src/cpu/x86/vm/jniFastGetField_x86_32.cpp >> No comments. >> >> src/cpu/x86/vm/jniFastGetField_x86_64.cpp >> No comments. >> >> src/cpu/x86/vm/macroAssembler_x86.cpp >> I like the new helper! >> >> src/cpu/x86/vm/macroAssembler_x86.hpp >> No comments. >> >> test/runtime/jni/CallWithJNIWeak/CallWithJNIWeak.java >> L50 - any reason for the blank line? > > > No reason whatsoever, I'll remove it. > >> >> test/runtime/jni/CallWithJNIWeak/libCallWithJNIWeak.c >> L32 : Java_CallWithJNIWeak_testJNIFieldAccessors (JNIEnv *env, >> jclass clazz, jobject this) { >> L102: Java_CallWithJNIWeak_runTests (JNIEnv *env, jclass clazz, >> jobject this) { >> L137: Java_CallWithJNIWeak_weakReceiverTest0 (JNIEnv *env, jobject >> obj) { >> Please delete the space before the first '('. > > > Oh, oops. I'm going to blame javah for that :) > >> >> You may want to add a comment between the two halves of >> the test for the literal values that you're setting in >> the .java file and testing for in the .c file. > > > Added short comments in the .java and .c files. > >> >> In your (Mikael G) original invite: >> >>> Testing: >>> Standard JPRT and tier2-3 HotSpot tests. >> >> >> Does tier2 and/or tier3 cover the tests that failed in Mach5? > > > Somehow I forgot to mention that I also ran jdk_tier3 and the tests that > failed in Mach5 did pass on all platforms. > >> >> >>> I've modified the CallWithJNIWeak test to call all the primitive >>> getters and some other interesting cases I came up with. >>> I've also run the CallWithJNIWeak test on all platforms on both >>> product and fastdebug builds (since fast JNI getters are implicitly >>> disabled in debug builds due to VerifyJNIFields being enabled by >>> default in debug builds. >> >> >> Great coverage with the new tests! >> >> >>> I've not built or tested the aarch64 port but I think it's correct >>> and I hope someone can test it for me. >> >> >> Have you heard back from anyone on aarch64? I haven't >> seen any e-mail on this OpenJDK thread... > > > I've cc:ed Andrew in this thread to see if I can catch his attention. > > New webrevs at: > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.full3/webrev/ > http://cr.openjdk.java.net/~mgerdin/8175085/webrev.incremental2/webrev/ > > Thanks > /Mikael > > >> >> Thumbs up! >> >> Dan >> >> >>> >>> Thanks >>> /Mikael >>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >> > From adinn at redhat.com Fri Feb 24 16:36:49 2017 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 24 Feb 2017 16:36:49 +0000 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <664fc325-3779-aadb-c6ae-eaba41648468@oracle.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> <664fc325-3779-aadb-c6ae-eaba41648468@oracle.com> Message-ID: Hi Mikael, On 24/02/17 12:02, Mikael Gerdin wrote: > Actually CC:ing Andrew. I'm still building this and will test it as soon as I can. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From martin.doerr at sap.com Fri Feb 24 16:40:39 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 24 Feb 2017 16:40:39 +0000 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <58AF5253.7020005@oracle.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> Message-ID: Hi Max, thank you very much for sharing your results and for sending the patch. I guess it covers the most relevant cases, but not all ones. I think it'd be better to modify dispatch_next instead of dispatch_epilog on x86. (dispatch_next is also used by generate_return_entry_for and generate_deopt_entry_for.) On s390, I'm using dispatch_next with: if (!EnableTosCache) { push(state); state = vtos; } dispatch_base(state, Interpreter::dispatch_table(state)); I also added an assertion to dispatch_base in order to make sure I'm hitting all dispatch usages: assert(EnableTosCache || state == vtos, "sanity"); Unfortunately, the performance results of SPEC jvm98 with -Xint seem to drop significantly with -XX:-EnableTosCache on both, PPC64 and s390. But we need to perform more measurements to get more reliable results. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Max Ockner Sent: Donnerstag, 23. Februar 2017 22:21 To: hotspot-dev at openjdk.java.net Subject: Re: Discussion: 8172978: Remove Interpreter TOS optimization Hi Volker, I have attached the patch that I have been testing. Thanks, Max On 2/20/2017 5:45 AM, Volker Simonis wrote: > Hi, > > besides the fact that this of course means some work for us :) I > currently don't see any problems for our porting platforms (ppc64 and > s390x). > > Are there any webrevs available, so we can see how big they are and > maybe do some own benchmarking? > > Thanks, > Volker > > > On Sun, Feb 19, 2017 at 11:11 PM, wrote: >> >> On 2/18/17 11:14 AM, coleen.phillimore at oracle.com wrote: >>> When Max gets back from the long weekend, he'll post the platforms in your >>> bug. >>> >>> It's amazing that for -Xint there's no significant difference. I've seen >>> -Xint performance of 15% slower cause a 2% slowdown with server but that was >>> before tiered compilation. >> >> I should clarify this. I've seen this slowdown for *different* interpreter >> optimizations, which *can* affect server performance. I was measuring >> specjvm98 on linux x64. If there's no significant difference for this TOS >> optimization, there is no chance of a degredation in overall performance. >> >> Coleen >> >>> The reason for this query was to see what developers for the other >>> platform ports think, since this change would affect all of the platforms. >>> >>> Thanks, >>> Coleen >>> >>> On 2/18/17 10:50 AM, Daniel D. Daugherty wrote: >>>> If Claes is happy with the perf testing, then I'm happy. :-) >>>> >>>> Dan >>>> >>>> >>>> On 2/18/17 3:46 AM, Claes Redestad wrote: >>>>> Hi, >>>>> >>>>> I've seen Max has run plenty of tests on our internal performance >>>>> infrastructure and everything I've seen there seems to corroborate the >>>>> idea that this removal is OK from a performance point of view, the >>>>> footprint improvements are small but significant and any negative >>>>> performance impact on throughput benchmarks is at noise levels even >>>>> with -Xint (it appears many benchmarks time out with this setting >>>>> both before and after, though; Max, let's discuss offline how to >>>>> deal with that :-)) >>>>> >>>>> I expect this will be tested more thoroughly once adapted to all >>>>> platforms (which I assume is the intent?), but see no concern from >>>>> a performance testing point of view: Do it! >>>>> >>>>> Thanks! >>>>> >>>>> /Claes >>>>> >>>>> On 2017-02-16 16:40, Daniel D. Daugherty wrote: >>>>>> Hi Max, >>>>>> >>>>>> Added a note to your bug. Interesting idea, but I think your data is >>>>>> a bit incomplete at the moment. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 2/15/17 3:18 PM, Max Ockner wrote: >>>>>>> Hello all, >>>>>>> >>>>>>> We have filed a bug to remove the interpreter stack caching >>>>>>> optimization for jdk10. Ideally we can make this change *early* >>>>>>> during the jdk10 development cycle. See below for justification: >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>>>>>> >>>>>>> Stack caching has been around for a long time and is intended to >>>>>>> replace some of the load/store (pop/push) operations with >>>>>>> corresponding register operations. The need for this optimization >>>>>>> arose before caching could adequately lessen the burden of memory >>>>>>> access. We have reevaluated the JVM stack caching optimization and >>>>>>> have found that it has a high memory footprint and is very costly to >>>>>>> maintain, but does not provide significant measurable or theoretical >>>>>>> benefit for us when used with modern hardware. >>>>>>> >>>>>>> Minimal Theoretical Benefit. >>>>>>> Because modern hardware does not slap us with the same cost for >>>>>>> accessing memory as it once did, the benefit of replacing memory >>>>>>> access with register access is far less dramatic now than it once was. >>>>>>> Additionally, the interpreter runs for a relatively short time before >>>>>>> relevant code sections are compiled. When the VM starts running >>>>>>> compiled code instead of interpreted code, performance should begin to >>>>>>> move asymptotically towards that of compiled code, diluting any >>>>>>> performance penalties from the interpreter to small performance >>>>>>> variations. >>>>>>> >>>>>>> No Measurable Benefit. >>>>>>> Please see the results files attached in the bug page. This change >>>>>>> was adapted for x86 and sparc, and interpreter performance was >>>>>>> measured with Specjvm98 (run with -Xint). No significant decrease in >>>>>>> performance was observed. >>>>>>> >>>>>>> Memory footprint and code complexity. >>>>>>> Stack caching in the JVM is implemented by switching the instruction >>>>>>> look-up table depending on the tos (top-of-stack) state. At any moment >>>>>>> there are is an active table consisting of one dispatch table for each >>>>>>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>>>>>> safepoint dispatch tables into the active table. The additional entry >>>>>>> code makes this copy less efficient and makes any work in the >>>>>>> interpreter harder to debug. >>>>>>> >>>>>>> If we remove this optimization, we will: >>>>>>> - decrease memory usage in the interpreter, >>>>>>> - eliminated wasteful memory transactions during safepoints, >>>>>>> - decrease code complexity (a lot). >>>>>>> >>>>>>> Please let me know what you think. >>>>>>> Thanks, >>>>>>> Max >>>>>>> From stuart.monteith at linaro.org Fri Feb 24 18:06:21 2017 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Fri, 24 Feb 2017 18:06:21 +0000 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> <664fc325-3779-aadb-c6ae-eaba41648468@oracle.com> Message-ID: Hello, After applying the patch to the latest tree jdk9/dev tree, it builds cleanly and the hotspot JTReg tests pass. I'm currently running with jdk and langtools. BR, Stuart On 24 February 2017 at 16:36, Andrew Dinn wrote: > Hi Mikael, > > On 24/02/17 12:02, Mikael Gerdin wrote: >> Actually CC:ing Andrew. > > I'm still building this and will test it as soon as I can. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From ioi.lam at oracle.com Fri Feb 24 18:50:07 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 24 Feb 2017 10:50:07 -0800 Subject: Determining the size of C++ vtables In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> Message-ID: <58B0805F.20205@oracle.com> On 2/23/17 11:17 PM, Thomas St?fe wrote: > Hi Ioi, > > After reading through the original mailthread referenced in the bug > report, the only reason the "just copy enough (200) slots from the > vtable in libjvm.so to the vtable in the image" approach does not work > for you is that you are afraid of hitting unmapped memory? If I > understand it correctly. > > If that is the case, why not copy 200 slots with SafeFetch and stop at > the first error? That should not incur much additional costs as long > as you do not hit an unmapped area, in which case you'd stop anyway. > Oh, that's a good simplification.I hadn't thought about using SafeFetch :-( One issue with it is the 200 slots are too big (ConstantPool needs just 10 slots), so finding the exact size is a little more efficient and saves a few KBs. Thanks - Ioi > That way you make yourself a bit less dependent on compiler internals. > I agree that your approach is more elegant, though. > > Kind Regards, Thomas > > On Fri, Feb 24, 2017 at 4:55 AM, Ioi Lam > wrote: > > > > On 2/23/17 7:47 PM, Ioi Lam wrote: > > Hi, > > I am working on > https://bugs.openjdk.java.net/browse/JDK-8005165 > (Remove > CPU-dependent code in self-patching vtables), I need a way > find out the size > of a C++ vtable. I ended up doing this: > > > // Objects of the Metadata types (such as Klass and > ConstantPool) have C++ vtables. > // (In GCC this is the field ::_vptr, i.e., first word > in the object.) > // > // Addresses of the vtables and the methods may be different > across JVM runs, > // if libjvm.so is dynamically loaded at a different base address. > // > // To ensure that the Metadata objects in the CDS archive > always have the correct vtable: > // > // + at dump time: we redirect the _vptr to point to our own > vtables inside > // the CDS image > // + at run time: we clone the actual contents of the > vtables from libjvm.so > // into our own tables. > // > // To determine the size of the vtable for each type, we use > the following > // trick by declaring 2 subclasses: > // > // class CppVtabTesterA: public InstanceKlass { > // virtual int last_virtual_method() {return 1;} > // }; > // class CppVtabTesterB: public InstanceKlass { > // virtual void* last_virtual_method() {return NULL}; > // }; > // > // CppVtabTesterA and CppVtabTesterB's vtables have the > following properties: > // - Their size (N+1) is exactly one more than the size of > InstanceKlass's vtable (N) > // - The first N entries have are exactly the same as in > InstanceKlass's vtable. > // - Their last entry is different. > // > // So to determine the value of N, we just walk CppVtabTesterA > and CppVtabTesterB's tables > // and find the first entry that's different > > > Could anyone comment if this is acceptable? I know it's not > 100% portable (C++ doesn't > specify where to find the vtable, or what's inside), but my > assumptions is the same as > the existing code. I.e., _vptr is a pointer located at offset > 0 of the object, and it > points to a one-dimensional array. > > So at least it's not any worse than before? > > Thanks > - Ioi > > By the way, I first tried having only a single "tester" class and > walk the vtable to look for &last_virtual_method, but the C++ > compiler told me that taking the address of a non-static function > is not allowed ..... so I ended up creating two tester classes and > checking their differences. > > > > From ioi.lam at oracle.com Fri Feb 24 18:56:02 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 24 Feb 2017 10:56:02 -0800 Subject: Determining the size of C++ vtables In-Reply-To: <58AFAEAE.2080205@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> Message-ID: <58B081C2.5090204@oracle.com> On 2/23/17 7:55 PM, Ioi Lam wrote: > > > On 2/23/17 7:47 PM, Ioi Lam wrote: >> Hi, >> >> I am working on https://bugs.openjdk.java.net/browse/JDK-8005165 (Remove >> CPU-dependent code in self-patching vtables), I need a way find out >> the size >> of a C++ vtable. I ended up doing this: >> >> >> // Objects of the Metadata types (such as Klass and ConstantPool) >> have C++ vtables. >> // (In GCC this is the field ::_vptr, i.e., first word in the >> object.) >> // >> // Addresses of the vtables and the methods may be different across >> JVM runs, >> // if libjvm.so is dynamically loaded at a different base address. >> // >> // To ensure that the Metadata objects in the CDS archive always have >> the correct vtable: >> // >> // + at dump time: we redirect the _vptr to point to our own vtables >> inside >> // the CDS image >> // + at run time: we clone the actual contents of the vtables from >> libjvm.so >> // into our own tables. >> // >> // To determine the size of the vtable for each type, we use the >> following >> // trick by declaring 2 subclasses: >> // >> // class CppVtabTesterA: public InstanceKlass { >> // virtual int last_virtual_method() {return 1;} >> // }; >> // class CppVtabTesterB: public InstanceKlass { >> // virtual void* last_virtual_method() {return NULL}; >> // }; >> // >> // CppVtabTesterA and CppVtabTesterB's vtables have the following >> properties: >> // - Their size (N+1) is exactly one more than the size of >> InstanceKlass's vtable (N) >> // - The first N entries have are exactly the same as in >> InstanceKlass's vtable. >> // - Their last entry is different. >> // >> // So to determine the value of N, we just walk CppVtabTesterA and >> CppVtabTesterB's tables >> // and find the first entry that's different >> >> >> Could anyone comment if this is acceptable? I know it's not 100% >> portable (C++ doesn't >> specify where to find the vtable, or what's inside), but my >> assumptions is the same as >> the existing code. I.e., _vptr is a pointer located at offset 0 of >> the object, and it >> points to a one-dimensional array. >> >> So at least it's not any worse than before? >> >> Thanks >> - Ioi >> > By the way, I first tried having only a single "tester" class and walk > the vtable to look for &last_virtual_method, but the C++ compiler told > me that taking the address of a non-static function is not allowed > ..... so I ended up creating two tester classes and checking their > differences. > > Just to clarify the above comments: taking a "reference" to a virtual function is allowed, but it's not specified what the "reference" is. With gcc, it's a 16-bit value: #include class CppVtabTester { public: virtual int func1() {return 1;} int func2() {return 2;} }; int main() { union { int (CppVtabTester::*ptr)(); void* val[2]; } a, b; a.ptr = &CppVtabTester::func1; b.ptr = &CppVtabTester::func2; printf("%d\n", (int)sizeof(a.ptr)); printf("ref: %p %p\n", a.val[0], a.val[1]); printf("ref: %p %p\n", b.val[0], b.val[1]); return 0; } ioimac ~/tmp$ gcc t.cpp ioimac ~/tmp$ ./a.out 16 ref: 0x1 0x0 ref: 0x102d93f70 0x0 Unfortunately, the reference for a virtual function is just its vtable index :-( Taking the "address" of a virtual function in some versions of gcc will give the real address (with a warning), and some versions of gcc will disallow it: printf("ptr: %p\n", (void*)(&CppVtabTester::func1)); ..... t.cpp:21:35: error: cannot cast from type 'int (CppVtabTester::*)()' to pointer type 'void *' printf("ptr : %p\n", (void*)(&CppVtabTester::func1)); From thomas.stuefe at gmail.com Sat Feb 25 07:52:56 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 25 Feb 2017 08:52:56 +0100 Subject: Determining the size of C++ vtables In-Reply-To: <58B0805F.20205@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> Message-ID: Hi ioi, On Fri, Feb 24, 2017 at 7:50 PM, Ioi Lam wrote: > > > On 2/23/17 11:17 PM, Thomas St?fe wrote: > > Hi Ioi, > > After reading through the original mailthread referenced in the bug > report, the only reason the "just copy enough (200) slots from the vtable > in libjvm.so to the vtable in the image" approach does not work for you is > that you are afraid of hitting unmapped memory? If I understand it > correctly. > > If that is the case, why not copy 200 slots with SafeFetch and stop at the > first error? That should not incur much additional costs as long as you do > not hit an unmapped area, in which case you'd stop anyway. > > Oh, that's a good simplification.I hadn't thought about using SafeFetch :-( > > One issue with it is the 200 slots are too big (ConstantPool needs just 10 > slots), so finding the exact size is a little more efficient and saves a > few KBs. > > How many vtables are you copying? Does it really matter? As I said, I think your approach is quite elegant, but not terribly portable. We already make some assumptions (that vtables are organized as one continuous block of function pointers) and you add some more assumptions about the order of the methods in the vtable. I think even if you can make sure that it works now with our set of platforms and compilers, this adds the risk that a future compiler update may change the vtable layout and break the code. E.g. a vtable dumped with a VM compilers by an older compiler could not be loaded with the newly compiled VM. Also, you set the bar a bit higher for future ports, because you add requirements for the C++ compiler. But I may be too cautious here and I am waiting for others to chime in. As a side note, it would make sense to write a tiny C++ test which copies vtables around using your technique and then checks if they still work; such a test we could use on out platforms to check if our compilers can cope. Such a test would also be a good sanity check to add to the VM initialization if CDS is active. Kind Regards, Thomas > Thanks > - Ioi > > > That way you make yourself a bit less dependent on compiler internals. I > agree that your approach is more elegant, though. > > Kind Regards, Thomas > > On Fri, Feb 24, 2017 at 4:55 AM, Ioi Lam wrote: > >> >> >> On 2/23/17 7:47 PM, Ioi Lam wrote: >> >>> Hi, >>> >>> I am working on https://bugs.openjdk.java.net/browse/JDK-8005165 (Remove >>> CPU-dependent code in self-patching vtables), I need a way find out the >>> size >>> of a C++ vtable. I ended up doing this: >>> >>> >>> // Objects of the Metadata types (such as Klass and ConstantPool) have >>> C++ vtables. >>> // (In GCC this is the field ::_vptr, i.e., first word in the >>> object.) >>> // >>> // Addresses of the vtables and the methods may be different across JVM >>> runs, >>> // if libjvm.so is dynamically loaded at a different base address. >>> // >>> // To ensure that the Metadata objects in the CDS archive always have >>> the correct vtable: >>> // >>> // + at dump time: we redirect the _vptr to point to our own vtables >>> inside >>> // the CDS image >>> // + at run time: we clone the actual contents of the vtables from >>> libjvm.so >>> // into our own tables. >>> // >>> // To determine the size of the vtable for each type, we use the >>> following >>> // trick by declaring 2 subclasses: >>> // >>> // class CppVtabTesterA: public InstanceKlass { >>> // virtual int last_virtual_method() {return 1;} >>> // }; >>> // class CppVtabTesterB: public InstanceKlass { >>> // virtual void* last_virtual_method() {return NULL}; >>> // }; >>> // >>> // CppVtabTesterA and CppVtabTesterB's vtables have the following >>> properties: >>> // - Their size (N+1) is exactly one more than the size of >>> InstanceKlass's vtable (N) >>> // - The first N entries have are exactly the same as in InstanceKlass's >>> vtable. >>> // - Their last entry is different. >>> // >>> // So to determine the value of N, we just walk CppVtabTesterA and >>> CppVtabTesterB's tables >>> // and find the first entry that's different >>> >>> >>> Could anyone comment if this is acceptable? I know it's not 100% >>> portable (C++ doesn't >>> specify where to find the vtable, or what's inside), but my assumptions >>> is the same as >>> the existing code. I.e., _vptr is a pointer located at offset 0 of the >>> object, and it >>> points to a one-dimensional array. >>> >>> So at least it's not any worse than before? >>> >>> Thanks >>> - Ioi >>> >>> By the way, I first tried having only a single "tester" class and walk >> the vtable to look for &last_virtual_method, but the C++ compiler told me >> that taking the address of a non-static function is not allowed ..... so I >> ended up creating two tester classes and checking their differences. >> >> >> >> > > From david.holmes at oracle.com Mon Feb 27 07:10:51 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 27 Feb 2017 17:10:51 +1000 Subject: Linux/PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: <58B020D7.3020508@linux.vnet.ibm.com> References: <5898EF94.5060904@linux.vnet.ibm.com> <8a0d9151-8ed7-9409-20a1-10258023ae7d@oracle.com> <58B020D7.3020508@linux.vnet.ibm.com> Message-ID: <4088ef7e-3620-3e6a-fd3f-0ad84baba2c9@oracle.com> Hi Gustavo, I am not a NUMA expert but it seems to me that our NUMA support is both incomplete and bit-rotting. It seems evident that UseNUMA is only working in limited contexts that match our testing environment. There were a couple of JEPS proposed to enhance NUMA support back in 2012: JDK-8046147 JEP 157: G1 GC: NUMA-Aware Allocation JDK-8046153 JEP 163: Enable NUMA Mode by Default When Appropriate but they have not progressed. If they were to progress then it seems our overall approach to NUMA would need serious review and update - as per your patch. I'm also unclear about the distinctions between memory and non-memory nodes wrt. the existing os::Linux NUMA API. It isn't at all clear to me what functions should only be dealing with memory-nodes and which should deal with any kind eg. I expect cpu to node map to map cpu to nodes not cpu to nearest node with memory configured. If that is what is needed then the API's need to be changed and the usage checked - aas that distinction does not presently exist in the code AFAICS. It is too late to take this patch into 9 IMHO as we don't have the ability to test it effectively, nor is there time for NUMA users to put it through its paces. I think this would have to be part of a bigger NUMA project for 10 that addresses the NUMA API and how it is used. Thanks, David On 24/02/2017 10:02 PM, Gustavo Romero wrote: > Hi Sangheon, > > Please find my comments inline. > > On 06-02-2017 20:23, sangheon wrote: >> Hi Gustavo, >> >> On 02/06/2017 01:50 PM, Gustavo Romero wrote: >>> Hi, >>> >>> On Linux/PPC64 I'm getting a series of "mbind: Invalid argument" that seems >>> exactly the same as reported for x64 [1]: >>> >>> [root at spocfire3 ~]# java -XX:+UseNUMA -version >>> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument >>> openjdk version "1.8.0_121" >>> OpenJDK Runtime Environment (build 1.8.0_121-b13) >>> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) >>> >>> [root at spocfire3 ~]# uname -a >>> Linux spocfire3.aus.stglabs.ibm.com 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux >>> >>> [root at spocfire3 ~]# lscpu >>> Architecture: ppc64le >>> Byte Order: Little Endian >>> CPU(s): 160 >>> On-line CPU(s) list: 0-159 >>> Thread(s) per core: 8 >>> Core(s) per socket: 10 >>> Socket(s): 2 >>> NUMA node(s): 2 >>> Model: 2.0 (pvr 004d 0200) >>> Model name: POWER8 (raw), altivec supported >>> L1d cache: 64K >>> L1i cache: 32K >>> L2 cache: 512K >>> L3 cache: 8192K >>> NUMA node0 CPU(s): 0-79 >>> NUMA node8 CPU(s): 80-159 >>> >>> On chasing down it, looks like it comes from PSYoungGen::initialize() in >>> src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp that calls >>> initialize_work(), that calls the MutableNUMASpace() constructor if >>> UseNUMA is set: >>> http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/567e410935e5/src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp#l77 >>> >>> MutableNUMASpace() then calls os::numa_make_local(), that in the end calls >>> numa_set_bind_policy() in libnuma.so.1 [2]. >>> >>> I've traced some values for which mbind() syscall fails: >>> http://termbin.com/ztfs (search for "Invalid argument" in the log). >>> >>> Assuming it's the same bug as reported in [1] and so it's not fixed on 9 and 10: >>> >>> - Is there any WIP or known workaround? >> There's no progress on JDK-8163796 and no workaround found yet. >> And unfortunately, I'm not planning to fix it soon. > > Hive, a critical component of Hadoop ecosystem, comes with a shell and uses java > (with UseNUMA flag) in the background to run mysql-kind of queries. On PPC64 the > mbind() messages in question make the shell pretty cumbersome. For instance: > > hive> show databases; > mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument (repeat message more 28 times...) > ... > OK > mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument > default > tpcds_bin_partitioned_orc_10 > tpcds_text_10 > Time taken: 1.036 seconds, Fetched: 3 row(s) > hive> mbind: Invalid argument > mbind: Invalid argument > mbind: Invalid argument > > Also, on PPC64 a simple "java -XX:+UseParallelGC -XX:+UseNUMA -version" will > trigger the problem, without any additional flags. So I'd like to correct that > behavior (please see my next comment on that). > > >>> - Should I append this output in [1] description or open a new one and make it >>> related to" [1]? >> I think your problem seems same as JDK-8163796, so adding your output on the CR seems good. >> And please add logs as well. I recommend to enabling something like "-Xlog:gc*,gc+heap*=trace". >> IIRC, the problem was only occurred when the -Xmx was small in my case. > > JVM code used to discover which numa nodes it can bind assumes that nodes are > consecutive and tries to bind from 0 to numa_max_node() [1, 2, 3, 4], i.e. from > 0 to the highest node number available on the system. However, at least on PPC64 > that assumption is not always true. For instance, consider the following numa > topology: > > available: 4 nodes (0-1,16-17) > node 0 cpus: 0 8 16 24 32 > node 0 size: 130706 MB > node 0 free: 145 MB > node 1 cpus: 40 48 56 64 72 > node 1 size: 0 MB > node 1 free: 0 MB > node 16 cpus: 80 88 96 104 112 > node 16 size: 130630 MB > node 16 free: 529 MB > node 17 cpus: 120 128 136 144 152 > node 17 size: 0 MB > node 17 free: 0 MB > node distances: > node 0 1 16 17 > 0: 10 20 40 40 > 1: 20 10 40 40 > 16: 40 40 10 20 > 17: 40 40 20 10 > > In that case we have four nodes, 2 without memory (1 and 17), where the > highest node id is 17. Hence if the JVM tries to bind from 0 to 17, mbind() will > fail except for nodes 0 and 16, which are configured and have memory. mbind() > failures will generate the "mbind: Invalid argument" messages. > > A solution would be to use in os::numa_get_group_num() not numa_max_node() but > instead numa_num_configured_nodes() which returns the total number of nodes with > memory in the system (so in our example above it will return exactly 2 nodes) > and then inspect numa_all_node_ptr in os::numa_get_leaf_groups() to find the > correct node ids to append (in our case, map ids[0] = 0 [node 0] and ids[1] = 16 > [node 16]). > > One thing is that os::numa_get_leaf_groups() argument "size" will not be > required anymore and will be loose, so the interface will have to be adapted on > other OSs besides Linux I guess [5]. > > It would be necessary to adapt os::Linux::rebuild_cpu_to_node_map() > since not all numa nodes are suitable to be returned by a call to > os::numa_get_group_id() as some cpus would be in a node without memory. > In that case we can return the closest numa node instead. A new way to translate > indices to nodes is also useful since nodes are not always consecutive. > > Finally, although "numa_nodes_ptr" is not present in libnuma's manual it's what > is used in numactl to find out the total number of nodes in the system [6]. I > could not find a function that would return that number readily. I asked to > libnuma ML if a better solution exists [7]. > > The following webrev implements the proposed changes on jdk9 (backport to 8 is > simple): > > webrev: http://cr.openjdk.java.net/~gromero/8175813/ > bug: https://bugs.openjdk.java.net/browse/JDK-8175813 > > Here are the logs with "-Xlog:gc*,gc+heap*=trace": > > http://cr.openjdk.java.net/~gromero/logs/pristine.log (current state) > http://cr.openjdk.java.net/~gromero/logs/numa_patched.log (proposed change) > > I've tested on 8 against SPECjvm2008 on the aforementioned machine and > performance improved ~5% in comparison to the same version packaged by > the distro, but I don't expect any difference on machines where nodes > are always consecutive and where nodes always have memory. > > After a due community review, could you sponsor that change? > > Thank you. > > > Best regards, > Gustavo > > [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l241 > [2] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2745 > [3] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l243 > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2761 > [5] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/runtime/os.hpp#l356 > [6] https://github.com/numactl/numactl/blob/master/numactl.c#L251 > [7] http://www.spinics.net/lists/linux-numa/msg01173.html > >> >> Thanks, >> Sangheon >> >> >>> >>> Thank you. >>> >>> >>> Best regards, >>> Gustavo >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8163796 >>> [2] https://da.gd/4vXF >>> >> > From martijnverburg at gmail.com Mon Feb 27 12:03:37 2017 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 27 Feb 2017 12:03:37 +0000 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) Message-ID: Hi all, The topic of adding more useful information to an NPE came up on the Java Champions list and after some digging we discovered some implementations elsewhere (SAP VM for example) and some historical conversations / issues. https://bugs.openjdk.java.net/browse/JDK-4834738 http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html The bug is listed as won't fix but didn't have any explanation added to it as to why. Is it just a case that the JIT and/or other mechanisms strips away that sort of information at runtime? Cheers, Martijn From david.holmes at oracle.com Mon Feb 27 12:45:04 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 27 Feb 2017 22:45:04 +1000 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) In-Reply-To: References: Message-ID: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> Hi Martijn, On 27/02/2017 10:03 PM, Martijn Verburg wrote: > Hi all, > > The topic of adding more useful information to an NPE came up on the Java > Champions list and after some digging we discovered some implementations > elsewhere (SAP VM for example) and some historical conversations / issues. > > https://bugs.openjdk.java.net/browse/JDK-4834738 > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html As I state in that linked topic that request was almost the inverse of other requests in this area - it wanted to report the method invoked on the null reference, whereas other requests were seeking more information on where the null arose eg in: a().b().c(); which invocation produced the NPE. But thinking about it again, perhaps reporting the member access that triggered the NPE is the most generally useful information to provide. > The bug is listed as won't fix but didn't have any explanation added to it > as to why. Is it just a case that the JIT and/or other mechanisms strips > away that sort of information at runtime? The bug was closed as "won't fix" as part of a general cleanup up of old issues for which there are no active plans to address the issue. As I've commented in some of the issues where this was raised (and they are all quite old) there is a general lack of context available to give a meaningful error message about the NPE. Cheers, David > Cheers, > Martijn > From adinn at redhat.com Mon Feb 27 12:52:07 2017 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 27 Feb 2017 12:52:07 +0000 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> <664fc325-3779-aadb-c6ae-eaba41648468@oracle.com> Message-ID: <1d5b9339-b8d1-5be7-89fe-c80189efd16d@redhat.com> Hi Stuart, On 24/02/17 18:06, Stuart Monteith wrote: > Hello, > After applying the patch to the latest tree jdk9/dev tree, it builds > cleanly and the hotspot JTReg tests pass. I'm currently running with > jdk and langtools. Apologies for the delay in reporting my results. My build also completed cleanly and pass the hotspot jtreg test. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From martijnverburg at gmail.com Mon Feb 27 13:06:08 2017 From: martijnverburg at gmail.com (Martijn Verburg) Date: Mon, 27 Feb 2017 13:06:08 +0000 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) In-Reply-To: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> References: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> Message-ID: Hi David, Thanks for the explanation, much appreciated! Cheers, Martijn On 27 February 2017 at 12:45, David Holmes wrote: > Hi Martijn, > > On 27/02/2017 10:03 PM, Martijn Verburg wrote: > >> Hi all, >> >> The topic of adding more useful information to an NPE came up on the Java >> Champions list and after some digging we discovered some implementations >> elsewhere (SAP VM for example) and some historical conversations / issues. >> >> https://bugs.openjdk.java.net/browse/JDK-4834738 >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html >> > > As I state in that linked topic that request was almost the inverse of > other requests in this area - it wanted to report the method invoked on the > null reference, whereas other requests were seeking more information on > where the null arose eg in: > > a().b().c(); > > which invocation produced the NPE. But thinking about it again, perhaps > reporting the member access that triggered the NPE is the most generally > useful information to provide. > > The bug is listed as won't fix but didn't have any explanation added to it >> as to why. Is it just a case that the JIT and/or other mechanisms strips >> away that sort of information at runtime? >> > > The bug was closed as "won't fix" as part of a general cleanup up of old > issues for which there are no active plans to address the issue. > > As I've commented in some of the issues where this was raised (and they > are all quite old) there is a general lack of context available to give a > meaningful error message about the NPE. > > Cheers, > David > > Cheers, >> Martijn >> >> From mikael.gerdin at oracle.com Mon Feb 27 15:58:12 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 27 Feb 2017 16:58:12 +0100 Subject: 8175085: [REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <1d5b9339-b8d1-5be7-89fe-c80189efd16d@redhat.com> References: <8a67c9e4-92b4-93ca-2277-7b4dbf3e7126@oracle.com> <812681d5-96f6-0419-82d5-7e8ac1768c48@oracle.com> <0ac068fc-4c94-e2b1-3475-ba8dc441d5fa@oracle.com> <8d3420a6-ced0-28c0-2ab2-552795ca09b3@oracle.com> <664fc325-3779-aadb-c6ae-eaba41648468@oracle.com> <1d5b9339-b8d1-5be7-89fe-c80189efd16d@redhat.com> Message-ID: <68753b8d-af19-7ad5-94a8-53a0fc2d5aa4@oracle.com> Hi, On 2017-02-27 13:52, Andrew Dinn wrote: > Hi Stuart, > > On 24/02/17 18:06, Stuart Monteith wrote: >> Hello, >> After applying the patch to the latest tree jdk9/dev tree, it builds >> cleanly and the hotspot JTReg tests pass. I'm currently running with >> jdk and langtools. > > Apologies for the delay in reporting my results. My build also completed > cleanly and pass the hotspot jtreg test. Thanks guys. I'll go ahead and push this tomorow. /Mikael > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From bob.vandette at oracle.com Mon Feb 27 16:15:13 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 27 Feb 2017 11:15:13 -0500 Subject: RFR: 8175567: Build of hotspot for arm-vfp-sflt fails Message-ID: BUG: https://bugs.openjdk.java.net/browse/JDK-8175567 DESCRIPTION: The changes that were done under (JDK-8160245 Clean up platform #defines in c1_LIR.hpp), broke the arm-vfp-sflt build. Here?s a link to that original bug: https://bugs.openjdk.java.net/browse/JDK-8160245 This change below corrects the issue. diff --git a/src/share/vm/c1/c1_LIR.hpp b/src/share/vm/c1/c1_LIR.hpp --- a/src/share/vm/c1/c1_LIR.hpp +++ b/src/share/vm/c1/c1_LIR.hpp @@ -613,7 +613,7 @@ // Platform dependant. static LIR_Opr double_fpu(int reg1, int reg2 = -1 /*fnoreg*/); -#ifdef __SOFTFP__ +#ifdef ARM32 static LIR_Opr single_softfp(int reg) { return (LIR_Opr)(intptr_t)((reg << LIR_OprDesc::reg1_shift) | LIR_OprDesc::float_type | Bob. From vladimir.kozlov at oracle.com Mon Feb 27 18:05:18 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 27 Feb 2017 10:05:18 -0800 Subject: RFR: 8175567: Build of hotspot for arm-vfp-sflt fails In-Reply-To: References: Message-ID: That change was done explicitly but that was not tested as I see it: "It guards single_softfp() and double_softfp() by __SOFTFP__. This is not used in any openJdk platform. I can not test this on the closed platforms ARM32 and PPC32." May be it should check both? Please, explain if not: #if defined(__SOFTFP__) || defined(ARM32) You need also update: #endif // __SOFTFP__ Thanks, Vladimir On 2/27/17 8:15 AM, Bob Vandette wrote: > BUG: > > https://bugs.openjdk.java.net/browse/JDK-8175567 > > DESCRIPTION: > > The changes that were done under (JDK-8160245 Clean up platform #defines in c1_LIR.hpp), > broke the arm-vfp-sflt build. > > Here?s a link to that original bug: > > https://bugs.openjdk.java.net/browse/JDK-8160245 > > This change below corrects the issue. > > diff --git a/src/share/vm/c1/c1_LIR.hpp b/src/share/vm/c1/c1_LIR.hpp > --- a/src/share/vm/c1/c1_LIR.hpp > +++ b/src/share/vm/c1/c1_LIR.hpp > @@ -613,7 +613,7 @@ > // Platform dependant. > static LIR_Opr double_fpu(int reg1, int reg2 = -1 /*fnoreg*/); > > -#ifdef __SOFTFP__ > +#ifdef ARM32 > static LIR_Opr single_softfp(int reg) { > return (LIR_Opr)(intptr_t)((reg << LIR_OprDesc::reg1_shift) | > LIR_OprDesc::float_type | > > Bob. > From bob.vandette at oracle.com Mon Feb 27 18:15:19 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 27 Feb 2017 13:15:19 -0500 Subject: RFR: 8175567: Build of hotspot for arm-vfp-sflt fails In-Reply-To: References: Message-ID: > On Feb 27, 2017, at 1:05 PM, Vladimir Kozlov wrote: > > That change was done explicitly but that was not tested as I see it: > > "It guards single_softfp() and double_softfp() by __SOFTFP__. > This is not used in any openJdk platform. I can not test this > on the closed platforms ARM32 and PPC32." > > May be it should check both? Please, explain if not: > > #if defined(__SOFTFP__) || defined(ARM32) > > You need also update: > > #endif // __SOFTFP__ __SOFTFP__ is only use for ARM32 ports so I simplified the ifdef. In reality, I should have used: #if defined(ARM32) && !defined(__ABI_HARD__) but the only downside is that we end up defining these two functions ARM32 platforms but they are not used in the __ABI_HARD__ case. Bob. > > Thanks, > Vladimir > > On 2/27/17 8:15 AM, Bob Vandette wrote: >> BUG: >> >> https://bugs.openjdk.java.net/browse/JDK-8175567 >> >> DESCRIPTION: >> >> The changes that were done under (JDK-8160245 Clean up platform #defines in c1_LIR.hpp), >> broke the arm-vfp-sflt build. >> >> Here?s a link to that original bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8160245 >> >> This change below corrects the issue. >> >> diff --git a/src/share/vm/c1/c1_LIR.hpp b/src/share/vm/c1/c1_LIR.hpp >> --- a/src/share/vm/c1/c1_LIR.hpp >> +++ b/src/share/vm/c1/c1_LIR.hpp >> @@ -613,7 +613,7 @@ >> // Platform dependant. >> static LIR_Opr double_fpu(int reg1, int reg2 = -1 /*fnoreg*/); >> >> -#ifdef __SOFTFP__ >> +#ifdef ARM32 >> static LIR_Opr single_softfp(int reg) { >> return (LIR_Opr)(intptr_t)((reg << LIR_OprDesc::reg1_shift) | >> LIR_OprDesc::float_type | >> >> Bob. >> From vladimir.kozlov at oracle.com Mon Feb 27 18:20:17 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 27 Feb 2017 10:20:17 -0800 Subject: RFR: 8175567: Build of hotspot for arm-vfp-sflt fails In-Reply-To: References: Message-ID: Thank you, Bob, for explanation. Reviewed. Vladimir On 2/27/17 10:15 AM, Bob Vandette wrote: > >> On Feb 27, 2017, at 1:05 PM, Vladimir Kozlov wrote: >> >> That change was done explicitly but that was not tested as I see it: >> >> "It guards single_softfp() and double_softfp() by __SOFTFP__. >> This is not used in any openJdk platform. I can not test this >> on the closed platforms ARM32 and PPC32." >> >> May be it should check both? Please, explain if not: >> >> #if defined(__SOFTFP__) || defined(ARM32) >> >> You need also update: >> >> #endif // __SOFTFP__ > > __SOFTFP__ is only use for ARM32 ports so I simplified the ifdef. > > In reality, I should have used: > > #if defined(ARM32) && !defined(__ABI_HARD__) > > but the only downside is that we end up defining these two functions ARM32 platforms > but they are not used in the __ABI_HARD__ case. > > Bob. > > >> >> Thanks, >> Vladimir >> >> On 2/27/17 8:15 AM, Bob Vandette wrote: >>> BUG: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8175567 >>> >>> DESCRIPTION: >>> >>> The changes that were done under (JDK-8160245 Clean up platform #defines in c1_LIR.hpp), >>> broke the arm-vfp-sflt build. >>> >>> Here?s a link to that original bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8160245 >>> >>> This change below corrects the issue. >>> >>> diff --git a/src/share/vm/c1/c1_LIR.hpp b/src/share/vm/c1/c1_LIR.hpp >>> --- a/src/share/vm/c1/c1_LIR.hpp >>> +++ b/src/share/vm/c1/c1_LIR.hpp >>> @@ -613,7 +613,7 @@ >>> // Platform dependant. >>> static LIR_Opr double_fpu(int reg1, int reg2 = -1 /*fnoreg*/); >>> >>> -#ifdef __SOFTFP__ >>> +#ifdef ARM32 >>> static LIR_Opr single_softfp(int reg) { >>> return (LIR_Opr)(intptr_t)((reg << LIR_OprDesc::reg1_shift) | >>> LIR_OprDesc::float_type | >>> >>> Bob. >>> > From sangheon.kim at oracle.com Mon Feb 27 23:17:37 2017 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Tue, 28 Feb 2017 00:17:37 +0100 Subject: Linux/PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: <4088ef7e-3620-3e6a-fd3f-0ad84baba2c9@oracle.com> References: <5898EF94.5060904@linux.vnet.ibm.com> <8a0d9151-8ed7-9409-20a1-10258023ae7d@oracle.com> <58B020D7.3020508@linux.vnet.ibm.com> <4088ef7e-3620-3e6a-fd3f-0ad84baba2c9@oracle.com> Message-ID: Hi Gustabvo, I am not in the PPC64 ML, so replying late. > >> After a due community review, could you sponsor that change? Sure, I can sponsor this patch after the review. Please initiate the review on jdk 10 base. Thanks, Sangheon > On Feb 27, 2017, at 8:10 AM, David Holmes wrote: > > Hi Gustavo, > > I am not a NUMA expert but it seems to me that our NUMA support is both incomplete and bit-rotting. It seems evident that UseNUMA is only working in limited contexts that match our testing environment. There were a couple of JEPS proposed to enhance NUMA support back in 2012: > > JDK-8046147 JEP 157: G1 GC: NUMA-Aware Allocation > JDK-8046153 JEP 163: Enable NUMA Mode by Default When Appropriate > > but they have not progressed. If they were to progress then it seems our overall approach to NUMA would need serious review and update - as per your patch. > > I'm also unclear about the distinctions between memory and non-memory nodes wrt. the existing os::Linux NUMA API. It isn't at all clear to me what functions should only be dealing with memory-nodes and which should deal with any kind eg. I expect cpu to node map to map cpu to nodes not cpu to nearest node with memory configured. If that is what is needed then the API's need to be changed and the usage checked - aas that distinction does not presently exist in the code AFAICS. > > It is too late to take this patch into 9 IMHO as we don't have the ability to test it effectively, nor is there time for NUMA users to put it through its paces. I think this would have to be part of a bigger NUMA project for 10 that addresses the NUMA API and how it is used. > > Thanks, > David > >> On 24/02/2017 10:02 PM, Gustavo Romero wrote: >> Hi Sangheon, >> >> Please find my comments inline. >> >>> On 06-02-2017 20:23, sangheon wrote: >>> Hi Gustavo, >>> >>>> On 02/06/2017 01:50 PM, Gustavo Romero wrote: >>>> Hi, >>>> >>>> On Linux/PPC64 I'm getting a series of "mbind: Invalid argument" that seems >>>> exactly the same as reported for x64 [1]: >>>> >>>> [root at spocfire3 ~]# java -XX:+UseNUMA -version >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> openjdk version "1.8.0_121" >>>> OpenJDK Runtime Environment (build 1.8.0_121-b13) >>>> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) >>>> >>>> [root at spocfire3 ~]# uname -a >>>> Linux spocfire3.aus.stglabs.ibm.com 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux >>>> >>>> [root at spocfire3 ~]# lscpu >>>> Architecture: ppc64le >>>> Byte Order: Little Endian >>>> CPU(s): 160 >>>> On-line CPU(s) list: 0-159 >>>> Thread(s) per core: 8 >>>> Core(s) per socket: 10 >>>> Socket(s): 2 >>>> NUMA node(s): 2 >>>> Model: 2.0 (pvr 004d 0200) >>>> Model name: POWER8 (raw), altivec supported >>>> L1d cache: 64K >>>> L1i cache: 32K >>>> L2 cache: 512K >>>> L3 cache: 8192K >>>> NUMA node0 CPU(s): 0-79 >>>> NUMA node8 CPU(s): 80-159 >>>> >>>> On chasing down it, looks like it comes from PSYoungGen::initialize() in >>>> src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp that calls >>>> initialize_work(), that calls the MutableNUMASpace() constructor if >>>> UseNUMA is set: >>>> http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/567e410935e5/src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp#l77 >>>> >>>> MutableNUMASpace() then calls os::numa_make_local(), that in the end calls >>>> numa_set_bind_policy() in libnuma.so.1 [2]. >>>> >>>> I've traced some values for which mbind() syscall fails: >>>> http://termbin.com/ztfs (search for "Invalid argument" in the log). >>>> >>>> Assuming it's the same bug as reported in [1] and so it's not fixed on 9 and 10: >>>> >>>> - Is there any WIP or known workaround? >>> There's no progress on JDK-8163796 and no workaround found yet. >>> And unfortunately, I'm not planning to fix it soon. >> >> Hive, a critical component of Hadoop ecosystem, comes with a shell and uses java >> (with UseNUMA flag) in the background to run mysql-kind of queries. On PPC64 the >> mbind() messages in question make the shell pretty cumbersome. For instance: >> >> hive> show databases; >> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument (repeat message more 28 times...) >> ... >> OK >> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument >> default >> tpcds_bin_partitioned_orc_10 >> tpcds_text_10 >> Time taken: 1.036 seconds, Fetched: 3 row(s) >> hive> mbind: Invalid argument >> mbind: Invalid argument >> mbind: Invalid argument >> >> Also, on PPC64 a simple "java -XX:+UseParallelGC -XX:+UseNUMA -version" will >> trigger the problem, without any additional flags. So I'd like to correct that >> behavior (please see my next comment on that). >> >> >>>> - Should I append this output in [1] description or open a new one and make it >>>> related to" [1]? >>> I think your problem seems same as JDK-8163796, so adding your output on the CR seems good. >>> And please add logs as well. I recommend to enabling something like "-Xlog:gc*,gc+heap*=trace". >>> IIRC, the problem was only occurred when the -Xmx was small in my case. >> >> JVM code used to discover which numa nodes it can bind assumes that nodes are >> consecutive and tries to bind from 0 to numa_max_node() [1, 2, 3, 4], i.e. from >> 0 to the highest node number available on the system. However, at least on PPC64 >> that assumption is not always true. For instance, consider the following numa >> topology: >> >> available: 4 nodes (0-1,16-17) >> node 0 cpus: 0 8 16 24 32 >> node 0 size: 130706 MB >> node 0 free: 145 MB >> node 1 cpus: 40 48 56 64 72 >> node 1 size: 0 MB >> node 1 free: 0 MB >> node 16 cpus: 80 88 96 104 112 >> node 16 size: 130630 MB >> node 16 free: 529 MB >> node 17 cpus: 120 128 136 144 152 >> node 17 size: 0 MB >> node 17 free: 0 MB >> node distances: >> node 0 1 16 17 >> 0: 10 20 40 40 >> 1: 20 10 40 40 >> 16: 40 40 10 20 >> 17: 40 40 20 10 >> >> In that case we have four nodes, 2 without memory (1 and 17), where the >> highest node id is 17. Hence if the JVM tries to bind from 0 to 17, mbind() will >> fail except for nodes 0 and 16, which are configured and have memory. mbind() >> failures will generate the "mbind: Invalid argument" messages. >> >> A solution would be to use in os::numa_get_group_num() not numa_max_node() but >> instead numa_num_configured_nodes() which returns the total number of nodes with >> memory in the system (so in our example above it will return exactly 2 nodes) >> and then inspect numa_all_node_ptr in os::numa_get_leaf_groups() to find the >> correct node ids to append (in our case, map ids[0] = 0 [node 0] and ids[1] = 16 >> [node 16]). >> >> One thing is that os::numa_get_leaf_groups() argument "size" will not be >> required anymore and will be loose, so the interface will have to be adapted on >> other OSs besides Linux I guess [5]. >> >> It would be necessary to adapt os::Linux::rebuild_cpu_to_node_map() >> since not all numa nodes are suitable to be returned by a call to >> os::numa_get_group_id() as some cpus would be in a node without memory. >> In that case we can return the closest numa node instead. A new way to translate >> indices to nodes is also useful since nodes are not always consecutive. >> >> Finally, although "numa_nodes_ptr" is not present in libnuma's manual it's what >> is used in numactl to find out the total number of nodes in the system [6]. I >> could not find a function that would return that number readily. I asked to >> libnuma ML if a better solution exists [7]. >> >> The following webrev implements the proposed changes on jdk9 (backport to 8 is >> simple): >> >> webrev: http://cr.openjdk.java.net/~gromero/8175813/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8175813 >> >> Here are the logs with "-Xlog:gc*,gc+heap*=trace": >> >> http://cr.openjdk.java.net/~gromero/logs/pristine.log (current state) >> http://cr.openjdk.java.net/~gromero/logs/numa_patched.log (proposed change) >> >> I've tested on 8 against SPECjvm2008 on the aforementioned machine and >> performance improved ~5% in comparison to the same version packaged by >> the distro, but I don't expect any difference on machines where nodes >> are always consecutive and where nodes always have memory. >> >> After a due community review, could you sponsor that change? >> >> Thank you. >> >> >> Best regards, >> Gustavo >> >> [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l241 >> [2] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2745 >> [3] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l243 >> [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2761 >> [5] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/runtime/os.hpp#l356 >> [6] https://github.com/numactl/numactl/blob/master/numactl.c#L251 >> [7] http://www.spinics.net/lists/linux-numa/msg01173.html >> >>> >>> Thanks, >>> Sangheon >>> >>> >>>> >>>> Thank you. >>>> >>>> >>>> Best regards, >>>> Gustavo >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8163796 >>>> [2] https://da.gd/4vXF >>>> >>> >> From ioi.lam at oracle.com Tue Feb 28 01:14:17 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 27 Feb 2017 17:14:17 -0800 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) In-Reply-To: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> References: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> Message-ID: <58B4CEE9.1000407@oracle.com> Now sure if this has been suggested before -- how about including the BCI in the exception message. That will tell you exactly where the exception happened. In fact we can do this for any exception, not just NPEs. Like: try { someThingSecure().someThingElseSecure().evenMoreSecure(); } catch (Security Exception e) { e.printStackTrace(); } - Ioi On 2/27/17 4:45 AM, David Holmes wrote: > Hi Martijn, > > On 27/02/2017 10:03 PM, Martijn Verburg wrote: >> Hi all, >> >> The topic of adding more useful information to an NPE came up on the >> Java >> Champions list and after some digging we discovered some implementations >> elsewhere (SAP VM for example) and some historical conversations / >> issues. >> >> https://bugs.openjdk.java.net/browse/JDK-4834738 >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html >> > > As I state in that linked topic that request was almost the inverse of > other requests in this area - it wanted to report the method invoked > on the null reference, whereas other requests were seeking more > information on where the null arose eg in: > > a().b().c(); > > which invocation produced the NPE. But thinking about it again, > perhaps reporting the member access that triggered the NPE is the most > generally useful information to provide. > >> The bug is listed as won't fix but didn't have any explanation added >> to it >> as to why. Is it just a case that the JIT and/or other mechanisms >> strips >> away that sort of information at runtime? > > The bug was closed as "won't fix" as part of a general cleanup up of > old issues for which there are no active plans to address the issue. > > As I've commented in some of the issues where this was raised (and > they are all quite old) there is a general lack of context available > to give a meaningful error message about the NPE. > > Cheers, > David > >> Cheers, >> Martijn >> From david.holmes at oracle.com Tue Feb 28 01:34:24 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 28 Feb 2017 11:34:24 +1000 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) In-Reply-To: <58B4CEE9.1000407@oracle.com> References: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> <58B4CEE9.1000407@oracle.com> Message-ID: On 28/02/2017 11:14 AM, Ioi Lam wrote: > Now sure if this has been suggested before -- how about including the > BCI in the exception message. That will tell you exactly where the > exception happened. In fact we can do this for any exception, not just > NPEs. Like: One of the bug reports references BCIs. Not sure how useful that is to the end developer though as they typically want to map an exception to a place in the source code. Cheers, David > > try { > someThingSecure().someThingElseSecure().evenMoreSecure(); > } catch (Security Exception e) { > e.printStackTrace(); > } > > - Ioi > > On 2/27/17 4:45 AM, David Holmes wrote: >> Hi Martijn, >> >> On 27/02/2017 10:03 PM, Martijn Verburg wrote: >>> Hi all, >>> >>> The topic of adding more useful information to an NPE came up on the >>> Java >>> Champions list and after some digging we discovered some implementations >>> elsewhere (SAP VM for example) and some historical conversations / >>> issues. >>> >>> https://bugs.openjdk.java.net/browse/JDK-4834738 >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html >>> >> >> As I state in that linked topic that request was almost the inverse of >> other requests in this area - it wanted to report the method invoked >> on the null reference, whereas other requests were seeking more >> information on where the null arose eg in: >> >> a().b().c(); >> >> which invocation produced the NPE. But thinking about it again, >> perhaps reporting the member access that triggered the NPE is the most >> generally useful information to provide. >> >>> The bug is listed as won't fix but didn't have any explanation added >>> to it >>> as to why. Is it just a case that the JIT and/or other mechanisms >>> strips >>> away that sort of information at runtime? >> >> The bug was closed as "won't fix" as part of a general cleanup up of >> old issues for which there are no active plans to address the issue. >> >> As I've commented in some of the issues where this was raised (and >> they are all quite old) there is a general lack of context available >> to give a meaningful error message about the NPE. >> >> Cheers, >> David >> >>> Cheers, >>> Martijn >>> > From ioi.lam at oracle.com Tue Feb 28 01:52:39 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 27 Feb 2017 17:52:39 -0800 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) In-Reply-To: References: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> <58B4CEE9.1000407@oracle.com> Message-ID: <58B4D7E7.3040504@oracle.com> On 2/27/17 5:34 PM, David Holmes wrote: > On 28/02/2017 11:14 AM, Ioi Lam wrote: >> Now sure if this has been suggested before -- how about including the >> BCI in the exception message. That will tell you exactly where the >> exception happened. In fact we can do this for any exception, not just >> NPEs. Like: > > One of the bug reports references BCIs. Not sure how useful that is to > the end developer though as they typically want to map an exception to > a place in the source code. > It's better than nothing. I'd assume if this problem has bothered you so much to cause you to file a bug, then you would feel enough motivation to use the BCI :-) Anyhow, the Java class simply doesn't carry enough debugging information to tell you which position of a source line triggered the exception, like foo.bar(x.y(), a.b); Is it the dereference of foo, x or a? One possibility is to update the Java class file format: LineNumberTable_attribute { u2 attribute_name_index; u4 attribute_length; u2 line_number_table_length; { u2 start_pc; u2 line_number; } line_number_table[line_number_table_length]; } To add a "column_number" next to the line_number. Then, the exception message could be enhanced to java.lang.NullPointerException at Test.testNPE(Test.java:14:5) at Test.main(Test.java:8:xx) Where "5" is the column on line 14 that triggered the exception. - Ioi > Cheers, > David > >> >> try { >> someThingSecure().someThingElseSecure().evenMoreSecure(); >> } catch (Security Exception e) { >> e.printStackTrace(); >> } >> >> - Ioi >> >> On 2/27/17 4:45 AM, David Holmes wrote: >>> Hi Martijn, >>> >>> On 27/02/2017 10:03 PM, Martijn Verburg wrote: >>>> Hi all, >>>> >>>> The topic of adding more useful information to an NPE came up on the >>>> Java >>>> Champions list and after some digging we discovered some >>>> implementations >>>> elsewhere (SAP VM for example) and some historical conversations / >>>> issues. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-4834738 >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html >>>> >>>> >>> >>> As I state in that linked topic that request was almost the inverse of >>> other requests in this area - it wanted to report the method invoked >>> on the null reference, whereas other requests were seeking more >>> information on where the null arose eg in: >>> >>> a().b().c(); >>> >>> which invocation produced the NPE. But thinking about it again, >>> perhaps reporting the member access that triggered the NPE is the most >>> generally useful information to provide. >>> >>>> The bug is listed as won't fix but didn't have any explanation added >>>> to it >>>> as to why. Is it just a case that the JIT and/or other mechanisms >>>> strips >>>> away that sort of information at runtime? >>> >>> The bug was closed as "won't fix" as part of a general cleanup up of >>> old issues for which there are no active plans to address the issue. >>> >>> As I've commented in some of the issues where this was raised (and >>> they are all quite old) there is a general lack of context available >>> to give a meaningful error message about the NPE. >>> >>> Cheers, >>> David >>> >>>> Cheers, >>>> Martijn >>>> >> From david.holmes at oracle.com Tue Feb 28 01:58:53 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 28 Feb 2017 11:58:53 +1000 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) In-Reply-To: <58B4D7E7.3040504@oracle.com> References: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> <58B4CEE9.1000407@oracle.com> <58B4D7E7.3040504@oracle.com> Message-ID: <862662e8-56f9-a87e-3bfc-784ce8e19808@oracle.com> On 28/02/2017 11:52 AM, Ioi Lam wrote: > On 2/27/17 5:34 PM, David Holmes wrote: >> On 28/02/2017 11:14 AM, Ioi Lam wrote: >>> Now sure if this has been suggested before -- how about including the >>> BCI in the exception message. That will tell you exactly where the >>> exception happened. In fact we can do this for any exception, not just >>> NPEs. Like: >> >> One of the bug reports references BCIs. Not sure how useful that is to >> the end developer though as they typically want to map an exception to >> a place in the source code. >> > It's better than nothing. I'd assume if this problem has bothered you so > much to cause you to file a bug, then you would feel enough motivation > to use the BCI :-) Most Java developers don't even know what BCI stands for! :) > Anyhow, the Java class simply doesn't carry enough debugging information > to tell you which position of a source line triggered the exception, like > > foo.bar(x.y(), a.b); > > Is it the dereference of foo, x or a? Hence the suggestions to report the member access that triggered the NPE - eg: - call to bar() on null reference - call to y() on null reference - access of b on null reference > One possibility is to update the Java class file format: > > LineNumberTable_attribute { > u2 attribute_name_index; > u4 attribute_length; > u2 line_number_table_length; > { u2 start_pc; > u2 line_number; > } line_number_table[line_number_table_length]; > } > > To add a "column_number" next to the line_number. Then, the exception > message could be enhanced to > > java.lang.NullPointerException > at Test.testNPE(Test.java:14:5) > at Test.main(Test.java:8:xx) > > Where "5" is the column on line 14 that triggered the exception. :) Not sure how you'd carry that through to JITed code, but I like it. Cheers, David > - Ioi > > >> Cheers, >> David >> >>> >>> try { >>> someThingSecure().someThingElseSecure().evenMoreSecure(); >>> } catch (Security Exception e) { >>> e.printStackTrace(); >>> } >>> >>> - Ioi >>> >>> On 2/27/17 4:45 AM, David Holmes wrote: >>>> Hi Martijn, >>>> >>>> On 27/02/2017 10:03 PM, Martijn Verburg wrote: >>>>> Hi all, >>>>> >>>>> The topic of adding more useful information to an NPE came up on the >>>>> Java >>>>> Champions list and after some digging we discovered some >>>>> implementations >>>>> elsewhere (SAP VM for example) and some historical conversations / >>>>> issues. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-4834738 >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html >>>>> >>>>> >>>> >>>> As I state in that linked topic that request was almost the inverse of >>>> other requests in this area - it wanted to report the method invoked >>>> on the null reference, whereas other requests were seeking more >>>> information on where the null arose eg in: >>>> >>>> a().b().c(); >>>> >>>> which invocation produced the NPE. But thinking about it again, >>>> perhaps reporting the member access that triggered the NPE is the most >>>> generally useful information to provide. >>>> >>>>> The bug is listed as won't fix but didn't have any explanation added >>>>> to it >>>>> as to why. Is it just a case that the JIT and/or other mechanisms >>>>> strips >>>>> away that sort of information at runtime? >>>> >>>> The bug was closed as "won't fix" as part of a general cleanup up of >>>> old issues for which there are no active plans to address the issue. >>>> >>>> As I've commented in some of the issues where this was raised (and >>>> they are all quite old) there is a general lack of context available >>>> to give a meaningful error message about the NPE. >>>> >>>> Cheers, >>>> David >>>> >>>>> Cheers, >>>>> Martijn >>>>> >>> > From volker.simonis at gmail.com Tue Feb 28 10:40:29 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 28 Feb 2017 11:40:29 +0100 Subject: Non urgent question on https://bugs.openjdk.java.net/browse/JDK-4834738 (Adding which Object is null to an NPE) In-Reply-To: <862662e8-56f9-a87e-3bfc-784ce8e19808@oracle.com> References: <8d0c996e-075d-5712-b714-561cd0830f0c@oracle.com> <58B4CEE9.1000407@oracle.com> <58B4D7E7.3040504@oracle.com> <862662e8-56f9-a87e-3bfc-784ce8e19808@oracle.com> Message-ID: Hi, as Martijn mentioned, we've implemented something like this in our SAP JVM [1]. For a the following example: public final NPE a() { return new NPE(); } public final NPE b() { return null; } public final NPE c() { return new NPE(); } public static void m(NPE n) { n.a().b().c(); } calling m(new NPE()) would result in the following exception: java.lang.NullPointerException: while trying to invoke the method NPE.c() of a null object returned from NPE.b() at NPE.m(NPE.java:66) at NPE.main(NPE.java:74) We're considering to contribute this (and other enhancements) to the OpenJDK - it's just a matter of resources :) Moreover, this particular enhancement is not trivial. It requires bytecode parsing in order to find the source of a null pointer. The implementation also interacts with the -XX:+/-OmitStackTraceInFastThrow option. We, in the SAP JVM, have switched it off, so we always get the full exception message as posted above. But that comes at the cost of more deoptimizations, because exceptions will not be thrown right from compiled code. Oracle/OpenJDK have switched OmitStackTraceInFastThrow on by default. This leads to exceptions being thrown right from the JITed code (i.e. "fast throw"). But the drawback is that these are just empty, preallocated exceptions without message and stack trace. Regards, Volker [1] https://tools.hana.ondemand.com/#cloud On Tue, Feb 28, 2017 at 2:58 AM, David Holmes wrote: > On 28/02/2017 11:52 AM, Ioi Lam wrote: >> >> On 2/27/17 5:34 PM, David Holmes wrote: >>> >>> On 28/02/2017 11:14 AM, Ioi Lam wrote: >>>> >>>> Now sure if this has been suggested before -- how about including the >>>> BCI in the exception message. That will tell you exactly where the >>>> exception happened. In fact we can do this for any exception, not just >>>> NPEs. Like: >>> >>> >>> One of the bug reports references BCIs. Not sure how useful that is to >>> the end developer though as they typically want to map an exception to >>> a place in the source code. >>> >> It's better than nothing. I'd assume if this problem has bothered you so >> much to cause you to file a bug, then you would feel enough motivation >> to use the BCI :-) > > > Most Java developers don't even know what BCI stands for! :) > >> Anyhow, the Java class simply doesn't carry enough debugging information >> to tell you which position of a source line triggered the exception, like >> >> foo.bar(x.y(), a.b); >> >> Is it the dereference of foo, x or a? > > > Hence the suggestions to report the member access that triggered the NPE - > eg: > > - call to bar() on null reference > - call to y() on null reference > - access of b on null reference > >> One possibility is to update the Java class file format: >> >> LineNumberTable_attribute { >> u2 attribute_name_index; >> u4 attribute_length; >> u2 line_number_table_length; >> { u2 start_pc; >> u2 line_number; >> } line_number_table[line_number_table_length]; >> } >> >> To add a "column_number" next to the line_number. Then, the exception >> message could be enhanced to >> >> java.lang.NullPointerException >> at Test.testNPE(Test.java:14:5) >> at Test.main(Test.java:8:xx) >> >> Where "5" is the column on line 14 that triggered the exception. > > > :) Not sure how you'd carry that through to JITed code, but I like it. > > Cheers, > David > > >> - Ioi >> >> >>> Cheers, >>> David >>> >>>> >>>> try { >>>> someThingSecure().someThingElseSecure().evenMoreSecure(); >>>> } catch (Security Exception e) { >>>> e.printStackTrace(); >>>> } >>>> >>>> - Ioi >>>> >>>> On 2/27/17 4:45 AM, David Holmes wrote: >>>>> >>>>> Hi Martijn, >>>>> >>>>> On 27/02/2017 10:03 PM, Martijn Verburg wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> The topic of adding more useful information to an NPE came up on the >>>>>> Java >>>>>> Champions list and after some digging we discovered some >>>>>> implementations >>>>>> elsewhere (SAP VM for example) and some historical conversations / >>>>>> issues. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-4834738 >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013535.html >>>>>> >>>>>> >>>>> >>>>> As I state in that linked topic that request was almost the inverse of >>>>> other requests in this area - it wanted to report the method invoked >>>>> on the null reference, whereas other requests were seeking more >>>>> information on where the null arose eg in: >>>>> >>>>> a().b().c(); >>>>> >>>>> which invocation produced the NPE. But thinking about it again, >>>>> perhaps reporting the member access that triggered the NPE is the most >>>>> generally useful information to provide. >>>>> >>>>>> The bug is listed as won't fix but didn't have any explanation added >>>>>> to it >>>>>> as to why. Is it just a case that the JIT and/or other mechanisms >>>>>> strips >>>>>> away that sort of information at runtime? >>>>> >>>>> >>>>> The bug was closed as "won't fix" as part of a general cleanup up of >>>>> old issues for which there are no active plans to address the issue. >>>>> >>>>> As I've commented in some of the issues where this was raised (and >>>>> they are all quite old) there is a general lack of context available >>>>> to give a meaningful error message about the NPE. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> Cheers, >>>>>> Martijn >>>>>> >>>> >> > From Jacob.Wieland at spirent.com Tue Feb 28 11:02:06 2017 From: Jacob.Wieland at spirent.com (Wieland, Jacob) Date: Tue, 28 Feb 2017 11:02:06 +0000 Subject: throwing static exceptions sometimes VERY slow! In-Reply-To: References: Message-ID: Hi, I am observing a very strange behavior. In our generated code (due to various reasons I won't go into here unless I have to, but trust me, they are legit), we throw static exceptions for control flow purposes, This seems to work fine and without performance loss most of the time. However, we are observing now that every few seconds, a throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to the normal almost non-measurable time). Update: it seems that the observed behavior was kind of related with some extraneous circumstances (the computer seems to have been more lagged than usual at that point). By now, I can only observe throws that take up to 20 milliseconds (which opposed to the normal time still seems slow). It does not seem to be GC related, the only idea that I have is the jitter. So, my question is. Is this a known (and for some strange reason maybe even accepted) behavior or is this a bug that I should file with Oracle (or you guys). BR [X] Dr. Jacob Wieland Senior Software Engineer main +49.30.7261919.34 mobile +49.173.6446443 jacob.wieland at spirent.com www.spirent.com Follow us on: Spirent Communications [X]|[X]|[X] Michaelkirchstra?e 17/18 10179 Berlin, Germany +++++ N E W S F L A S H +++++ Spirent Communications and Testing Technologies join forces to become your test automation power-house. Read more at http://conta.cc/1S62BEY. Spirent Communications e-mail confidentiality. ------------------------------------------------------------------------ This e-mail contains confidential and / or privileged information belonging to Spirent Communications plc, its affiliates and / or subsidiaries. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution and / or the taking of any action based upon reliance on the contents of this transmission is strictly forbidden. If you have received this message in error please notify the sender by return e-mail and delete it from your system. Spirent Communications plc Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. Tel No. +44 (0) 1293 767676 Fax No. +44 (0) 1293 767677 Registered in England Number 470893 Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. Or if within the US, Spirent Communications, 27349 Agoura Road, Calabasas, CA, 91301, USA. Tel No. 1-818-676- 2300 From david.holmes at oracle.com Tue Feb 28 12:37:54 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 28 Feb 2017 22:37:54 +1000 Subject: throwing static exceptions sometimes VERY slow! In-Reply-To: References: Message-ID: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> Hi, On 28/02/2017 9:02 PM, Wieland, Jacob wrote: > Hi, > > I am observing a very strange behavior. > > In our generated code (due to various reasons I won't go into here unless I have to, but trust me, they are legit), we throw static exceptions for control flow purposes, This seems to work fine and without performance loss most of the time. However, we are observing now that every few seconds, a throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to the normal almost non-measurable time). > > Update: it seems that the observed behavior was kind of related with some extraneous circumstances (the computer seems to have been more lagged than usual at that point). By now, I can only observe throws that take up to 20 milliseconds (which opposed to the normal time still seems slow). > > It does not seem to be GC related, the only idea that I have is the jitter. What jitter? > So, my question is. Is this a known (and for some strange reason maybe even accepted) behavior or is this a bug that I should file with Oracle (or you guys). You really are not giving us anything to go on here. How are you observing this slowness? Exactly how do you throw? What exactly are you measuring? What's the execution context, the machine, processors etc etc etc. Regards, David > > BR > > > [X] > > > > > Dr. Jacob Wieland > > Senior Software Engineer > > main +49.30.7261919.34 > > mobile +49.173.6446443 > > > jacob.wieland at spirent.com > > > www.spirent.com > > > > > > > > > Follow us on: > > > > > Spirent Communications > > > [X]|[X]|[X] > > > > > Michaelkirchstra?e 17/18 > > 10179 Berlin, Germany > > > > > > +++++ N E W S F L A S H +++++ > > > > Spirent Communications and Testing Technologies join forces to become your test automation power-house. Read more at http://conta.cc/1S62BEY. > > > > > Spirent Communications e-mail confidentiality. > ------------------------------------------------------------------------ > This e-mail contains confidential and / or privileged information belonging to Spirent Communications plc, its affiliates and / or subsidiaries. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution and / or the taking of any action based upon reliance on the contents of this transmission is strictly forbidden. If you have received this message in error please notify the sender by return e-mail and delete it from your system. > > Spirent Communications plc > Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. > Tel No. +44 (0) 1293 767676 > Fax No. +44 (0) 1293 767677 > > Registered in England Number 470893 > Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. > > Or if within the US, > > Spirent Communications, > 27349 Agoura Road, Calabasas, CA, 91301, USA. > Tel No. 1-818-676- 2300 > From vitalyd at gmail.com Tue Feb 28 12:46:09 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 28 Feb 2017 07:46:09 -0500 Subject: throwing static exceptions sometimes VERY slow! In-Reply-To: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> References: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> Message-ID: On Tue, Feb 28, 2017 at 7:37 AM, David Holmes wrote: > Hi, > > On 28/02/2017 9:02 PM, Wieland, Jacob wrote: > >> Hi, >> >> I am observing a very strange behavior. >> >> In our generated code (due to various reasons I won't go into here unless >> I have to, but trust me, they are legit), we throw static exceptions for >> control flow purposes, This seems to work fine and without performance loss >> most of the time. However, we are observing now that every few seconds, a >> throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to the >> normal almost non-measurable time). >> >> Update: it seems that the observed behavior was kind of related with some >> extraneous circumstances (the computer seems to have been more lagged than >> usual at that point). By now, I can only observe throws that take up to 20 >> milliseconds (which opposed to the normal time still seems slow). >> >> It does not seem to be GC related, the only idea that I have is the >> jitter. >> > > What jitter? I'm assuming Jacob is referring to the JIT. But yes, he needs to provide more information. In particular, it would be good to know the following, as David mentioned, (off the top of my head): 1) JVM cmdline flags 2) Type of exception thrown (with a stacktrace or not) 3) Is the call stack depth about the same during the slow and fast throws? 4) Is the exception thrown frequently or infrequently? 5) Is there -XX:+PrintCompilation output available around the time when the slowdown is observed > > So, my question is. Is this a known (and for some strange reason maybe >> even accepted) behavior or is this a bug that I should file with Oracle (or >> you guys). >> > > You really are not giving us anything to go on here. How are you observing > this slowness? Exactly how do you throw? What exactly are you measuring? > What's the execution context, the machine, processors etc etc etc. > > Regards, > David > > > >> BR >> >> >> [X] >> >> >> >> >> Dr. Jacob Wieland >> >> Senior Software Engineer >> >> main +49.30.7261919.34 >> >> mobile +49.173.6446443 >> >> >> jacob.wieland at spirent.com >> >> >> www.spirent.com >> >> >> >> >> >> >> >> >> Follow us on: >> >> >> >> >> Spirent Communications >> >> >> [X]< >> https://www.linkedin.com/company/spirent-communications>| < >> https://twitter.com/Spirent>[X]| < >> https://www.facebook.com/spirent>[X] >> >> >> >> >> Michaelkirchstra?e 17/18 >> >> 10179 Berlin, Germany >> >> >> >> >> >> +++++ N E W S F L A S H +++++ >> >> >> >> Spirent Communications and Testing Technologies join forces to become >> your test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> >> >> >> Spirent Communications e-mail confidentiality. >> ------------------------------------------------------------------------ >> This e-mail contains confidential and / or privileged information >> belonging to Spirent Communications plc, its affiliates and / or >> subsidiaries. If you are not the intended recipient, you are hereby >> notified that any disclosure, copying, distribution and / or the taking of >> any action based upon reliance on the contents of this transmission is >> strictly forbidden. If you have received this message in error please >> notify the sender by return e-mail and delete it from your system. >> >> Spirent Communications plc >> Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United >> Kingdom. >> Tel No. +44 (0) 1293 767676 >> Fax No. +44 (0) 1293 767677 >> >> Registered in England Number 470893 >> Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 >> 9XN, United Kingdom. >> >> Or if within the US, >> >> Spirent Communications, >> 27349 Agoura Road, Calabasas, CA, 91301, USA. >> Tel No. 1-818-676- 2300 >> >> From martin.doerr at sap.com Tue Feb 28 15:00:12 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 28 Feb 2017 15:00:12 +0000 Subject: Discussion: 8172978: Remove Interpreter TOS optimization References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> Message-ID: <2c200ce3225445d49eb91c25b4094cc0@sap.com> Hi, I've ran jvm98 with -Xint on several PPC64 machines and on a recent s390x machine. Surprisingly, disabling of the Tos optimization does not hurt on older hardware (Power 5 and 6). Seems like some sub benchmarks don't suffer at all or even benefit. But it really hurts on recent Power 8. Measured performance change on AIX 7.1 on Power 8 -XX:+EnableTosCache vs. -XX:-EnableTosCache: Compress 38989 vs. 47686 -22% Jess 11256 vs. 11849 -5% Raytrace 17647 vs. 18300 -4% Db 22713 vs. 25181 -11% Javac 14554 vs. 15130 -4% These sub benchmarks are relatively stable. It's possible to reproduce these results. We lose about 3% on recent s390x hardware (z13). I first liked the idea to remove the optimization, but these numbers speak against doing it. Best regards, Martin -----Original Message----- From: Doerr, Martin Sent: Freitag, 24. Februar 2017 17:41 To: 'Max Ockner' ; hotspot-dev at openjdk.java.net Subject: RE: Discussion: 8172978: Remove Interpreter TOS optimization Hi Max, thank you very much for sharing your results and for sending the patch. I guess it covers the most relevant cases, but not all ones. I think it'd be better to modify dispatch_next instead of dispatch_epilog on x86. (dispatch_next is also used by generate_return_entry_for and generate_deopt_entry_for.) On s390, I'm using dispatch_next with: if (!EnableTosCache) { push(state); state = vtos; } dispatch_base(state, Interpreter::dispatch_table(state)); I also added an assertion to dispatch_base in order to make sure I'm hitting all dispatch usages: assert(EnableTosCache || state == vtos, "sanity"); Unfortunately, the performance results of SPEC jvm98 with -Xint seem to drop significantly with -XX:-EnableTosCache on both, PPC64 and s390. But we need to perform more measurements to get more reliable results. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Max Ockner Sent: Donnerstag, 23. Februar 2017 22:21 To: hotspot-dev at openjdk.java.net Subject: Re: Discussion: 8172978: Remove Interpreter TOS optimization Hi Volker, I have attached the patch that I have been testing. Thanks, Max On 2/20/2017 5:45 AM, Volker Simonis wrote: > Hi, > > besides the fact that this of course means some work for us :) I > currently don't see any problems for our porting platforms (ppc64 and > s390x). > > Are there any webrevs available, so we can see how big they are and > maybe do some own benchmarking? > > Thanks, > Volker > > > On Sun, Feb 19, 2017 at 11:11 PM, wrote: >> >> On 2/18/17 11:14 AM, coleen.phillimore at oracle.com wrote: >>> When Max gets back from the long weekend, he'll post the platforms in your >>> bug. >>> >>> It's amazing that for -Xint there's no significant difference. I've seen >>> -Xint performance of 15% slower cause a 2% slowdown with server but that was >>> before tiered compilation. >> >> I should clarify this. I've seen this slowdown for *different* interpreter >> optimizations, which *can* affect server performance. I was measuring >> specjvm98 on linux x64. If there's no significant difference for this TOS >> optimization, there is no chance of a degredation in overall performance. >> >> Coleen >> >>> The reason for this query was to see what developers for the other >>> platform ports think, since this change would affect all of the platforms. >>> >>> Thanks, >>> Coleen >>> >>> On 2/18/17 10:50 AM, Daniel D. Daugherty wrote: >>>> If Claes is happy with the perf testing, then I'm happy. :-) >>>> >>>> Dan >>>> >>>> >>>> On 2/18/17 3:46 AM, Claes Redestad wrote: >>>>> Hi, >>>>> >>>>> I've seen Max has run plenty of tests on our internal performance >>>>> infrastructure and everything I've seen there seems to corroborate the >>>>> idea that this removal is OK from a performance point of view, the >>>>> footprint improvements are small but significant and any negative >>>>> performance impact on throughput benchmarks is at noise levels even >>>>> with -Xint (it appears many benchmarks time out with this setting >>>>> both before and after, though; Max, let's discuss offline how to >>>>> deal with that :-)) >>>>> >>>>> I expect this will be tested more thoroughly once adapted to all >>>>> platforms (which I assume is the intent?), but see no concern from >>>>> a performance testing point of view: Do it! >>>>> >>>>> Thanks! >>>>> >>>>> /Claes >>>>> >>>>> On 2017-02-16 16:40, Daniel D. Daugherty wrote: >>>>>> Hi Max, >>>>>> >>>>>> Added a note to your bug. Interesting idea, but I think your data is >>>>>> a bit incomplete at the moment. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 2/15/17 3:18 PM, Max Ockner wrote: >>>>>>> Hello all, >>>>>>> >>>>>>> We have filed a bug to remove the interpreter stack caching >>>>>>> optimization for jdk10. Ideally we can make this change *early* >>>>>>> during the jdk10 development cycle. See below for justification: >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8172978 >>>>>>> >>>>>>> Stack caching has been around for a long time and is intended to >>>>>>> replace some of the load/store (pop/push) operations with >>>>>>> corresponding register operations. The need for this optimization >>>>>>> arose before caching could adequately lessen the burden of memory >>>>>>> access. We have reevaluated the JVM stack caching optimization and >>>>>>> have found that it has a high memory footprint and is very costly to >>>>>>> maintain, but does not provide significant measurable or theoretical >>>>>>> benefit for us when used with modern hardware. >>>>>>> >>>>>>> Minimal Theoretical Benefit. >>>>>>> Because modern hardware does not slap us with the same cost for >>>>>>> accessing memory as it once did, the benefit of replacing memory >>>>>>> access with register access is far less dramatic now than it once was. >>>>>>> Additionally, the interpreter runs for a relatively short time before >>>>>>> relevant code sections are compiled. When the VM starts running >>>>>>> compiled code instead of interpreted code, performance should begin to >>>>>>> move asymptotically towards that of compiled code, diluting any >>>>>>> performance penalties from the interpreter to small performance >>>>>>> variations. >>>>>>> >>>>>>> No Measurable Benefit. >>>>>>> Please see the results files attached in the bug page. This change >>>>>>> was adapted for x86 and sparc, and interpreter performance was >>>>>>> measured with Specjvm98 (run with -Xint). No significant decrease in >>>>>>> performance was observed. >>>>>>> >>>>>>> Memory footprint and code complexity. >>>>>>> Stack caching in the JVM is implemented by switching the instruction >>>>>>> look-up table depending on the tos (top-of-stack) state. At any moment >>>>>>> there are is an active table consisting of one dispatch table for each >>>>>>> of the 10 tos states. When we enter a safepoint, we copy all 10 >>>>>>> safepoint dispatch tables into the active table. The additional entry >>>>>>> code makes this copy less efficient and makes any work in the >>>>>>> interpreter harder to debug. >>>>>>> >>>>>>> If we remove this optimization, we will: >>>>>>> - decrease memory usage in the interpreter, >>>>>>> - eliminated wasteful memory transactions during safepoints, >>>>>>> - decrease code complexity (a lot). >>>>>>> >>>>>>> Please let me know what you think. >>>>>>> Thanks, >>>>>>> Max >>>>>>> From aph at redhat.com Tue Feb 28 15:05:06 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 28 Feb 2017 15:05:06 +0000 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <2c200ce3225445d49eb91c25b4094cc0@sap.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> <2c200ce3225445d49eb91c25b4094cc0@sap.com> Message-ID: <52c6a32c-6b36-ed5b-598f-0d3a1f9bf50b@redhat.com> On 28/02/17 15:00, Doerr, Martin wrote: > I first liked the idea to remove the optimization, but these numbers speak against doing it. Looks like it. I haven't investigated making such a change on AArch64, but I could if more data were needed. Andrew. From shade at redhat.com Tue Feb 28 15:14:03 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 28 Feb 2017 16:14:03 +0100 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <2c200ce3225445d49eb91c25b4094cc0@sap.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> <2c200ce3225445d49eb91c25b4094cc0@sap.com> Message-ID: <26eb5b4e-08ef-f5a7-957a-b8569f47ce7f@redhat.com> On 02/28/2017 04:00 PM, Doerr, Martin wrote: > I've ran jvm98 with -Xint on several PPC64 machines and on a recent s390x > machine. > > Surprisingly, disabling of the Tos optimization does not hurt on older > hardware (Power 5 and 6). Seems like some sub benchmarks don't suffer at all > or even benefit. But it really hurts on recent Power 8. I don't think it makes sense to run performance tests with -Xint alone. Of course removing interpreter optimizations would affect interpreter performance. The real question one should ask if turning off an interpreter optimization affects peak performance and time-to-performance when compilers are enabled. That's because in 2017 we should not expect that users who need performance would run with interpreter only. And removing complexity from interpreter without sacrificing the performance in tiered/compiled mode is certainly a plus in my book. Thanks, -Aleksey From martin.doerr at sap.com Tue Feb 28 15:23:25 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 28 Feb 2017 15:23:25 +0000 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <26eb5b4e-08ef-f5a7-957a-b8569f47ce7f@redhat.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> <2c200ce3225445d49eb91c25b4094cc0@sap.com> <26eb5b4e-08ef-f5a7-957a-b8569f47ce7f@redhat.com> Message-ID: <3d99a9def06c4c2d9823086173b898ad@sap.com> Hi Aleksey, of course, the peak performance is not really affected by this change. Ideally, one should measure the startup performance. However, these measurements are highly unstable, so one would need to spend more effort and many runs. That's why we use interpreter only tests. They are very stable and give us a hint about what's happening. The assumption behind this proposal was that interpreter performance would not suffer much on modern hardware. The -Xint benchmark is able to show that this is not true. Best regards, Martin -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Dienstag, 28. Februar 2017 16:14 To: Doerr, Martin ; Max Ockner ; hotspot-dev at openjdk.java.net Subject: Re: Discussion: 8172978: Remove Interpreter TOS optimization On 02/28/2017 04:00 PM, Doerr, Martin wrote: > I've ran jvm98 with -Xint on several PPC64 machines and on a recent s390x > machine. > > Surprisingly, disabling of the Tos optimization does not hurt on older > hardware (Power 5 and 6). Seems like some sub benchmarks don't suffer at all > or even benefit. But it really hurts on recent Power 8. I don't think it makes sense to run performance tests with -Xint alone. Of course removing interpreter optimizations would affect interpreter performance. The real question one should ask if turning off an interpreter optimization affects peak performance and time-to-performance when compilers are enabled. That's because in 2017 we should not expect that users who need performance would run with interpreter only. And removing complexity from interpreter without sacrificing the performance in tiered/compiled mode is certainly a plus in my book. Thanks, -Aleksey From shade at redhat.com Tue Feb 28 15:24:55 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 28 Feb 2017 16:24:55 +0100 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <3d99a9def06c4c2d9823086173b898ad@sap.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> <2c200ce3225445d49eb91c25b4094cc0@sap.com> <26eb5b4e-08ef-f5a7-957a-b8569f47ce7f@redhat.com> <3d99a9def06c4c2d9823086173b898ad@sap.com> Message-ID: On 02/28/2017 04:23 PM, Doerr, Martin wrote: > The assumption behind this proposal was that interpreter performance would > not suffer much on modern hardware. The -Xint benchmark is able to show that > this is not true. Ah, I see. I can't fathom why this is a success metric then. -Aleksey From martin.doerr at sap.com Tue Feb 28 15:41:29 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 28 Feb 2017 15:41:29 +0000 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> <2c200ce3225445d49eb91c25b4094cc0@sap.com> <26eb5b4e-08ef-f5a7-957a-b8569f47ce7f@redhat.com> <3d99a9def06c4c2d9823086173b898ad@sap.com> Message-ID: <615381e5a00c43178d2079785ba9c812@sap.com> It would be a success metric the other way round: If we didn't lose interpreter performance, we could have been certain not to lose startup performance. Now, we don't know how much the impact on startup performance is. But it may be affected. -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Dienstag, 28. Februar 2017 16:25 To: Doerr, Martin ; Max Ockner ; hotspot-dev at openjdk.java.net Subject: Re: Discussion: 8172978: Remove Interpreter TOS optimization On 02/28/2017 04:23 PM, Doerr, Martin wrote: > The assumption behind this proposal was that interpreter performance would > not suffer much on modern hardware. The -Xint benchmark is able to show that > this is not true. Ah, I see. I can't fathom why this is a success metric then. -Aleksey From martin.doerr at sap.com Tue Feb 28 15:59:00 2017 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 28 Feb 2017 15:59:00 +0000 Subject: Discussion: 8172978: Remove Interpreter TOS optimization In-Reply-To: <52c6a32c-6b36-ed5b-598f-0d3a1f9bf50b@redhat.com> References: <58A4D2C8.80603@oracle.com> <58A4D3CA.2060700@oracle.com> <8821d6f7-60a4-e494-20d9-8266ea2f0295@oracle.com> <9a092624-fb50-dea0-a011-6cc01101826f@oracle.com> <225ec2cf-063b-7b63-6207-ca9d1b3bc418@oracle.com> <58AF5253.7020005@oracle.com> <2c200ce3225445d49eb91c25b4094cc0@sap.com> <52c6a32c-6b36-ed5b-598f-0d3a1f9bf50b@redhat.com> Message-ID: Hi Andrew, I think it would be interesting to see how it impacts more platforms. Thanks and best regards, Martin -----Original Message----- From: Andrew Haley [mailto:aph at redhat.com] Sent: Dienstag, 28. Februar 2017 16:05 To: Doerr, Martin ; Max Ockner ; hotspot-dev at openjdk.java.net Subject: Re: Discussion: 8172978: Remove Interpreter TOS optimization On 28/02/17 15:00, Doerr, Martin wrote: > I first liked the idea to remove the optimization, but these numbers speak against doing it. Looks like it. I haven't investigated making such a change on AArch64, but I could if more data were needed. Andrew. From robbin.ehn at oracle.com Tue Feb 28 16:52:29 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 28 Feb 2017 17:52:29 +0100 Subject: [10]RFR: 8136650: Add support for custom jtreg native tests Message-ID: <62136515-9397-08d2-2baa-1957cae14eb2@oracle.com> Hi, please review. This adds support for custom extensions the jtreg native tests. The issues is closed due to internal tests but contains no more info than this subject: https://bugs.openjdk.java.net/browse/JDK-8136650 Open webrev: http://cr.openjdk.java.net/~rehn/8136650/ Tested with dummy native test. Thanks, Robbin From christian.tornqvist at oracle.com Tue Feb 28 18:48:25 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Tue, 28 Feb 2017 13:48:25 -0500 Subject: [10]RFR: 8136650: Add support for custom jtreg native tests In-Reply-To: <62136515-9397-08d2-2baa-1957cae14eb2@oracle.com> References: <62136515-9397-08d2-2baa-1957cae14eb2@oracle.com> Message-ID: <0a0001d291f3$40b09340$c211b9c0$@oracle.com> Hi Robbin, This looks good, thanks for doing this! Thanks, Christian -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Robbin Ehn Sent: Tuesday, February 28, 2017 11:52 AM To: hotspot-dev at openjdk.java.net Subject: [10]RFR: 8136650: Add support for custom jtreg native tests Hi, please review. This adds support for custom extensions the jtreg native tests. The issues is closed due to internal tests but contains no more info than this subject: https://bugs.openjdk.java.net/browse/JDK-8136650 Open webrev: http://cr.openjdk.java.net/~rehn/8136650/ Tested with dummy native test. Thanks, Robbin From christian.tornqvist at oracle.com Tue Feb 28 21:41:50 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Tue, 28 Feb 2017 16:41:50 -0500 Subject: RFR(S): 8176012 - Remove unused groups in hotspot/test/TEST.groups Message-ID: <0b1701d2920b$784f3c80$68edb580$@oracle.com> Hi everyone, Please review this small change for JDK 10 that removes unused Hotspot test groups from TEST.groups. Tested locally by running a small set of jtreg tests to verify consistency of TEST.groups. Bug: https://bugs.openjdk.java.net/browse/JDK-8176012 Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8176012/webrev.00/ Thanks, Christian From igor.ignatyev at oracle.com Tue Feb 28 22:14:30 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 28 Feb 2017 14:14:30 -0800 Subject: RFR(S): 8176012 - Remove unused groups in hotspot/test/TEST.groups In-Reply-To: <0b1701d2920b$784f3c80$68edb580$@oracle.com> References: <0b1701d2920b$784f3c80$68edb580$@oracle.com> Message-ID: <8D7A25B2-8BAF-414B-9F83-10869F3D471D@oracle.com> Hi Christian, Looks good to me. Thanks, ? Igor > On Feb 28, 2017, at 1:41 PM, Christian Tornqvist wrote: > > Hi everyone, > > > > Please review this small change for JDK 10 that removes unused Hotspot test > groups from TEST.groups. Tested locally by running a small set of jtreg > tests to verify consistency of TEST.groups. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176012 > > Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8176012/webrev.00/ > > > > Thanks, > > Christian > > > From george.triantafillou at oracle.com Tue Feb 28 22:29:08 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 28 Feb 2017 17:29:08 -0500 Subject: RFR(S): 8176012 - Remove unused groups in hotspot/test/TEST.groups In-Reply-To: <0b1701d2920b$784f3c80$68edb580$@oracle.com> References: <0b1701d2920b$784f3c80$68edb580$@oracle.com> Message-ID: <2e233f4f-4e98-f32f-a2b3-e877cef6a7b1@oracle.com> Hi Christian, Looks good. -George On 2/28/2017 4:41 PM, Christian Tornqvist wrote: > Hi everyone, > > > > Please review this small change for JDK 10 that removes unused Hotspot test > groups from TEST.groups. Tested locally by running a small set of jtreg > tests to verify consistency of TEST.groups. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176012 > > Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8176012/webrev.00/ > > > > Thanks, > > Christian > > >