From vladimir.kozlov at oracle.com Fri Mar 1 01:01:13 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 28 Feb 2019 17:01:13 -0800 Subject: RFR 8219951: Build failure on Mac and Windows after JDK-8219922 Message-ID: <161667b5-de09-4de8-53ed-b87a8bcc1f2f@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8219951 Add missing parentheses. Tested tier1 (includes builds), hs-tier2 -- Thanks, Vladimir diff -r 86ee52ca11e3 src/hotspot/share/opto/indexSet.hpp --- a/src/hotspot/share/opto/indexSet.hpp +++ b/src/hotspot/share/opto/indexSet.hpp @@ -440,7 +440,7 @@ uint current = _current; if (current != 0) { uint advance = count_trailing_zeros(current); - assert((current >> advance) & 0x1 == 1, "sanity"); + assert(((current >> advance) & 0x1) == 1, "sanity"); _current = (current >> advance) - 1; _value += advance; return _value; From david.holmes at oracle.com Fri Mar 1 01:05:10 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Mar 2019 11:05:10 +1000 Subject: RFR 8219951: Build failure on Mac and Windows after JDK-8219922 In-Reply-To: <161667b5-de09-4de8-53ed-b87a8bcc1f2f@oracle.com> References: <161667b5-de09-4de8-53ed-b87a8bcc1f2f@oracle.com> Message-ID: <1eedf5b0-442c-527b-1197-589a45ea6acc@oracle.com> I think your email came through in HTML and my reply is plain text and so has lost part of it. :( But the added parantheses looks good and trivial. Thanks, David On 1/03/2019 11:01 am, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8219951 > > Add missing parentheses. > > Tested tier1 (includes builds), hs-tier2 > From vladimir.kozlov at oracle.com Fri Mar 1 01:06:44 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 28 Feb 2019 17:06:44 -0800 Subject: RFR 8219951: Build failure on Mac and Windows after JDK-8219922 In-Reply-To: <1eedf5b0-442c-527b-1197-589a45ea6acc@oracle.com> References: <161667b5-de09-4de8-53ed-b87a8bcc1f2f@oracle.com> <1eedf5b0-442c-527b-1197-589a45ea6acc@oracle.com> Message-ID: <442b2e6a-6278-4081-aad8-ace87c016958@oracle.com> Thank you, David Vladimir On 2/28/19 5:05 PM, David Holmes wrote: > I think your email came through in HTML and my reply is plain text and so has lost part of it. :( > > But the added parantheses looks good and trivial. > > Thanks, > David > > On 1/03/2019 11:01 am, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8219951 >> >> Add missing parentheses. >> >> Tested tier1 (includes builds), hs-tier2 >> From goetz.lindenmaier at sap.com Fri Mar 1 06:49:37 2019 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 1 Mar 2019 06:49:37 +0000 Subject: RFR(XS): 8219961: [ppc64] Increase code size for interpreter generation. Message-ID: Hi, please review this tiny fix for OpenJDK 8. If an agent is specified, the VM does not start on ppc64. The space for the template interpreter does not suffice any more. http://cr.openjdk.java.net/~goetz/wr19/8219961-ppc64_interpreter_8/01/ Best regards, Goetz. From goetz.lindenmaier at sap.com Fri Mar 1 07:06:35 2019 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 1 Mar 2019 07:06:35 +0000 Subject: 8219918: ProblemList tests failing in SAP testing. In-Reply-To: <50a7ad12-9d70-c838-1baa-5d0ee6337c85@oracle.com> References: <7c80c989-48cd-3689-36ff-cdc0f2ac6003@oracle.com> <50a7ad12-9d70-c838-1baa-5d0ee6337c85@oracle.com> Message-ID: Hi, I changed this to only ProblemList hotspot tests. Please review. http://cr.openjdk.java.net/~goetz/wr19/8219918-problemList/02/ Best regards, Goetz. > -----Original Message----- > From: Daniel D. Daugherty > Sent: Thursday, February 28, 2019 4:48 PM > To: Alan Bateman ; Lindenmaier, Goetz > ; hotspot-dev at openjdk.java.net > Subject: Re: 8219918: ProblemList tests failing in SAP testing. > > On 2/28/19 8:36 AM, Alan Bateman wrote: > > On 28/02/2019 13:05, Lindenmaier, Goetz wrote: > >> Hi, > >> > >> in our testing we see a few tests failing at least since the jdk 11 > >> release. > >> I opened bugs for these issues and would like to add them to > >> the ProblemLists for now: > >> > >> http://cr.openjdk.java.net/~goetz/wr19/8219918-problemList/01/ > >> > >> I don't think any of these issues are critical. > >> > > > > This excludes the the CopyAndMove.java test which is critical to > > testing the Files.copy/move APIs. Can you start a discussion on > > nio-dev with info about the test environment rather than excluding > > this really important test. > > > > -Alan > > In the bug there is a mix of hotspot and jdk tests and this review > request was only sent to hotspot-dev at ... I think trying to do this > in one bug is not quite appropriate for review purposes. Perhaps > an umbrella bug with subtasks? Dunno... > > Dan From david.holmes at oracle.com Fri Mar 1 07:41:10 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Mar 2019 17:41:10 +1000 Subject: 8219918: ProblemList tests failing in SAP testing. In-Reply-To: References: <7c80c989-48cd-3689-36ff-cdc0f2ac6003@oracle.com> <50a7ad12-9d70-c838-1baa-5d0ee6337c85@oracle.com> Message-ID: <56bc2cba-7629-e3cd-df6d-c6812e4fdd93@oracle.com> Hi Goetz, On 1/03/2019 5:06 pm, Lindenmaier, Goetz wrote: > Hi, > > I changed this to only ProblemList hotspot tests. > Please review. > http://cr.openjdk.java.net/~goetz/wr19/8219918-problemList/02/ I understand the AIX specific issues but for the Windows ones ... + compiler/ciReplay/TestServerVM.java 8219651 windows-all + compiler/ciReplay/TestVMNoCompLevel.java 8219651 windows-all have you seen latest comments in the bug? It's suggested the problem is the executable is not being found. Thanks, David > Best regards, > Goetz. > > >> -----Original Message----- >> From: Daniel D. Daugherty >> Sent: Thursday, February 28, 2019 4:48 PM >> To: Alan Bateman ; Lindenmaier, Goetz >> ; hotspot-dev at openjdk.java.net >> Subject: Re: 8219918: ProblemList tests failing in SAP testing. >> >> On 2/28/19 8:36 AM, Alan Bateman wrote: >>> On 28/02/2019 13:05, Lindenmaier, Goetz wrote: >>>> Hi, >>>> >>>> in our testing we see a few tests failing at least since the jdk 11 >>>> release. >>>> I opened bugs for these issues and would like to add them to >>>> the ProblemLists for now: >>>> >>>> http://cr.openjdk.java.net/~goetz/wr19/8219918-problemList/01/ >>>> >>>> I don't think any of these issues are critical. >>>> >>> >>> This excludes the the CopyAndMove.java test which is critical to >>> testing the Files.copy/move APIs. Can you start a discussion on >>> nio-dev with info about the test environment rather than excluding >>> this really important test. >>> >>> -Alan >> >> In the bug there is a mix of hotspot and jdk tests and this review >> request was only sent to hotspot-dev at ... I think trying to do this >> in one bug is not quite appropriate for review purposes. Perhaps >> an umbrella bug with subtasks? Dunno... >> >> Dan From matthias.baesken at sap.com Fri Mar 1 08:08:42 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 1 Mar 2019 08:08:42 +0000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> Message-ID: Hi David , I think a clean way would be to have os::print_virtualization_info(outputStream* st) with platform and/or if needed platform/cpu - specific coding at the right places . I suggested something similar back then . This would be also a useful structure to bring in the AIX virtualization related info output . Regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 28. Februar 2019 22:49 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > hs_error file on linux/windows x86_64 > > Hi Matthias, > > Combining this with 8219746 I have to insist that we use a nice clean > OS-CPU abstraction for dealing with this and get rid of the platform > ifdefs in os::Linux::print_virtualization_info. What you've added in > VM_Version would seem to be the right kind of platform specific > functionality and it seems to me that you could get rid of > os::Linux::print_virtualization_info and just use > VM_Version::print_detected_virtualization in its place. > > Thanks, > David > > On 1/03/2019 2:04 am, Baesken, Matthias wrote: > > Hello, I provided a second version . > > The x86_64 assembler code was simplified (thanks to Martin for the hint) ; > additionally some comments were adjusted . > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.2/ > > > > > > Best regards , Matthias > > > > > > > > Hello , please review the following change. > > It provides basic virtualization related information in the hs_error file on > linux/windows x86_64. > > We use cpuid to get the information (detection of VMWare, Xen, KVM and > HyperV is included). > > > > Some info about how to detect the virtualization info can be found here : > > https://kb.vmware.com/s/article/1009458 > > > > > > > > (it is VMware related but can be applied for other virtualization > technologies as well) > > > > > > An overview of the ID strings can be found here as well : > > > > https://en.wikipedia.org/wiki/CPUID > > The following are known ID strings from virtual machines: ... > > > > > > > > bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8219241 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.1/ > > > > > > > > Thanks, Matthias > > From goetz.lindenmaier at sap.com Fri Mar 1 08:55:15 2019 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 1 Mar 2019 08:55:15 +0000 Subject: 8219918: ProblemList tests failing in SAP testing. In-Reply-To: <56bc2cba-7629-e3cd-df6d-c6812e4fdd93@oracle.com> References: <7c80c989-48cd-3689-36ff-cdc0f2ac6003@oracle.com> <50a7ad12-9d70-c838-1baa-5d0ee6337c85@oracle.com> <56bc2cba-7629-e3cd-df6d-c6812e4fdd93@oracle.com> Message-ID: > > I changed this to only ProblemList hotspot tests. > > Please review. > > http://cr.openjdk.java.net/~goetz/wr19/8219918-problemList/02/ > > I understand the AIX specific issues but for the Windows ones ... > > + compiler/ciReplay/TestServerVM.java 8219651 windows-all > + compiler/ciReplay/TestVMNoCompLevel.java 8219651 windows-all > > have you seen latest comments in the bug? It's suggested the problem is > the executable is not being found. Well, that's the correct path to the executable. It is there. Best regards, Goetz. > > Thanks, > David > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: Daniel D. Daugherty > >> Sent: Thursday, February 28, 2019 4:48 PM > >> To: Alan Bateman ; Lindenmaier, Goetz > >> ; hotspot-dev at openjdk.java.net > >> Subject: Re: 8219918: ProblemList tests failing in SAP testing. > >> > >> On 2/28/19 8:36 AM, Alan Bateman wrote: > >>> On 28/02/2019 13:05, Lindenmaier, Goetz wrote: > >>>> Hi, > >>>> > >>>> in our testing we see a few tests failing at least since the jdk 11 > >>>> release. > >>>> I opened bugs for these issues and would like to add them to > >>>> the ProblemLists for now: > >>>> > >>>> http://cr.openjdk.java.net/~goetz/wr19/8219918-problemList/01/ > >>>> > >>>> I don't think any of these issues are critical. > >>>> > >>> > >>> This excludes the the CopyAndMove.java test which is critical to > >>> testing the Files.copy/move APIs. Can you start a discussion on > >>> nio-dev with info about the test environment rather than excluding > >>> this really important test. > >>> > >>> -Alan > >> > >> In the bug there is a mix of hotspot and jdk tests and this review > >> request was only sent to hotspot-dev at ... I think trying to do this > >> in one bug is not quite appropriate for review purposes. Perhaps > >> an umbrella bug with subtasks? Dunno... > >> > >> Dan From aph at redhat.com Fri Mar 1 09:07:14 2019 From: aph at redhat.com (Andrew Haley) Date: Fri, 1 Mar 2019 09:07:14 +0000 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> Message-ID: <89eac42d-b6b5-6272-b6a0-ac45a85f89f9@redhat.com> On 2/28/19 8:22 PM, Harold Seigel wrote: > Please review this fix for JDK-8191278.? The fix uses the > GuardUnsafeAccess class in unsafe.cpp so that the signal handler will > know to convert SIGBUS signals to exceptions. Are there any performance implications to this? It looks to me like there won't be, but I wonder if tests were done to confirm this. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From christoph.langer at sap.com Fri Mar 1 09:18:01 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 1 Mar 2019 09:18:01 +0000 Subject: RFR(S): 8219650: [Testbug] Fix potential crashes in new test hotspot gtest "test_print_hex_dump" In-Reply-To: <33b1d1bb-3837-40e3-08f7-313be235c213@redhat.com> References: <19194f67-ed19-0031-e1a9-e79b56a777f0@redhat.com> <04555104-3ad3-0135-e45c-2ae4e0a621bd@redhat.com> <33b1d1bb-3837-40e3-08f7-313be235c213@redhat.com> Message-ID: Hi Thomas, > > Unless it really irks you, lets keep it that way. We all spent way too much > time on this thing, and > > while it is a really fascinating topic... :) > > Yeah, no reason to sink more time into it. Let's move on! It's not done yet ?? Looking at our nightly results where we test the patch in 11u context, we can see that the crash on Solaris is gone but the test error in Win32 is not fixed yet and on AIX the test fails now. All the platforms we love most ?? Best regards Christoph From christoph.langer at sap.com Fri Mar 1 09:19:34 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 1 Mar 2019 09:19:34 +0000 Subject: RFR (S) 8219712: code_size2 is too small on new Skylake CPUs In-Reply-To: <408b4714-8204-0540-0aff-dfaeb6a73464@oracle.com> References: <960a11003daa403fbc8bb841206b8853@sap.com> <860c3fcdf7da4af58add6bf1af290ff4@sap.com> <6f1b7776-1f52-00a6-4d35-859bb90b907c@oracle.com> <3eb93f5bd5ad44c4860e46434c2142e1@sap.com> <408b4714-8204-0540-0aff-dfaeb6a73464@oracle.com> Message-ID: <4c07333bc1844be0b745c11a75075614@sap.com> Ok, I've pushed to 12u as well. Thank you, Vladimir. > -----Original Message----- > From: hotspot-dev On Behalf Of > Vladimir Kozlov > Sent: Donnerstag, 28. Februar 2019 20:33 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR (S) 8219712: code_size2 is too small on new Skylake CPUs > > Since it is seen only in debug build it is not urgent. I agree to fix it in 12 update > - I approved request. > > Thanks, > Vladimir > > On 2/28/19 7:14 AM, Langer, Christoph wrote: > > Hi, > > > > I've pushed Ralf's fix today after running it through the submit repo. > > > > As it seems to relate to JDK-8214074 (Optimize Ghash using AVX > instructions) and this one is in jdk12, I guess it is quite urgent to bring this fix > to jdk12 as well. We see the assertion in debug builds. And in the opt builds, I > guess the memory would potentially be overwritten. How would you assess > the urgency? > > > > Best regards > > Christoph > > > >> -----Original Message----- > >> From: hotspot-dev On Behalf > Of > >> David Holmes > >> Sent: Dienstag, 26. Februar 2019 22:33 > >> To: Schmelter, Ralf ; HotSpot Open Source > >> Developers > >> Subject: Re: RFR (S) 8219712: code_size2 is too small on new Skylake CPUs > >> > >> Hi Ralf, > >> > >> webrev.1 looks good. > >> > >> Thanks, > >> David > >> > >> On 27/02/2019 1:50 am, Schmelter, Ralf wrote: > >>> Hi David, > >>> > >>>> I agree with Matthias: can this be a DEBUG_ONLY change? > >>> > >>> code_size2 is the size of the BufferBlob which is allocated to contain the > >> "stub routines 2" created by StubGenerator. And in the debug build we > >> check via assertions, that we don't overwrite the buffer. In the optimized > >> version, we would later just overwrite part of the stub routines. So > making it > >> DEBUG_ONLY defeats the purpose of the change. > >>> > >>>> Further is the problem with 32-bit and 64-bit or only 64-bit? Your > >>> change affects 32-bit. > >>> > >>> Since I only build the 64bit VM, I've changed the webrev: > >> http://cr.openjdk.java.net/~rschmelter/webrevs/8219712/webrev.1/ > >>> It should now only increase the 64 bit version. > >>> > >>> In the end, it would be nice if the initial BufferBlob size could be more > >> generous and then later adjusted to the really needed space, since it is > really > >> not easy to get a tight upper bound reliably, with it being dependent on > the > >> OS, GC, CPU features and other flags. > >>> > >>> Best regards, > >>> Ralf > >>> From aph at redhat.com Fri Mar 1 14:46:07 2019 From: aph at redhat.com (Andrew Haley) Date: Fri, 1 Mar 2019 14:46:07 +0000 Subject: RFR(XS): 8219961: [ppc64] Increase code size for interpreter generation. In-Reply-To: References: Message-ID: On 3/1/19 6:49 AM, Lindenmaier, Goetz wrote: > If an agent is specified, the VM does not start on ppc64. > The space for the template interpreter does not suffice any more. > http://cr.openjdk.java.net/~goetz/wr19/8219961-ppc64_interpreter_8/01/ OK, thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From sgehwolf at redhat.com Fri Mar 1 16:10:56 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 01 Mar 2019 17:10:56 +0100 Subject: [8u] RFR: 8195153: [test] runtime/6981737/Test6981737.java shouldn't check 'java.vendor' and 'java.vm.vendor' properties Message-ID: <017a668fe502ff2dd816d9a595e35d1d6f384b8e.camel@redhat.com> Hi, Could somebody please review this 8u backport for a test which checks the java.vendor and java.vm.vendor property? The JDK 8 patch is the same as JDK 10's version modulo contextual changes and reshuffeling. the JDK 10 patch does not apply cleanly because there is no Runtime.version() in JDK 8 (that is part of the patch context). Bug: https://bugs.openjdk.java.net/browse/JDK-8195153 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8195153/jdk8/01/webrev/ Testing: Test fails with AdoptOpenJDK's JDK 8 build, and passes after the fix. Thanks, Severin From mark.reinhold at oracle.com Fri Mar 1 17:08:37 2019 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 1 Mar 2019 09:08:37 -0800 (PST) Subject: New candidate JEP: 350: Dynamic CDS Archives Message-ID: <20190301170837.124C426ACC3@eggemoggin.niobe.net> https://openjdk.java.net/jeps/350 - Mark From gnu.andrew at redhat.com Fri Mar 1 19:46:01 2019 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Fri, 1 Mar 2019 19:46:01 +0000 Subject: [8u] RFR: 8195153: [test] runtime/6981737/Test6981737.java shouldn't check 'java.vendor' and 'java.vm.vendor' properties In-Reply-To: <017a668fe502ff2dd816d9a595e35d1d6f384b8e.camel@redhat.com> References: <017a668fe502ff2dd816d9a595e35d1d6f384b8e.camel@redhat.com> Message-ID: On Fri, 1 Mar 2019 at 16:12, Severin Gehwolf wrote: > > Hi, > > Could somebody please review this 8u backport for a test which checks > the java.vendor and java.vm.vendor property? The JDK 8 patch is the > same as JDK 10's version modulo contextual changes and reshuffeling. > > the JDK 10 patch does not apply cleanly because there is no > Runtime.version() in JDK 8 (that is part of the patch context). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195153 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8195153/jdk8/01/webrev/ > > Testing: Test fails with AdoptOpenJDK's JDK 8 build, and passes after > the fix. > > Thanks, > Severin > Looks good to me. I think we also need https://bugs.openjdk.java.net/browse/JDK-8189761 to make this really worthwhile. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From harold.seigel at oracle.com Fri Mar 1 20:58:24 2019 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 1 Mar 2019 15:58:24 -0500 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <89eac42d-b6b5-6272-b6a0-ac45a85f89f9@redhat.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <89eac42d-b6b5-6272-b6a0-ac45a85f89f9@redhat.com> Message-ID: No performance tests were run but we will run some and get back to you. Thanks, Harold On 3/1/2019 4:07 AM, Andrew Haley wrote: > On 2/28/19 8:22 PM, Harold Seigel wrote: >> Please review this fix for JDK-8191278.? The fix uses the >> GuardUnsafeAccess class in unsafe.cpp so that the signal handler will >> know to convert SIGBUS signals to exceptions. > Are there any performance implications to this? It looks to me like there > won't be, but I wonder if tests were done to confirm this. > From manc at google.com Fri Mar 1 21:42:56 2019 From: manc at google.com (Man Cao) Date: Fri, 1 Mar 2019 13:42:56 -0800 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> <7e0c775d-86c1-b80c-b1a6-373ca21206ba@oracle.com> Message-ID: Hi, I rebased the patch to latest tip: https://cr.openjdk.java.net/~manc/8212206/webrev.03/ Could someone give a second "looks good"? -Man On Sat, Jan 26, 2019 at 4:35 PM Man Cao wrote: > Friendly ping. Could anyone give a second "looks good"? > > As for the develop flag > AdaptiveSizePolicyGCTimeLimitThreshold/GCOverheadLimitThreshold, I added a > note about it in https://bugs.openjdk.java.net/browse/JDK-8212084. > > -Man > > > On Tue, Jan 15, 2019 at 6:41 PM Man Cao wrote: > >> Hi, >> >> I rebased the patch to tip and updated year in some headers to 2019, >> without making any real change: >> http://cr.openjdk.java.net/~manc/8212206/webrev.02/ >> >> >> I don't foresee that this will be implemented, or even makes sense, for >>> ZGC. As I see it, this is only a thing STW collectors. For that reason, >>> I don't think it belongs in CollectedHeap. Keeping it as a separate >>> utility class for collectors that want to use it sounds better. >>> >> Sounds good to keep this patch in the current state, without further >> changing the CollectedHeap class. >> >> I haven't looked very closely at the patch, but couldn't help to notice >>> that the option is called "GCOverheapLimitThreshold" (and >>> "AdaptiveSizePolicyGCTimeLimitThreshold" before that), which is a >>> tautology and a not very good description of what it is. >>> How about we take the opportunity to clean this up and completely ditch >>> the "gc_overhead_limit_count" thing and get rid of this option? It's a >>> "develop" option, so it's not available to normal users anyway. Has >>> anyone of you ever used this option and actually find it valuable? >> >> I didn't find any users inside Google that require changing this option. >> That said, some users did complain that UseGCOverheadLimit for ParallelGC >> or CMS is too difficult to get >> triggered, because of the requirement for 5 consecutive full GCs, which >> is set by this option. >> I think if it were a normal "product" option, there will definitely be >> users setting it. >> I never understand why it is a "develop" option. I think we could either >> remove it, >> or make it an "experimental" option. >> I'm leaning towards not removing it for now, as I'm not sure if 5 is >> still a reasonable >> default value for UseGCOverheadLimit for G1. >> How about we decide whether to keep or remove this option after >> JDK-8212084 (UseGCOverheadLimit for G1) is fixed? >> >> Also for the hsperfdata counter change, I created >> https://bugs.openjdk.java.net/browse/JDK-8217221. I will draft a CSR for >> it later. >> >> -Man >> > From coleen.phillimore at oracle.com Fri Mar 1 23:22:27 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 1 Mar 2019 18:22:27 -0500 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> Message-ID: <7144fe67-ee8c-459a-e595-b70f397489db@oracle.com> http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/src/hotspot/share/prims/unsafe.cpp.frames.html 448 GuardUnsafeAccess guard(thread); I can't find 'thread' in this scope.? Does line 433 need to be in the scope above it? 150 /** 151 * Helper class to wrap memory accesses in JavaThread::doing_unsafe_access() 152 */ Can you use // comments here? http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/test/hotspot/jtreg/runtime/Unsafe/InternalErrorTest.java.html 33 * @build sun.hotspot.WhiteBox Can you fix the indentation here? I don't see the compiler changes. Coleen On 2/28/19 3:22 PM, Harold Seigel wrote: > Hi, > > Please review this fix for JDK-8191278.? The fix uses the > GuardUnsafeAccess class in unsafe.cpp so that the signal handler will > know to convert SIGBUS signals to exceptions. > > The compiler related changes and much of the test were written by > Jamsheesh (jcm). > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/index.html > > JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8191278 > > The fix was tested on Linux x64, Mac OS, Solaris Sparc, and Windows.? > Testing was not done on the other platforms changed by this fix. > > The fix was regression tested by running Mach5 tiers 1 and 2 tests and > builds on Linux x64, Solaris, Windows, and Mac OS X, by running Mach5 > tiers 3 -5 on Linux x64, and by running JCK-13 Lang and VM tests on > Linux x64. > > Thanks, Harold > From jamsheed.c.m at oracle.com Sat Mar 2 06:42:19 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Sat, 2 Mar 2019 12:12:19 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <7144fe67-ee8c-459a-e595-b70f397489db@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <7144fe67-ee8c-459a-e595-b70f397489db@oracle.com> Message-ID: <4b052070-f7ec-a224-2daf-f5871927f442@oracle.com> Hi Coleen, On 02/03/19 4:52 AM, coleen.phillimore at oracle.com wrote: > I don't see the compiler changes. compiler changes are specific to c2 intrinsic usage. and is done in signal handler(ie StubRoutines::contains(pc)) Best regards, Jamsheed From jamsheed.c.m at oracle.com Sun Mar 3 01:52:27 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Sun, 3 Mar 2019 07:22:27 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <89eac42d-b6b5-6272-b6a0-ac45a85f89f9@redhat.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <89eac42d-b6b5-6272-b6a0-ac45a85f89f9@redhat.com> Message-ID: <41c85b1e-e95e-7e1e-fddc-c4d3aa754e15@oracle.com> Hi Andrew, On 01/03/19 2:37 PM, Andrew Haley wrote: > Are there any performance implications to this? It looks to me like there > won't be, but I wonder if tests were done to confirm this. normal mode(with c2 + intrinsic), there shouldn't be any impact on performance. while failover mechanism could get impacted as it is bulk access.? previously it used to crash immediately, now it can hang for sometime before exit(or get killed). Best regards, Jamsheed From david.holmes at oracle.com Mon Mar 4 07:17:00 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Mar 2019 17:17:00 +1000 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class Message-ID: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ bug: https://bugs.openjdk.java.net/browse/JDK-8219974 JDK-8219492 was previously reviewed: http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html and pushed: http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 but the new assertions failed on a single test that had an invokespecial call to an interface method for an interface that was not (and should not be) initialized. So it was backed out. The safest fix is to ensure the new assertions only apply when considering invoke_static. So that's the only difference between this version and the original. Testing: - failing test from tier 4 in regular and Xcomp mode - tiers 1-3 Thanks, David ----- From martin.doerr at sap.com Mon Mar 4 09:06:03 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 4 Mar 2019 09:06:03 +0000 Subject: RFR(XS): 8219961: [ppc64] Increase code size for interpreter generation. In-Reply-To: References: Message-ID: Hi G?tz, reviewed. I think it's critical enough for 8u (instead of 8u-dev). Risk seems to be very low. Best regards, Martin -----Original Message----- From: jdk8u-dev On Behalf Of Andrew Haley Sent: Freitag, 1. M?rz 2019 15:46 To: Lindenmaier, Goetz ; jdk8u-dev at openjdk.java.net; hotspot-dev at openjdk.java.net Subject: Re: RFR(XS): 8219961: [ppc64] Increase code size for interpreter generation. On 3/1/19 6:49 AM, Lindenmaier, Goetz wrote: > If an agent is specified, the VM does not start on ppc64. > The space for the template interpreter does not suffice any more. > http://cr.openjdk.java.net/~goetz/wr19/8219961-ppc64_interpreter_8/01/ OK, thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From goetz.lindenmaier at sap.com Mon Mar 4 09:22:32 2019 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 4 Mar 2019 09:22:32 +0000 Subject: RFR(XS): 8219961: [ppc64] Increase code size for interpreter generation. In-Reply-To: References: Message-ID: Thanks Martin! Best regards, Goetz. > -----Original Message----- > From: Doerr, Martin > Sent: Monday, March 4, 2019 10:06 AM > To: Andrew Haley ; Lindenmaier, Goetz > ; jdk8u-dev at openjdk.java.net; hotspot- > dev at openjdk.java.net > Subject: RE: RFR(XS): 8219961: [ppc64] Increase code size for interpreter > generation. > > Hi G?tz, > > reviewed. I think it's critical enough for 8u (instead of 8u-dev). Risk seems to > be very low. > > Best regards, > Martin > > > -----Original Message----- > From: jdk8u-dev On Behalf Of > Andrew Haley > Sent: Freitag, 1. M?rz 2019 15:46 > To: Lindenmaier, Goetz ; jdk8u- > dev at openjdk.java.net; hotspot-dev at openjdk.java.net > Subject: Re: RFR(XS): 8219961: [ppc64] Increase code size for interpreter > generation. > > On 3/1/19 6:49 AM, Lindenmaier, Goetz wrote: > > If an agent is specified, the VM does not start on ppc64. > > The space for the template interpreter does not suffice any more. > > http://cr.openjdk.java.net/~goetz/wr19/8219961-ppc64_interpreter_8/01/ > > OK, thanks. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From per.liden at oracle.com Mon Mar 4 09:48:59 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 4 Mar 2019 10:48:59 +0100 Subject: RFR: 8219816: Add IsArray/RemoveArray type traits utilities In-Reply-To: <387BFF37-0572-4341-BA12-BC0857D55ABA@oracle.com> References: <8FDD08D8-6187-41DB-8773-8921A9DFF0E5@oracle.com> <387BFF37-0572-4341-BA12-BC0857D55ABA@oracle.com> Message-ID: <10f422e7-9fef-4876-23fd-608f1487dbb7@oracle.com> On 2/27/19 9:21 PM, Kim Barrett wrote: >> On Feb 27, 2019, at 10:08 AM, Kim Barrett wrote: >> >>> On Feb 27, 2019, at 3:53 AM, Per Liden wrote: >>> >>> Add IsArray/RemoveArray to our collection of type traits metaprogramming utilities. >>> >>> This patch only adds the IsArray/RemoveArray utilities and their respective tests. A later patch I have in my queue that will actually make use of these. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8219816 >>> Webrev: http://cr.openjdk.java.net/~pliden/8219816/webrev.0 >>> >>> /Per >> >> For completeness, there should probably be IsArray test cases for >> const/volatile with size. >> >> Otherwise, looks good. I don't need another webrev for those >> additional test cases. > > HotSpot's metaprogramming utilities have been intentionally modelled > on those from , both in naming and (with some limitations > in a few cases) behavior. The C++11 type trait for removing array > dimensions from an array type is "remove_extent" (and "remove_all_extents" > to remove all of them from a multi-dimensional array type). > > So what this change calls RemoveArray ought to be called RemoveExtent. > Sorry that I forgot to check on this earlier. Ah, good catch. Renamed accordingly. I'm not implementing remove_all_extents at this time, let's add that when there's a use case for it. Updated webrev: http://cr.openjdk.java.net/~pliden/8219816/webrev.1 /Per From claes.redestad at oracle.com Mon Mar 4 10:14:45 2019 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 4 Mar 2019 11:14:45 +0100 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class In-Reply-To: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> References: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> Message-ID: <9f298b00-3066-0cf8-f4ad-5139ffccd65b@oracle.com> Hi David, moving the new assertion into the invokestatic branch looks good. Thanks! /Claes On 2019-03-04 08:17, David Holmes wrote: > webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ > bug: https://bugs.openjdk.java.net/browse/JDK-8219974 > > JDK-8219492 was previously reviewed: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html > > > and pushed: > > http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 > > but the new assertions failed on a single test that had an invokespecial > call to an interface method for an interface that was not (and should > not be) initialized. So it was backed out. > > The safest fix is to ensure the new assertions only apply when > considering invoke_static. So that's the only difference between this > version and the original. > > Testing: > ? - failing test from tier 4 in regular and Xcomp mode > ? - tiers 1-3 > > Thanks, > David > ----- From david.holmes at oracle.com Mon Mar 4 11:24:16 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Mar 2019 21:24:16 +1000 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class In-Reply-To: <9f298b00-3066-0cf8-f4ad-5139ffccd65b@oracle.com> References: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> <9f298b00-3066-0cf8-f4ad-5139ffccd65b@oracle.com> Message-ID: <700e86d6-d8c6-e60d-9548-2057674c7be6@oracle.com> Thanks Claes! David On 4/03/2019 8:14 pm, Claes Redestad wrote: > Hi David, > > moving the new assertion into the invokestatic branch looks good. > > Thanks! > > /Claes > > On 2019-03-04 08:17, David Holmes wrote: >> webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8219974 >> >> JDK-8219492 was previously reviewed: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html >> >> >> and pushed: >> >> http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 >> >> but the new assertions failed on a single test that had an >> invokespecial call to an interface method for an interface that was >> not (and should not be) initialized. So it was backed out. >> >> The safest fix is to ensure the new assertions only apply when >> considering invoke_static. So that's the only difference between this >> version and the original. >> >> Testing: >> ?? - failing test from tier 4 in regular and Xcomp mode >> ?? - tiers 1-3 >> >> Thanks, >> David >> ----- From markus.gronlund at oracle.com Mon Mar 4 11:31:23 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Mon, 4 Mar 2019 03:31:23 -0800 (PST) Subject: Results: New HotSpot Group Member: Erik Gahlin Message-ID: <57fc7175-864a-41f8-baef-84daa2648707@default> Voting for Erik Gahlin (egahlin) [1] is now closed. Yes: 15 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Thanks for helping with the approval of Erik Markus [1] https://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/036854.html From kim.barrett at oracle.com Mon Mar 4 15:33:59 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 4 Mar 2019 10:33:59 -0500 Subject: RFR: 8219816: Add IsArray/RemoveArray type traits utilities In-Reply-To: <10f422e7-9fef-4876-23fd-608f1487dbb7@oracle.com> References: <8FDD08D8-6187-41DB-8773-8921A9DFF0E5@oracle.com> <387BFF37-0572-4341-BA12-BC0857D55ABA@oracle.com> <10f422e7-9fef-4876-23fd-608f1487dbb7@oracle.com> Message-ID: <9E619488-6692-4DFF-B968-D98FF87105B3@oracle.com> > On Mar 4, 2019, at 4:48 AM, Per Liden wrote: > Ah, good catch. Renamed accordingly. I'm not implementing remove_all_extents at this time, let's add that when there's a use case for it. > > Updated webrev: http://cr.openjdk.java.net/~pliden/8219816/webrev.1 > > /Per Not implementing remove_all_extents at this time is fine. Looks good. From jianglizhou at google.com Sun Mar 3 19:48:40 2019 From: jianglizhou at google.com (Jiangli Zhou) Date: Sun, 3 Mar 2019 11:48:40 -0800 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> <7e0c775d-86c1-b80c-b1a6-373ca21206ba@oracle.com> Message-ID: Hi Man, As both Thomas and Per seemed to be okay with the refactoring, you probably had sufficient approval already. The following comment in adaptiveSizePolicy.cpp seems to be outdated. The comment predates the PermGen removal. It's probably a good idea to also cleanup the comment as part of your refactoring change. 281 // Note that the gc time limit test only works for the collections 282 // of the young gen + tenured gen and not for collections of the 283 // permanent gen. That is because the calculation of the space 284 // freed by the collection is the free space in the young gen + 285 // tenured gen. Thanks and regards, Jiangli On Fri, Mar 1, 2019 at 1:43 PM Man Cao wrote: > Hi, > > I rebased the patch to latest tip: > https://cr.openjdk.java.net/~manc/8212206/webrev.03/ > > Could someone give a second "looks good"? > > -Man > > > On Sat, Jan 26, 2019 at 4:35 PM Man Cao wrote: > >> Friendly ping. Could anyone give a second "looks good"? >> >> As for the develop flag >> AdaptiveSizePolicyGCTimeLimitThreshold/GCOverheadLimitThreshold, I added a >> note about it in https://bugs.openjdk.java.net/browse/JDK-8212084. >> >> -Man >> >> >> On Tue, Jan 15, 2019 at 6:41 PM Man Cao wrote: >> >>> Hi, >>> >>> I rebased the patch to tip and updated year in some headers to 2019, >>> without making any real change: >>> http://cr.openjdk.java.net/~manc/8212206/webrev.02/ >>> >>> >>> I don't foresee that this will be implemented, or even makes sense, for >>>> ZGC. As I see it, this is only a thing STW collectors. For that reason, >>>> I don't think it belongs in CollectedHeap. Keeping it as a separate >>>> utility class for collectors that want to use it sounds better. >>>> >>> Sounds good to keep this patch in the current state, without further >>> changing the CollectedHeap class. >>> >>> I haven't looked very closely at the patch, but couldn't help to notice >>>> that the option is called "GCOverheapLimitThreshold" (and >>>> "AdaptiveSizePolicyGCTimeLimitThreshold" before that), which is a >>>> tautology and a not very good description of what it is. >>>> How about we take the opportunity to clean this up and completely ditch >>>> the "gc_overhead_limit_count" thing and get rid of this option? It's a >>>> "develop" option, so it's not available to normal users anyway. Has >>>> anyone of you ever used this option and actually find it valuable? >>> >>> I didn't find any users inside Google that require changing this option. >>> That said, some users did complain that UseGCOverheadLimit for >>> ParallelGC or CMS is too difficult to get >>> triggered, because of the requirement for 5 consecutive full GCs, which >>> is set by this option. >>> I think if it were a normal "product" option, there will definitely be >>> users setting it. >>> I never understand why it is a "develop" option. I think we could either >>> remove it, >>> or make it an "experimental" option. >>> I'm leaning towards not removing it for now, as I'm not sure if 5 is >>> still a reasonable >>> default value for UseGCOverheadLimit for G1. >>> How about we decide whether to keep or remove this option after >>> JDK-8212084 (UseGCOverheadLimit for G1) is fixed? >>> >>> Also for the hsperfdata counter change, I created >>> https://bugs.openjdk.java.net/browse/JDK-8217221. I will draft a CSR >>> for it later. >>> >>> -Man >>> >> From david.holmes at oracle.com Mon Mar 4 21:32:29 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Mar 2019 07:32:29 +1000 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class In-Reply-To: <700e86d6-d8c6-e60d-9548-2057674c7be6@oracle.com> References: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> <9f298b00-3066-0cf8-f4ad-5139ffccd65b@oracle.com> <700e86d6-d8c6-e60d-9548-2057674c7be6@oracle.com> Message-ID: Can I get a second (re)review please. Thanks, David On 4/03/2019 9:24 pm, David Holmes wrote: > Thanks Claes! > > David > > On 4/03/2019 8:14 pm, Claes Redestad wrote: >> Hi David, >> >> moving the new assertion into the invokestatic branch looks good. >> >> Thanks! >> >> /Claes >> >> On 2019-03-04 08:17, David Holmes wrote: >>> webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8219974 >>> >>> JDK-8219492 was previously reviewed: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html >>> >>> >>> and pushed: >>> >>> http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 >>> >>> but the new assertions failed on a single test that had an >>> invokespecial call to an interface method for an interface that was >>> not (and should not be) initialized. So it was backed out. >>> >>> The safest fix is to ensure the new assertions only apply when >>> considering invoke_static. So that's the only difference between this >>> version and the original. >>> >>> Testing: >>> ?? - failing test from tier 4 in regular and Xcomp mode >>> ?? - tiers 1-3 >>> >>> Thanks, >>> David >>> ----- From vladimir.x.ivanov at oracle.com Mon Mar 4 21:53:50 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 4 Mar 2019 13:53:50 -0800 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class In-Reply-To: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> References: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> Message-ID: <24d69f60-3fe3-92af-fbcf-62b33b21c240@oracle.com> Looks good. Best regards, Vladimir Ivanov On 03/03/2019 23:17, David Holmes wrote: > webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ > bug: https://bugs.openjdk.java.net/browse/JDK-8219974 > > JDK-8219492 was previously reviewed: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html > > > and pushed: > > http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 > > but the new assertions failed on a single test that had an invokespecial > call to an interface method for an interface that was not (and should > not be) initialized. So it was backed out. > > The safest fix is to ensure the new assertions only apply when > considering invoke_static. So that's the only difference between this > version and the original. > > Testing: > ? - failing test from tier 4 in regular and Xcomp mode > ? - tiers 1-3 > > Thanks, > David > ----- From coleen.phillimore at oracle.com Mon Mar 4 21:54:23 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 4 Mar 2019 16:54:23 -0500 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class In-Reply-To: References: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> <9f298b00-3066-0cf8-f4ad-5139ffccd65b@oracle.com> <700e86d6-d8c6-e60d-9548-2057674c7be6@oracle.com> Message-ID: This looks good. Coleen On 3/4/19 4:32 PM, David Holmes wrote: > Can I get a second (re)review please. > > Thanks, > David > > On 4/03/2019 9:24 pm, David Holmes wrote: >> Thanks Claes! >> >> David >> >> On 4/03/2019 8:14 pm, Claes Redestad wrote: >>> Hi David, >>> >>> moving the new assertion into the invokestatic branch looks good. >>> >>> Thanks! >>> >>> /Claes >>> >>> On 2019-03-04 08:17, David Holmes wrote: >>>> webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8219974 >>>> >>>> JDK-8219492 was previously reviewed: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html >>>> >>>> >>>> and pushed: >>>> >>>> http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 >>>> >>>> but the new assertions failed on a single test that had an >>>> invokespecial call to an interface method for an interface that was >>>> not (and should not be) initialized. So it was backed out. >>>> >>>> The safest fix is to ensure the new assertions only apply when >>>> considering invoke_static. So that's the only difference between >>>> this version and the original. >>>> >>>> Testing: >>>> ?? - failing test from tier 4 in regular and Xcomp mode >>>> ?? - tiers 1-3 >>>> >>>> Thanks, >>>> David >>>> ----- From mikhailo.seledtsov at oracle.com Mon Mar 4 21:57:54 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 4 Mar 2019 13:57:54 -0800 Subject: RFR(S) : 8219139 : move hotspot tests from test/jdk/vm In-Reply-To: References: <036DE223-2FAB-43A4-BAC7-63A9BBFCA65D@oracle.com> <132812c3-56be-edfd-f812-78b30c75973b@oracle.com> Message-ID: <511d81a3-847c-49d0-76ff-fdc836211f37@oracle.com> Hi Igor, ? Sorry it took a while to get back to you on this one. See my comment below On 2/22/19 10:35 AM, mikhailo.seledtsov at oracle.com wrote: > Overall the change looks good; thank you Igor for doing this. I have > couple of comments: > > ? - I am in favor of the approach where we move tests first under > corresponding sub-component directories in > ??? test/hotspot sub-tree, and then figure out whether to keep them. > Even though in general I am in favor > ??? of removing tests that are obsolete or of questionable value, this > requires time, consideration and discussions. > ??? Hence, I recommend filing an RFE for that, which can be addressed > after the migration. > > ? - Runtime normally avoids tests directly in test/hotspot/jtreg/runtime > ??? The tests should go into underlying sub-directories which best > match functional area or topic of that test. > ??? In most cases you did it in your change, but there are several > tests that your change places directly under > ???? test/hotspot/jtreg/runtime/: > > ???? ExplicitArithmeticCheck.java > ???? MonitorCacheMaybeExpand_DeadLock.java > ???? ReflectStackOverflow.java > ???? ShiftTest.java - David commented this can go under compiler (a > jit test) > ???? WideStrictInline.java I have looked at the tests in more detail, and here are my recommendation of placements: ??? ExplicitArithmeticCheck ??????? This test checks that ArithmeticException is thrown when appropriate ??????? I would recommend placing it under runtime/ErrorHandling ??? MonitorCacheMaybeExpand_DeadLock ??????? Existing folder: runtime/Thread (it does have a locking test) ??????? Or, alternatively, create a new folder: 'locking' or 'monitors' ??? ReflectStackOverflow ??????? Uses recursive reflection attempting to provoke stack overflow ??????? Can go under: runtime/reflect ??? WideStrictInline: ??????? checks for correct FP inlining by the interpreter ??????? I could not find existing sections; perhaps create 'interpreter' ??????? folder under 'runtime' Thank you, Misha > > ???? Since we plan (as discussed) to follow up this work with an RFE > to review and consider removal of old and > ???? not-that-useful tests, you could place them under 'misc' for now. > Alternatively, find the best match > ???? or create new sub-directories under runtime/ if necessary. > > > Thank you, > Misha > > > On 2/21/19 11:53 AM, Igor Ignatyev wrote: >> >>> On Feb 21, 2019, at 12:11 AM, David Holmes >>> wrote: >>> >>> Hi Igor, >>> >>> On 21/02/2019 3:19 pm, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>> 40 lines changed: 17 ins; 2 del; 21 mod; >>>> Hi all, >>>> could you please review this small patch which moves tests from >>>> test/jdk/vm? >>> I find some of these tests - the runtime ones at least - of >>> extremely dubious value. They either cover basic functionality that >>> is already well covered, or are regression tests for bugs in code >>> that hasn't existed for many many years! >> as I wrote in another related email: "one of the reason I'm proposing >> this move is exactly questionable value of these tests, I want to >> believe that having these tests in hotspot/ test directories will >> bring more attention to them from corresponding component teams and >> hence they will be removed/reworked/re-whatever faster and better. >> and I also believe that one of the reason we got duplications exactly >> because these tests were located in jdk test suite." >> >>> BTW: >>> >>> test/hotspot/jtreg/runtime/ShiftTest.java >>> >>> is actually a jit test according to the test comment. >> sure, I will move it to hotspot/compiler. >>>> there are 16 tests in test/jdk/vm directory. all but >>>> JniInvocationTest are hotspot tests, so they are moved to >>>> test/hotspot test suite; JniInvocationTest is a launcher test >>> No its a JNI invocation API test - nothing to do with our launcher. >>> It belongs in runtime/jni. But we already have tests in runtime that >>> use the JNI invocation API so this test adds no new coverage. >> this is actually was my first reaction, and I even have the webrev >> which moves it to runtime/jni, but then I looked at the associated >> bug and it is filed against tools/launcher. and I even got a false >> (as I know by now) memory that I saw JLI_ method being called from >> the test. there is actually another test >> (dk/tools/launcher/exeJliLaunchTest.c) associated w/ this bug which >> calls JLI_Launch. anyhow, I'll move this test to hotspot/runtime/jni. >> >>> I really think the value of these tests needs to be examined before >>> they are brought over. >> I'd prefer to have follow-up RFEs/tasks, b/c the longer we keep >> jdk/vm directory the more tests can end up there and the more rotten >> these tests become. >> >> Thanks, >> -- Igor >>> Thanks, >>> David >>> ----- >>> >>>> and hence it's moved to test/jdk/tools/launcher. as >>>> hotspot/compiler and hotspot/gc tests use packages, the tests moved >>>> there have been updated to have a package statement. >>>> webrev: >>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219139 >>>> testing: tier[1-2] (in progress), all the touched tests locally >>>> Thanks, >>>> -- Igor > From manc at google.com Mon Mar 4 22:41:43 2019 From: manc at google.com (Man Cao) Date: Mon, 4 Mar 2019 14:41:43 -0800 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> <7e0c775d-86c1-b80c-b1a6-373ca21206ba@oracle.com> Message-ID: Thanks for taking a look, Jiangli. I renamed "permanent gen" to "metaspace". I double checked that currently full GCs due to metaspace expansion will not affect the calculation of GC overhead . New webrev: https://cr.openjdk.java.net/~manc/8212206/webrev.04/ -Man On Sun, Mar 3, 2019 at 11:48 AM Jiangli Zhou wrote: > Hi Man, > > As both Thomas and Per seemed to be okay with the refactoring, you > probably had sufficient approval already. > > The following comment in adaptiveSizePolicy.cpp seems to be outdated. The > comment predates the PermGen removal. It's probably a good idea to also > cleanup the comment as part of your refactoring change. > > 281 // Note that the gc time limit test only works for the collections 282 // of the young gen + tenured gen and not for collections of the 283 // permanent gen. That is because the calculation of the space 284 // freed by the collection is the free space in the young gen + 285 // tenured gen. > > Thanks and regards, > > Jiangli > > > On Fri, Mar 1, 2019 at 1:43 PM Man Cao wrote: > >> Hi, >> >> I rebased the patch to latest tip: >> https://cr.openjdk.java.net/~manc/8212206/webrev.03/ >> >> Could someone give a second "looks good"? >> >> -Man >> >> >> On Sat, Jan 26, 2019 at 4:35 PM Man Cao wrote: >> >>> Friendly ping. Could anyone give a second "looks good"? >>> >>> As for the develop flag >>> AdaptiveSizePolicyGCTimeLimitThreshold/GCOverheadLimitThreshold, I added a >>> note about it in https://bugs.openjdk.java.net/browse/JDK-8212084. >>> >>> -Man >>> >>> >>> On Tue, Jan 15, 2019 at 6:41 PM Man Cao wrote: >>> >>>> Hi, >>>> >>>> I rebased the patch to tip and updated year in some headers to 2019, >>>> without making any real change: >>>> http://cr.openjdk.java.net/~manc/8212206/webrev.02/ >>>> >>>> >>>> I don't foresee that this will be implemented, or even makes sense, for >>>>> ZGC. As I see it, this is only a thing STW collectors. For that >>>>> reason, >>>>> I don't think it belongs in CollectedHeap. Keeping it as a separate >>>>> utility class for collectors that want to use it sounds better. >>>>> >>>> Sounds good to keep this patch in the current state, without further >>>> changing the CollectedHeap class. >>>> >>>> I haven't looked very closely at the patch, but couldn't help to notice >>>>> that the option is called "GCOverheapLimitThreshold" (and >>>>> "AdaptiveSizePolicyGCTimeLimitThreshold" before that), which is a >>>>> tautology and a not very good description of what it is. >>>>> How about we take the opportunity to clean this up and completely >>>>> ditch >>>>> the "gc_overhead_limit_count" thing and get rid of this option? It's a >>>>> "develop" option, so it's not available to normal users anyway. Has >>>>> anyone of you ever used this option and actually find it valuable? >>>> >>>> I didn't find any users inside Google that require changing this option. >>>> That said, some users did complain that UseGCOverheadLimit for >>>> ParallelGC or CMS is too difficult to get >>>> triggered, because of the requirement for 5 consecutive full GCs, which >>>> is set by this option. >>>> I think if it were a normal "product" option, there will definitely be >>>> users setting it. >>>> I never understand why it is a "develop" option. I think we could >>>> either remove it, >>>> or make it an "experimental" option. >>>> I'm leaning towards not removing it for now, as I'm not sure if 5 is >>>> still a reasonable >>>> default value for UseGCOverheadLimit for G1. >>>> How about we decide whether to keep or remove this option after >>>> JDK-8212084 (UseGCOverheadLimit for G1) is fixed? >>>> >>>> Also for the hsperfdata counter change, I created >>>> https://bugs.openjdk.java.net/browse/JDK-8217221. I will draft a CSR >>>> for it later. >>>> >>>> -Man >>>> >>> From david.holmes at oracle.com Tue Mar 5 00:35:59 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Mar 2019 10:35:59 +1000 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class In-Reply-To: <24d69f60-3fe3-92af-fbcf-62b33b21c240@oracle.com> References: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> <24d69f60-3fe3-92af-fbcf-62b33b21c240@oracle.com> Message-ID: Thanks Vladimir! David On 5/03/2019 7:53 am, Vladimir Ivanov wrote: > Looks good. > > Best regards, > Vladimir Ivanov > > On 03/03/2019 23:17, David Holmes wrote: >> webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8219974 >> >> JDK-8219492 was previously reviewed: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html >> >> >> and pushed: >> >> http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 >> >> but the new assertions failed on a single test that had an >> invokespecial call to an interface method for an interface that was >> not (and should not be) initialized. So it was backed out. >> >> The safest fix is to ensure the new assertions only apply when >> considering invoke_static. So that's the only difference between this >> version and the original. >> >> Testing: >> ?? - failing test from tier 4 in regular and Xcomp mode >> ?? - tiers 1-3 >> >> Thanks, >> David >> ----- From david.holmes at oracle.com Tue Mar 5 00:36:10 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Mar 2019 10:36:10 +1000 Subject: RFR: 8219974: REDO JDK-8219492: Restore static callsite resolution for the current class In-Reply-To: References: <5ccb2be8-d7fe-8320-0f87-d48d31c431d0@oracle.com> <9f298b00-3066-0cf8-f4ad-5139ffccd65b@oracle.com> <700e86d6-d8c6-e60d-9548-2057674c7be6@oracle.com> Message-ID: <1f08c75b-ae9f-055a-d605-43972d49c9d2@oracle.com> Thanks Coleen! David On 5/03/2019 7:54 am, coleen.phillimore at oracle.com wrote: > > This looks good. > Coleen > > On 3/4/19 4:32 PM, David Holmes wrote: >> Can I get a second (re)review please. >> >> Thanks, >> David >> >> On 4/03/2019 9:24 pm, David Holmes wrote: >>> Thanks Claes! >>> >>> David >>> >>> On 4/03/2019 8:14 pm, Claes Redestad wrote: >>>> Hi David, >>>> >>>> moving the new assertion into the invokestatic branch looks good. >>>> >>>> Thanks! >>>> >>>> /Claes >>>> >>>> On 2019-03-04 08:17, David Holmes wrote: >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8219974/webrev/ >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8219974 >>>>> >>>>> JDK-8219492 was previously reviewed: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037036.html >>>>> >>>>> >>>>> and pushed: >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/raw-rev/86ee52ca11e3 >>>>> >>>>> but the new assertions failed on a single test that had an >>>>> invokespecial call to an interface method for an interface that was >>>>> not (and should not be) initialized. So it was backed out. >>>>> >>>>> The safest fix is to ensure the new assertions only apply when >>>>> considering invoke_static. So that's the only difference between >>>>> this version and the original. >>>>> >>>>> Testing: >>>>> ?? - failing test from tier 4 in regular and Xcomp mode >>>>> ?? - tiers 1-3 >>>>> >>>>> Thanks, >>>>> David >>>>> ----- > From manc at google.com Tue Mar 5 02:20:03 2019 From: manc at google.com (Man Cao) Date: Mon, 4 Mar 2019 18:20:03 -0800 Subject: RFR (M): 8212206: Refactor AdaptiveSizePolicy to separate out code related to GC overhead In-Reply-To: References: <6b1e59ec7f4746e8e071fd44ec91ca966fac8d78.camel@oracle.com> <7e0c775d-86c1-b80c-b1a6-373ca21206ba@oracle.com> Message-ID: After offline discussion with Jiangli, we think it is better to just remove this comment block. The reason is that full GCs due to metaspace expansion will not take the code path to execute AdaptiveSizePolicy::check_gc_overhead_limit(), because of the check for AdaptiveSizePolicy::should_update_promo_stats(). New webrev: https://cr.openjdk.java.net/~manc/8212206/webrev.05/ -Man On Mon, Mar 4, 2019 at 2:41 PM Man Cao wrote: > Thanks for taking a look, Jiangli. I renamed "permanent gen" to > "metaspace". > I double checked that currently full GCs due to metaspace expansion will > not affect the calculation of GC overhead . > > New webrev: https://cr.openjdk.java.net/~manc/8212206/webrev.04/ > > -Man > > > On Sun, Mar 3, 2019 at 11:48 AM Jiangli Zhou > wrote: > >> Hi Man, >> >> As both Thomas and Per seemed to be okay with the refactoring, you >> probably had sufficient approval already. >> >> The following comment in adaptiveSizePolicy.cpp seems to be outdated. The >> comment predates the PermGen removal. It's probably a good idea to also >> cleanup the comment as part of your refactoring change. >> >> 281 // Note that the gc time limit test only works for the collections 282 // of the young gen + tenured gen and not for collections of the 283 // permanent gen. That is because the calculation of the space 284 // freed by the collection is the free space in the young gen + 285 // tenured gen. >> >> Thanks and regards, >> >> Jiangli >> >> From erik.osterlund at oracle.com Tue Mar 5 08:09:41 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 5 Mar 2019 09:09:41 +0100 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> Message-ID: <04328CE3-6CCE-4009-AD39-6DFE7A66C7B7@oracle.com> Hi Tobias, > On 27 Feb 2019, at 13:13, Tobias Hartmann wrote: > > Hi Vladimir, > > thanks for the review! > >> On 26.02.19 19:36, Vladimir Kozlov wrote: >> My only concern is CompileTask::select_for_compilation() where you may have small window between >> releasing weak reference and adding strong. Can we add strong reference fist before removing weak? >> >> With concurrent class unloading we may hit the assert in select_for_compilation() I think. May be >> make the method get task as argument and return NULL if is_unloaded() is true. What do you think? > > I thought we should be safe because we can't have a safepoint after the is_unloaded() check in > CompilationPolicy::policy()->select_task() and this assert (there is a NoSafepointVerifier in the > caller). But I'm not sure what the guarantees are with concurrent class unloading. > > I've changed the code to create the (local) handle before removing the weak handles: > http://cr.openjdk.java.net/~thartmann/8163511/webrev.01/ > > Now I'm wondering what will happen if concurrent class unloading kicks in right after the > is_unloaded() check and before we create the method_holder handle. > > I think someone from GC/runtime (Erik ?sterlund?) should have a look at this. Liveness of objects only change in safepoints. So if you are in the vm thread state, and ask if a holder is alive, its result is stable. With concurrent class unloading, marking terminates in a safepoint. At that snapshot in time, it is determined what is alive (strongly and through finalizers) and dead (although it might be lazily evaluated during concurrent execution). When that safepoint is released, that view will be consistent. As there are seemingly no safepoint checks in this code, all liveness information will be consistent and none of the imagined races exist with concurrent class unloading. It behaves as if it was all done in a safepoint, even though it was not. The key is that the klass_holder liveness (as well as the jweaks you use) is determined using a phantom load via the access API. That phantom load gives consistent results in the vm thread state as-if reference processing was done in a safepoint. The phantom loads can lazily compute what the value would have been, had reference processing been done in a safepoint, as it has stable liveness information from that safepoint. Then concurrent reference processing races for producing the same values. Basically there is nothing to worry about here as far as I can see. Looks good. Although freeing the jweaks in one place instead of two would make the code easier to follow. Up to you though and I don?t need another webrev for that. Thanks, /Erik > >> Do you kept output format for flag -XX:+PrintCompilation and changed only hs_err output? Or you >> changed both? > > I've changed both because both are using CompileTask::print -> CompileTask::print_impl. What about > guarding the additional output by -XX:+Verbose (see new webrev)? > > Best regards, > Tobias > > P.S. I'll be out until Wednesday next week. > > >>> On 2/26/19 8:39 AM, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following patch: >>> https://bugs.openjdk.java.net/browse/JDK-8163511 >>> http://cr.openjdk.java.net/~thartmann/8163511/webrev.00/ >>> >>> We hit an assert while trying to allocate a compile task because the compile queue already contains >>> 10.000 tasks. The root cause is that the compiler threads are not able to keep up with the high >>> number of compilation requests. The (internal) test that triggers this runs many different tests in >>> the same VM by using different class loaders. Many of these tests use the same framework and >>> therefore the same methods are enqueued for compilation again and again (up to 100x). >>> >>> The bug shows up more often when running with -XX:-TieredCompilation which is due to a lower number >>> of compiler threads and a different threshold policy that does not remove stale (unused) methods >>> from the compile queue. But the same problem can and does happen with TieredCompilation enabled (for >>> example, we never remove "old" methods that were executed a lot, even if they are stale). >>> >>> After a test has been executed, the class loader becomes dead and therefore all test classes/methods >>> should be unloaded. However, a compile task in the queue keeps the referenced method alive through a >>> global handle (see CompileTask::initialize -> JNIHandles::make_global) until compilation finished or >>> the task was removed from the queue for another reason. This causes compilation of thousands of dead >>> methods. >>> >>> The regression test TestOverloadCompileQueues.java triggers this reliably for the C1 and C2 compile >>> queues. The blue lines show the growth of the number of tasks in the queue until we hit the assert, >>> the red lines show the growth with the fix: >>> http://cr.openjdk.java.net/~thartmann/8163511/8163511_results.pdf >>> >>> The proposed fix uses weak handles for the compile tasks that don't prevent unloading but can be >>> used to check if the referenced method is still alive (CompileTask::is_unloaded()). Unloaded tasks >>> are aggressively removed with both threshold policies. If the method is still alive when the task is >>> selected for compilation, the weak references are replaced by strong references (to avoid unloading >>> during compilation which would crash the compiler). >>> >>> Of course, overloading the queue is still possible if classes are not unloaded fast enough or not at >>> all (in the regression test, simply remove the System.gc() call). But I think this is very unlikely >>> for real world applications and this assert does not affect product builds. >>> >>> The fix also changes the printing of compile tasks such that the time when they were enqueued and >>> the time when compilation started is shown in the hs_err file: >>> >>> Threads with active compile tasks: >>> C2 CompilerThread0 22446 18824 200 5372 4 java.lang.reflect.Method:: (68 bytes) >>> C1 CompilerThread0 22446 21474 199 3377 2 jdk.internal.loader.URLClassPath$3::run (172 bytes) >>> >>> I'm currently running extended testing. >>> >>> Thanks, >>> Tobias >>> From harold.seigel at oracle.com Tue Mar 5 14:51:20 2019 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 5 Mar 2019 09:51:20 -0500 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <7144fe67-ee8c-459a-e595-b70f397489db@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <7144fe67-ee8c-459a-e595-b70f397489db@oracle.com> Message-ID: <7d21cc77-29e6-efa9-d418-0d98cdcafdb4@oracle.com> Hi Coleen, Thanks for reviewing this! See comments inline. On 3/1/2019 6:22 PM, coleen.phillimore at oracle.com wrote: > > http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/src/hotspot/share/prims/unsafe.cpp.frames.html > > > 448 GuardUnsafeAccess guard(thread); > > > I can't find 'thread' in this scope.? Does line 433 need to be in the > scope above it? The JVM_ENTRY_FROM_LEAF at line 440 defines 'thread': #define JVM_ENTRY_FROM_LEAF(env, result_type, header)??????????????? \ ? { { \ ??? JavaThread* thread=JavaThread::thread_from_jni_environment(env); \ ??? ThreadInVMfromNative __tiv(thread);????????????????????????????? \ ??? debug_only(VMNativeEntryWrapper __vew;)????????????????????????? \ ??? VM_ENTRY_BASE_FROM_LEAF(result_type, header, thread) > > 150 /** > 151 * Helper class to wrap memory accesses in > JavaThread::doing_unsafe_access() > 152 */ > > Can you use // comments here? Done > > http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/test/hotspot/jtreg/runtime/Unsafe/InternalErrorTest.java.html > > > ? 33? *?? @build sun.hotspot.WhiteBox > > > Can you fix the indentation here? Done. Thanks, Harold > > I don't see the compiler changes. > > Coleen > > On 2/28/19 3:22 PM, Harold Seigel wrote: >> Hi, >> >> Please review this fix for JDK-8191278.? The fix uses the >> GuardUnsafeAccess class in unsafe.cpp so that the signal handler will >> know to convert SIGBUS signals to exceptions. >> >> The compiler related changes and much of the test were written by >> Jamsheesh (jcm). >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/index.html >> >> JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8191278 >> >> The fix was tested on Linux x64, Mac OS, Solaris Sparc, and Windows.? >> Testing was not done on the other platforms changed by this fix. >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests >> and builds on Linux x64, Solaris, Windows, and Mac OS X, by running >> Mach5 tiers 3 -5 on Linux x64, and by running JCK-13 Lang and VM >> tests on Linux x64. >> >> Thanks, Harold >> > From erik.osterlund at oracle.com Tue Mar 5 15:01:56 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 5 Mar 2019 16:01:56 +0100 Subject: RFR: 8219816: Add IsArray/RemoveArray type traits utilities In-Reply-To: <10f422e7-9fef-4876-23fd-608f1487dbb7@oracle.com> References: <8FDD08D8-6187-41DB-8773-8921A9DFF0E5@oracle.com> <387BFF37-0572-4341-BA12-BC0857D55ABA@oracle.com> <10f422e7-9fef-4876-23fd-608f1487dbb7@oracle.com> Message-ID: <3d79ea75-87fc-d774-00dc-9fdbbfc1e542@oracle.com> Hi Per, Looks good to me. Thanks, /Erik On 2019-03-04 10:48, Per Liden wrote: > On 2/27/19 9:21 PM, Kim Barrett wrote: >>> On Feb 27, 2019, at 10:08 AM, Kim Barrett >>> wrote: >>> >>>> On Feb 27, 2019, at 3:53 AM, Per Liden wrote: >>>> >>>> Add IsArray/RemoveArray to our collection of type traits >>>> metaprogramming utilities. >>>> >>>> This patch only adds the IsArray/RemoveArray utilities and their >>>> respective tests. A later patch I have in my queue that will >>>> actually make use of these. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8219816 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8219816/webrev.0 >>>> >>>> /Per >>> >>> For completeness, there should probably be IsArray test cases for >>> const/volatile with size. >>> >>> Otherwise, looks good.? I don't need another webrev for those >>> additional test cases. >> >> HotSpot's metaprogramming utilities have been intentionally modelled >> on those from , both in naming and (with some limitations >> in a few cases) behavior.? The C++11 type trait for removing array >> dimensions from an array type is "remove_extent" (and >> "remove_all_extents" >> to remove all of them from a multi-dimensional array type). >> >> So what this change calls RemoveArray ought to be called RemoveExtent. >> Sorry that I forgot to check on this earlier. > > Ah, good catch. Renamed accordingly. I'm not implementing > remove_all_extents at this time, let's add that when there's a use > case for it. > > Updated webrev: http://cr.openjdk.java.net/~pliden/8219816/webrev.1 > > /Per From volker.simonis at gmail.com Tue Mar 5 15:53:08 2019 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 5 Mar 2019 16:53:08 +0100 Subject: Result: New hotspot Group Member: Martin Doerr Message-ID: The vote for Martin Doerr [1] is now closed. Yes: 18 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Volker Simonis [1] https://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/thread.html#36816 From volker.simonis at gmail.com Tue Mar 5 15:53:20 2019 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 5 Mar 2019 16:53:20 +0100 Subject: Result: New hotspot Group Member: Thomas Stuefe Message-ID: The vote for Thomas Stuefe [1] is now closed. Yes: 17 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Volker Simonis [1] https://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/thread.html#36815 From tobias.hartmann at oracle.com Wed Mar 6 10:44:28 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 6 Mar 2019 11:44:28 +0100 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: <7f975ada-a265-533e-e9a9-e90b2e2121a4@oracle.com> References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> <7f975ada-a265-533e-e9a9-e90b2e2121a4@oracle.com> Message-ID: Hi Coleen, thanks for the review! On 27.02.19 17:25, coleen.phillimore at oracle.com wrote: > Nit, can you use // comments. Sure, I was just using /* */ comments in compileTask.cpp to adapt to existing code. Fixed it. > Also, there should be a HandleMark here. There already is a HandleMark in the compiler thread loop: http://hg.openjdk.java.net/jdk/jdk/file/8b341d6728fe/src/hotspot/share/compiler/compileBroker.cpp#l1774 Do you want me to add another one to CompileTask::select_for_compilation()? > I think this method also needs to check for is_unloading() before touching method(). > > // RedefineClasses support > void CompileTask::metadata_do(void f(Metadata*)) { > ? f(method()); > ? if (hot_method() != NULL && hot_method() != method()) { > ??? f(hot_method()); > ? } > } Right, fixed it. I've also added an assert to CompileTask::mark_on_stack(). New webrev: http://cr.openjdk.java.net/~thartmann/8163511/webrev.02 > This change makes sense to me.? It would be nice if these were in VM mode, if you could use > WeakHandles instead of JNI, but you would need your own weak handle area.? For future consideration, > anyway. Yes, I think that cleanup is out of the scope of this fix. Thanks, Tobias From tobias.hartmann at oracle.com Wed Mar 6 10:44:10 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 6 Mar 2019 11:44:10 +0100 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: <784b2fc7-6038-53d3-9bab-09482b0a887c@oracle.com> References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> <784b2fc7-6038-53d3-9bab-09482b0a887c@oracle.com> Message-ID: Hi Vladimir, thanks for the review. On 27.02.19 18:21, Vladimir Kozlov wrote: > I am fine with Verbose flag if you are also fine that it is debug flag - you will get this output > only with debug VM. Yes, I think that's fine. We can enable it once we suspect a compilation that is stuck. Best regards, Tobias From tobias.hartmann at oracle.com Wed Mar 6 10:50:34 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 6 Mar 2019 11:50:34 +0100 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: <04328CE3-6CCE-4009-AD39-6DFE7A66C7B7@oracle.com> References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> <04328CE3-6CCE-4009-AD39-6DFE7A66C7B7@oracle.com> Message-ID: Hi Erik, On 05.03.19 09:09, Erik Osterlund wrote: > Liveness of objects only change in safepoints. So if you are in the vm thread state, and ask if a holder is alive, its result is stable. > > With concurrent class unloading, marking terminates in a safepoint. At that snapshot in time, it is determined what is alive (strongly and through finalizers) and dead (although it might be lazily evaluated during concurrent execution). When that safepoint is released, that view will be consistent. > > As there are seemingly no safepoint checks in this code, all liveness information will be consistent and none of the imagined races exist with concurrent class unloading. It behaves as if it was all done in a safepoint, even though it was not. > > The key is that the klass_holder liveness (as well as the jweaks you use) is determined using a phantom load via the access API. That phantom load gives consistent results in the vm thread state as-if reference processing was done in a safepoint. The phantom loads can lazily compute what the value would have been, had reference processing been done in a safepoint, as it has stable liveness information from that safepoint. Then concurrent reference processing races for producing the same values. > > Basically there is nothing to worry about here as far as I can see. Looks good. Thanks a lot for the thorough explanation and the review. > Although freeing the jweaks in one place instead of two would make the code easier to follow. Up to you though and I don?t need another webrev for that. I could only factor the freeing into a separate method and call it from CompileTask::free() and CompileTask::select_for_compilation(). But I don't think that would make the code easier to read. Relying on the assumption that either both _method_holder and _hot_method_holder are weak references or both are strong, I've slightly refactored the code in CompileTask::free(): http://cr.openjdk.java.net/~thartmann/8163511/webrev.02 Best regards, Tobias From erik.osterlund at oracle.com Wed Mar 6 11:14:49 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 6 Mar 2019 12:14:49 +0100 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> <04328CE3-6CCE-4009-AD39-6DFE7A66C7B7@oracle.com> Message-ID: Hi Tobias, On 2019-03-06 11:50, Tobias Hartmann wrote: > Hi Erik, > > On 05.03.19 09:09, Erik Osterlund wrote: >> Liveness of objects only change in safepoints. So if you are in the vm thread state, and ask if a holder is alive, its result is stable. >> >> With concurrent class unloading, marking terminates in a safepoint. At that snapshot in time, it is determined what is alive (strongly and through finalizers) and dead (although it might be lazily evaluated during concurrent execution). When that safepoint is released, that view will be consistent. >> >> As there are seemingly no safepoint checks in this code, all liveness information will be consistent and none of the imagined races exist with concurrent class unloading. It behaves as if it was all done in a safepoint, even though it was not. >> >> The key is that the klass_holder liveness (as well as the jweaks you use) is determined using a phantom load via the access API. That phantom load gives consistent results in the vm thread state as-if reference processing was done in a safepoint. The phantom loads can lazily compute what the value would have been, had reference processing been done in a safepoint, as it has stable liveness information from that safepoint. Then concurrent reference processing races for producing the same values. >> >> Basically there is nothing to worry about here as far as I can see. Looks good. > Thanks a lot for the thorough explanation and the review. > >> Although freeing the jweaks in one place instead of two would make the code easier to follow. Up to you though and I don?t need another webrev for that. > I could only factor the freeing into a separate method and call it from CompileTask::free() and > CompileTask::select_for_compilation(). But I don't think that would make the code easier to read. Ah, okay. > Relying on the assumption that either both _method_holder and _hot_method_holder are weak references > or both are strong, I've slightly refactored the code in CompileTask::free(): > > http://cr.openjdk.java.net/~thartmann/8163511/webrev.02 The new webrev looks good to me. Thanks, /Erik > > Best regards, > Tobias From thomas.stuefe at gmail.com Wed Mar 6 11:27:06 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 6 Mar 2019 12:27:06 +0100 Subject: RFR(S): 8219650: [Testbug] Fix potential crashes in new test hotspot gtest "test_print_hex_dump" In-Reply-To: References: <19194f67-ed19-0031-e1a9-e79b56a777f0@redhat.com> <04555104-3ad3-0135-e45c-2ae4e0a621bd@redhat.com> <33b1d1bb-3837-40e3-08f7-313be235c213@redhat.com> Message-ID: Okay next try: http://cr.openjdk.java.net/~stuefe/webrevs/8219650-shade-print-hex/webrev.01/webrev/ Fixed problems: - On AIX, we can read from the zero page. So, test has to use another address. Briefly considered reserving and protecting a page, but this is actually simpler. - On Windows 32bit, we cannot call any VM functions from gtest which require signal handling to work (e.g. anything relying on SafeFetch) since that depends on SEH handler to be set up correctly somewhere above the caller frame. This is done for all "normal" entry points into the VM, e.g. CreateJavaVM or thread start functions. However, if one just calls directly into the VM, we do not have signal handling in place. Or, more specifically, we have the SEH handler of the gtest framework, which assumes the test segfaulted. I am actually confused why this works on 64bit windows. One possibility would be that someone uses VectoredException handling, which is a handling mechanism similar to Unix. I see it switched on for AOT - maybe that is the reason it "accidentally" works. Not sure, and actually not eager to solve this now, so I openend a new issue: https://bugs.openjdk.java.net/browse/JDK-8220220. Note the pretty bug id. So, for this patch, I just excluded the part of the test reading unreadable memory from Windows. Cheers, Thomas On Fri, Mar 1, 2019 at 10:18 AM Langer, Christoph wrote: > Hi Thomas, > > > > Unless it really irks you, lets keep it that way. We all spent way too > much > > time on this thing, and > > > while it is a really fascinating topic... :) > > > > Yeah, no reason to sink more time into it. Let's move on! > > It's not done yet ?? > > Looking at our nightly results where we test the patch in 11u context, we > can see that the crash on Solaris is gone but the test error in Win32 is > not fixed yet and on AIX the test fails now. All the platforms we love most > ?? > > Best regards > Christoph > > From shade at redhat.com Wed Mar 6 11:27:25 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 6 Mar 2019 12:27:25 +0100 Subject: RFR (XS) 8220228: Improve Shenandoah pacing histogram message Message-ID: <758bafed-fbae-2d98-b8a2-d74c3574d882@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8220228 Patch: diff -r f88fbb5fc085 src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp Wed Mar 06 12:01:55 2019 +0100 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp Wed Mar 06 12:26:06 2019 +0100 > @@ -297,6 +297,12 @@ > > out->print_cr("%7d ms - %7d ms: " SIZE_FORMAT_W(12) SIZE_FORMAT_W(12) " ms", l, r, count, sum); > } > out->print_cr("%23s: " SIZE_FORMAT_W(12) SIZE_FORMAT_W(12) " ms", "Total", total_count, total_sum); > out->cr(); > + out->print_cr("Pacing delays are measured from entering the pacing code till exiting it. Therefore,"); > + out->print_cr("observed pacing delays may be higher than the threshold when paced thread spent more"); > + out->print_cr("time in the pacing code. It usually happens when thread is de-scheduled while paced,"); > + out->print_cr("OS takes longer to unblock the thread, or JVM experiences the STW pause."); > + out->cr(); > + > } Testing: Linux x86_64 build, eyeballing -Xlog:gc+stats Thanks, -Aleksey From shade at redhat.com Wed Mar 6 11:44:18 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 6 Mar 2019 12:44:18 +0100 Subject: RFR(S): 8219650: [Testbug] Fix potential crashes in new test hotspot gtest "test_print_hex_dump" In-Reply-To: References: <19194f67-ed19-0031-e1a9-e79b56a777f0@redhat.com> <04555104-3ad3-0135-e45c-2ae4e0a621bd@redhat.com> <33b1d1bb-3837-40e3-08f7-313be235c213@redhat.com> Message-ID: <4a6a31b7-2b51-76cf-c6ad-9891b908926f@redhat.com> On 3/6/19 12:27 PM, Thomas St?fe wrote: > Okay next try: > > http://cr.openjdk.java.net/~stuefe/webrevs/8219650-shade-print-hex/webrev.01/webrev/ Looks okay. Boy, what a ride. Thanks for doing this! *) I have no strong opinion about this, but maybe we would universally call that non-zero address? It would probably require more work figuring out different bitness, so this one is also good. Maybe one-line it with AIX_ONLY(...) NOT_AIX(0)? 188 // On AIX, zero page is readable. 189 address unreadable = 190 #ifdef AIX 191 (address) 0xFFFFFFFFFFFF0000ULL; 192 #else 193 (address) 0 194 #endif 195 ; -Aleksey From tobias.hartmann at oracle.com Wed Mar 6 11:48:07 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 6 Mar 2019 12:48:07 +0100 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> <04328CE3-6CCE-4009-AD39-6DFE7A66C7B7@oracle.com> Message-ID: <5f1f28f3-4824-7e8b-c91b-e1f973563f34@oracle.com> Hi Erik, thanks again for the review. Best regards, Tobias On 06.03.19 12:14, Erik ?sterlund wrote: > Hi Tobias, > > On 2019-03-06 11:50, Tobias Hartmann wrote: >> Hi Erik, >> >> On 05.03.19 09:09, Erik Osterlund wrote: >>> Liveness of objects only change in safepoints. So if you are in the vm thread state, and ask if a >>> holder is alive, its result is stable. >>> >>> With concurrent class unloading, marking terminates in a safepoint. At that snapshot in time, it >>> is determined what is alive (strongly and through finalizers) and dead (although it might be >>> lazily evaluated during concurrent execution). When that safepoint is released, that view will be >>> consistent. >>> >>> As there are seemingly no safepoint checks in this code, all liveness information will be >>> consistent and none of the imagined races exist with concurrent class unloading. It behaves as if >>> it was all done in a safepoint, even though it was not. >>> >>> The key is that the klass_holder liveness (as well as the jweaks you use) is determined using a >>> phantom load via the access API. That phantom load gives consistent results in the vm thread >>> state as-if reference processing was done in a safepoint. The phantom loads can lazily compute >>> what the value would have been, had reference processing been done in a safepoint, as it has >>> stable liveness information from that safepoint. Then concurrent reference processing races for >>> producing the same values. >>> >>> Basically there is nothing to worry about here as far as I can see. Looks good. >> Thanks a lot for the thorough explanation and the review. >> >>> Although freeing the jweaks in one place instead of two would make the code easier to follow. Up >>> to you though and I don?t need another webrev for that. >> I could only factor the freeing into a separate method and call it from CompileTask::free() and >> CompileTask::select_for_compilation(). But I don't think that would make the code easier to read. > > Ah, okay. > >> Relying on the assumption that either both _method_holder and _hot_method_holder are weak references >> or both are strong, I've slightly refactored the code in CompileTask::free(): >> >> http://cr.openjdk.java.net/~thartmann/8163511/webrev.02 > > The new webrev looks good to me. > > Thanks, > /Erik > >> >> Best regards, >> Tobias > From thomas.stuefe at gmail.com Wed Mar 6 11:49:47 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 6 Mar 2019 12:49:47 +0100 Subject: RFR(S): 8219650: [Testbug] Fix potential crashes in new test hotspot gtest "test_print_hex_dump" In-Reply-To: <4a6a31b7-2b51-76cf-c6ad-9891b908926f@redhat.com> References: <19194f67-ed19-0031-e1a9-e79b56a777f0@redhat.com> <04555104-3ad3-0135-e45c-2ae4e0a621bd@redhat.com> <33b1d1bb-3837-40e3-08f7-313be235c213@redhat.com> <4a6a31b7-2b51-76cf-c6ad-9891b908926f@redhat.com> Message-ID: On Wed, Mar 6, 2019 at 12:44 PM Aleksey Shipilev wrote: > On 3/6/19 12:27 PM, Thomas St?fe wrote: > > Okay next try: > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8219650-shade-print-hex/webrev.01/webrev/ > > Looks okay. Boy, what a ride. Thanks for doing this! > I was feeling guilty since I talked you into this... :) > > *) I have no strong opinion about this, but maybe we would universally > call that non-zero address? > It would probably require more work figuring out different bitness, so > this one is also good. Maybe > one-line it with AIX_ONLY(...) NOT_AIX(0)? > > 188 // On AIX, zero page is readable. > 189 address unreadable = > 190 #ifdef AIX > 191 (address) 0xFFFFFFFFFFFF0000ULL; > 192 #else > 193 (address) 0 > 194 #endif > 195 ; > > Lets keep it AIX specific, otherwise in years from now questions come why we did not use 0. Also, I did not feel like checking if that address happens to be readable on all platforms. Then, there is 32bit... But I'll use the AIX_ONLY macro, that is better. ..Thomas > -Aleksey > > From rkennke at redhat.com Wed Mar 6 11:51:16 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 6 Mar 2019 12:51:16 +0100 Subject: RFR (XS) 8220228: Improve Shenandoah pacing histogram message In-Reply-To: <758bafed-fbae-2d98-b8a2-d74c3574d882@redhat.com> References: <758bafed-fbae-2d98-b8a2-d74c3574d882@redhat.com> Message-ID: <04e2b835-9d68-729e-6609-1fcfcd87dc94@redhat.com> I would say 'an STW pause' rather than 'the STW pause', other than that looks good. Thanks! Roman > RFE: > https://bugs.openjdk.java.net/browse/JDK-8220228 > > Patch: > > diff -r f88fbb5fc085 src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp >> --- a/src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp Wed Mar 06 12:01:55 2019 +0100 >> +++ b/src/hotspot/share/gc/shenandoah/shenandoahPacer.cpp Wed Mar 06 12:26:06 2019 +0100 >> @@ -297,6 +297,12 @@ >> >> out->print_cr("%7d ms - %7d ms: " SIZE_FORMAT_W(12) SIZE_FORMAT_W(12) " ms", l, r, count, sum); >> } >> out->print_cr("%23s: " SIZE_FORMAT_W(12) SIZE_FORMAT_W(12) " ms", "Total", total_count, total_sum); >> out->cr(); >> + out->print_cr("Pacing delays are measured from entering the pacing code till exiting it. Therefore,"); >> + out->print_cr("observed pacing delays may be higher than the threshold when paced thread spent more"); >> + out->print_cr("time in the pacing code. It usually happens when thread is de-scheduled while paced,"); >> + out->print_cr("OS takes longer to unblock the thread, or JVM experiences the STW pause."); >> + out->cr(); >> + >> } > > Testing: Linux x86_64 build, eyeballing -Xlog:gc+stats > > Thanks, > -Aleksey > From shade at redhat.com Wed Mar 6 14:04:53 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 6 Mar 2019 15:04:53 +0100 Subject: RFR (XS) 8220228: Improve Shenandoah pacing histogram message In-Reply-To: <04e2b835-9d68-729e-6609-1fcfcd87dc94@redhat.com> References: <758bafed-fbae-2d98-b8a2-d74c3574d882@redhat.com> <04e2b835-9d68-729e-6609-1fcfcd87dc94@redhat.com> Message-ID: <19f3b55f-a53b-339a-6c75-660aec4af6a1@redhat.com> On 3/6/19 12:51 PM, Roman Kennke wrote: > I would say 'an STW pause' rather than 'the STW pause', other than that looks good. Thanks! Right. Replaced with "an STW pause". -Aleksey From aph at redhat.com Wed Mar 6 15:23:22 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 6 Mar 2019 15:23:22 +0000 Subject: 8219993: AArch64: Compiled CI stubs are unsafely modified Message-ID: <3abc4b04-a6d4-62f7-4cd9-36a89fe020e4@redhat.com> According to the Arm manual, except in a few limited cases it is not safe to modify code while it is being executed. We mostly get this right, but our CompiledStaticCall stubs are patched at runtime by rewriting immediate loads. We can fix this by moving the rewritable data fields into memory so that they are loaded by means of load(mem) rather than mov(immediate data). After this patch, CI stubs look like: dmb ld ldr rmethod, L1 ldr rscratch1, L2 br [rscratch1] L1: .data 0 L2: .data 0 When we modify such a stub we rewrite the two data fields to the method metadata and the interpreter entry point, execute a memory fence, then rewrite the compiled jump to point to the stub. http://cr.openjdk.java.net/~aph/8219006.1/ -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From matthias.baesken at sap.com Wed Mar 6 17:13:46 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 6 Mar 2019 17:13:46 +0000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> Message-ID: Hello , could I push the latest version of 8219241 ? A change that moves the coding more into platform/cpu - specific coding (if this is wanted) could be done when bringing in the AIX virtualization related info in another patch which I plan to do . Thanks, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 1. M?rz 2019 09:09 > To: 'David Holmes' ; 'hotspot- > dev at openjdk.java.net' > Subject: RE: RFR: 8219241: Provide basic virtualization related info in the > hs_error file on linux/windows x86_64 > > Hi David , > > I think a clean way would be to have > > os::print_virtualization_info(outputStream* st) > > with platform and/or if needed platform/cpu - specific coding at the right > places . > > I suggested something similar back then . > > This would be also a useful structure to bring in the AIX virtualization > related info output . > > > Regards, Matthias > > > > -----Original Message----- > > From: David Holmes > > Sent: Donnerstag, 28. Februar 2019 22:49 > > To: Baesken, Matthias ; 'hotspot- > > dev at openjdk.java.net' > > Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > > hs_error file on linux/windows x86_64 > > > > Hi Matthias, > > > > Combining this with 8219746 I have to insist that we use a nice clean > > OS-CPU abstraction for dealing with this and get rid of the platform > > ifdefs in os::Linux::print_virtualization_info. What you've added in > > VM_Version would seem to be the right kind of platform specific > > functionality and it seems to me that you could get rid of > > os::Linux::print_virtualization_info and just use > > VM_Version::print_detected_virtualization in its place. > > > > Thanks, > > David > > > > On 1/03/2019 2:04 am, Baesken, Matthias wrote: > > > Hello, I provided a second version . > > > The x86_64 assembler code was simplified (thanks to Martin for the > hint) ; > > additionally some comments were adjusted . > > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.2/ > > > > > > > > > Best regards , Matthias > > > > > > > > > > > > Hello , please review the following change. > > > It provides basic virtualization related information in the hs_error file on > > linux/windows x86_64. > > > We use cpuid to get the information (detection of VMWare, Xen, KVM > and > > HyperV is included). > > > > > > Some info about how to detect the virtualization info can be found here : > > > https://kb.vmware.com/s/article/1009458 > > > > > > > > > > > > (it is VMware related but can be applied for other virtualization > > technologies as well) > > > > > > > > > An overview of the ID strings can be found here as well : > > > > > > https://en.wikipedia.org/wiki/CPUID > > > The following are known ID strings from virtual machines: ... > > > > > > > > > > > > bug/webrev : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8219241 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.1/ > > > > > > > > > > > > Thanks, Matthias > > > From gnu.andrew at redhat.com Wed Mar 6 18:25:49 2019 From: gnu.andrew at redhat.com (Andrew John Hughes) Date: Wed, 6 Mar 2019 18:25:49 +0000 Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) In-Reply-To: <5784a2dbe5944f07b1feb03b7b1f87e7@sap.com> References: <5784a2dbe5944f07b1feb03b7b1f87e7@sap.com> Message-ID: <4633f2bd-2036-e89c-5811-559d2f2b566a@redhat.com> On 22/11/2018 08:51, Lindenmaier, Goetz wrote: > Hi, > > Doesn't this have to be posted to jdk8u-dev? > > I had a look at the backport. > Including 7127191 confused me a bit. Is it good to hide the fact that > this was backported in the repository? > In os_linux one fix is missing, is this on purpose? I don't think this is a > critical issue, though, so leaving it out is fine. > >> the dropping of the changes to ... >> src/share/vm/runtime/task.cpp and >> src/os/windows/vm/attachListener_windows.cpp > These changes are included in the webrev ...? > > The webrev looks good to me. > > Best regards, > Goetz. > > > > > >> -----Original Message----- >> From: hotspot-dev On Behalf Of >> Andrew Hughes >> Sent: Mittwoch, 21. November 2018 07:45 >> To: serviceability-dev ; hotspot-dev >> >> Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8140482 >> Original changeset: >> https://hg.openjdk.java.net/jdk-updates/jdk9u/hotspot/rev/cd86b5699825 >> Webrev: >> https://cr.openjdk.java.net/~andrew/openjdk8/8140482/webrev.01/ >> >> The patch largely applies as is, with some adjustment for context and >> the dropping of the changes to src/cpu/x86/vm/stubRoutines_x86.cpp, >> src/share/vm/runtime/task.cpp and >> src/os/windows/vm/attachListener_windows.cpp >> which don't exist in 8u. A clean backport of 7127191 is included, which >> allows the changes to agent/src/os/linux/libproc_impl.c to apply as-is. >> >> Applying the change to 8u improves the code quality there and aids >> in backporting other changes, such as 8210836 [0]. >> >> Ok for 8u? >> >> [0] https://mail.openjdk.java.net/pipermail/serviceability-dev/2018- >> November/025991.html >> >> Thanks, >> -- >> Andrew :) >> >> Senior Free Java Software Engineer >> Red Hat, Inc. (http://www.redhat.com) >> >> Web Site: http://fuseyism.com >> Twitter: https://twitter.com/gnu_andrew_java >> PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) >> Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 Coming back to this after the 2019-01 CPU... I've flagged 7127191 with jdk8u-fix-request and I'll get that in first before revising this as a webrev for 8140482 only. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 https://keybase.io/gnu_andrew From vladimir.kozlov at oracle.com Wed Mar 6 18:27:13 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 6 Mar 2019 10:27:13 -0800 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> <04328CE3-6CCE-4009-AD39-6DFE7A66C7B7@oracle.com> Message-ID: Yes, this looks good to me too. Thanks, Vladimir On 3/6/19 3:14 AM, Erik ?sterlund wrote: > Hi Tobias, > > On 2019-03-06 11:50, Tobias Hartmann wrote: >> Hi Erik, >> >> On 05.03.19 09:09, Erik Osterlund wrote: >>> Liveness of objects only change in safepoints. So if you are in the vm thread state, and ask if a holder is alive, >>> its result is stable. >>> >>> With concurrent class unloading, marking terminates in a safepoint. At that snapshot in time, it is determined what >>> is alive (strongly and through finalizers) and dead (although it might be lazily evaluated during concurrent >>> execution). When that safepoint is released, that view will be consistent. >>> >>> As there are seemingly no safepoint checks in this code, all liveness information will be consistent and none of the >>> imagined races exist with concurrent class unloading. It behaves as if it was all done in a safepoint, even though it >>> was not. >>> >>> The key is that the klass_holder liveness (as well as the jweaks you use) is determined using a phantom load via the >>> access API. That phantom load gives consistent results in the vm thread state as-if reference processing was done in >>> a safepoint. The phantom loads can lazily compute what the value would have been, had reference processing been done >>> in a safepoint, as it has stable liveness information from that safepoint. Then concurrent reference processing races >>> for producing the same values. >>> >>> Basically there is nothing to worry about here as far as I can see. Looks good. >> Thanks a lot for the thorough explanation and the review. >> >>> Although freeing the jweaks in one place instead of two would make the code easier to follow. Up to you though and I >>> don?t need another webrev for that. >> I could only factor the freeing into a separate method and call it from CompileTask::free() and >> CompileTask::select_for_compilation(). But I don't think that would make the code easier to read. > > Ah, okay. > >> Relying on the assumption that either both _method_holder and _hot_method_holder are weak references >> or both are strong, I've slightly refactored the code in CompileTask::free(): >> >> http://cr.openjdk.java.net/~thartmann/8163511/webrev.02 > > The new webrev looks good to me. > > Thanks, > /Erik > >> >> Best regards, >> Tobias > From ioi.lam at oracle.com Wed Mar 6 18:42:18 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 6 Mar 2019 10:42:18 -0800 Subject: Result: New hotspot Group Member: Calvin Cheung Message-ID: <5ab79422-4309-fff9-a769-9e65884c7556@oracle.com> The vote for Calvin Cheung [1] is now closed. Yes: 16 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Ioi Lam [1] https://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/thread.html#36897 From nati.zelig at overopshq.com Tue Mar 5 07:48:58 2019 From: nati.zelig at overopshq.com (Nati Zelig) Date: Tue, 5 Mar 2019 09:48:58 +0200 Subject: JDK-8210457 : JVM crash in ResolvedMethodTable::add_method(Handle) Message-ID: Hello all, The company I work for is developing a solution for JVM's, as a native agent which performing redefinition of some of the classes loaded. Since JDK 10 (Tested up to 11), the JVM is crashing as noted in OpenJDK bug list: https://bugs.openjdk.java.net/browse/JDK-8210457. This issue is affecting our ability to support those versions. 1. What are the changes cause this crash to appear? (maybe link to a commit). 2. Is this crash will be fixed for version 10, 11 and 12? (I saw the fix is intended for JDK 13). Thanks, Netanel. From tobias.hartmann at oracle.com Thu Mar 7 07:27:29 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 7 Mar 2019 08:27:29 +0100 Subject: [13] RFR(M): 8163511: Allocation of compile task fails with assert: "Leaking compilation tasks?" In-Reply-To: References: <8476ab17-665a-383a-8dc3-bf5fb69fc746@oracle.com> <9159da86-8330-24b6-b783-6dbfdad2fe87@oracle.com> <04328CE3-6CCE-4009-AD39-6DFE7A66C7B7@oracle.com> Message-ID: <1e4f6309-3503-8569-4f2b-82ec9b30094b@oracle.com> Thanks again, Vladimir. Best regards, Tobias On 06.03.19 19:27, Vladimir Kozlov wrote: > Yes, this looks good to me too. > > Thanks, > Vladimir > > On 3/6/19 3:14 AM, Erik ?sterlund wrote: >> Hi Tobias, >> >> On 2019-03-06 11:50, Tobias Hartmann wrote: >>> Hi Erik, >>> >>> On 05.03.19 09:09, Erik Osterlund wrote: >>>> Liveness of objects only change in safepoints. So if you are in the vm thread state, and ask if >>>> a holder is alive, its result is stable. >>>> >>>> With concurrent class unloading, marking terminates in a safepoint. At that snapshot in time, it >>>> is determined what is alive (strongly and through finalizers) and dead (although it might be >>>> lazily evaluated during concurrent execution). When that safepoint is released, that view will >>>> be consistent. >>>> >>>> As there are seemingly no safepoint checks in this code, all liveness information will be >>>> consistent and none of the imagined races exist with concurrent class unloading. It behaves as >>>> if it was all done in a safepoint, even though it was not. >>>> >>>> The key is that the klass_holder liveness (as well as the jweaks you use) is determined using a >>>> phantom load via the access API. That phantom load gives consistent results in the vm thread >>>> state as-if reference processing was done in a safepoint. The phantom loads can lazily compute >>>> what the value would have been, had reference processing been done in a safepoint, as it has >>>> stable liveness information from that safepoint. Then concurrent reference processing races for >>>> producing the same values. >>>> >>>> Basically there is nothing to worry about here as far as I can see. Looks good. >>> Thanks a lot for the thorough explanation and the review. >>> >>>> Although freeing the jweaks in one place instead of two would make the code easier to follow. Up >>>> to you though and I don?t need another webrev for that. >>> I could only factor the freeing into a separate method and call it from CompileTask::free() and >>> CompileTask::select_for_compilation(). But I don't think that would make the code easier to read. >> >> Ah, okay. >> >>> Relying on the assumption that either both _method_holder and _hot_method_holder are weak references >>> or both are strong, I've slightly refactored the code in CompileTask::free(): >>> >>> http://cr.openjdk.java.net/~thartmann/8163511/webrev.02 >> >> The new webrev looks good to me. >> >> Thanks, >> /Erik >> >>> >>> Best regards, >>> Tobias >> From nati.zelig at overopshq.com Thu Mar 7 08:09:08 2019 From: nati.zelig at overopshq.com (Nati Zelig) Date: Thu, 7 Mar 2019 10:09:08 +0200 Subject: JDK-8210457 : JVM crash in ResolvedMethodTable::add_method(Handle) Message-ID: Hello all, The company I work for is developing a solution for JVM's, as a native agent which performing redefinition of some of the classes loaded. Since JDK 10 (Tested up to 11), the JVM is crashing as noted in OpenJDK bug list: https://bugs.openjdk.java.net/browse/JDK-8210457. This issue is affecting our ability to support those versions. 1. What are the changes cause this crash to appear? (maybe link to a commit). 2. Is this crash will be fixed for version 10, 11 and 12? (I saw the fix is intended for JDK 13). Thanks, Netanel. From thomas.stuefe at gmail.com Thu Mar 7 11:37:27 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 7 Mar 2019 12:37:27 +0100 Subject: RFR(S): 8219650: [Testbug] Fix potential crashes in new test hotspot gtest "test_print_hex_dump" In-Reply-To: References: <19194f67-ed19-0031-e1a9-e79b56a777f0@redhat.com> <04555104-3ad3-0135-e45c-2ae4e0a621bd@redhat.com> <33b1d1bb-3837-40e3-08f7-313be235c213@redhat.com> <4a6a31b7-2b51-76cf-c6ad-9891b908926f@redhat.com> Message-ID: Oops. Pushed and forgot to change the macro to AIX_ONLY... sorry for that. Can you live with that or should I make a follow up patch? On Wed, Mar 6, 2019 at 12:49 PM Thomas St?fe wrote: > > > On Wed, Mar 6, 2019 at 12:44 PM Aleksey Shipilev wrote: > >> On 3/6/19 12:27 PM, Thomas St?fe wrote: >> > Okay next try: >> > >> > >> http://cr.openjdk.java.net/~stuefe/webrevs/8219650-shade-print-hex/webrev.01/webrev/ >> >> Looks okay. Boy, what a ride. Thanks for doing this! >> > > I was feeling guilty since I talked you into this... :) > > >> >> *) I have no strong opinion about this, but maybe we would universally >> call that non-zero address? >> It would probably require more work figuring out different bitness, so >> this one is also good. Maybe >> one-line it with AIX_ONLY(...) NOT_AIX(0)? >> >> 188 // On AIX, zero page is readable. >> 189 address unreadable = >> 190 #ifdef AIX >> 191 (address) 0xFFFFFFFFFFFF0000ULL; >> 192 #else >> 193 (address) 0 >> 194 #endif >> 195 ; >> >> > Lets keep it AIX specific, otherwise in years from now questions come why > we did not use 0. Also, I did not feel like checking if that address > happens to be readable on all platforms. Then, there is 32bit... > > But I'll use the AIX_ONLY macro, that is better. > > ..Thomas > > > >> -Aleksey >> >> From shade at redhat.com Thu Mar 7 11:44:50 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 7 Mar 2019 12:44:50 +0100 Subject: RFR(S): 8219650: [Testbug] Fix potential crashes in new test hotspot gtest "test_print_hex_dump" In-Reply-To: References: <19194f67-ed19-0031-e1a9-e79b56a777f0@redhat.com> <04555104-3ad3-0135-e45c-2ae4e0a621bd@redhat.com> <33b1d1bb-3837-40e3-08f7-313be235c213@redhat.com> <4a6a31b7-2b51-76cf-c6ad-9891b908926f@redhat.com> Message-ID: On 3/7/19 12:37 PM, Thomas St?fe wrote: > Oops. Pushed and forgot to change the macro to AIX_ONLY... sorry for that. Can you live with that or > should I make a follow up patch? I can live with that. -Aleksey From sgehwolf at redhat.com Thu Mar 7 17:23:05 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 07 Mar 2019 18:23:05 +0100 Subject: RFR: 8220283: ZGC fails to build on GCC 4.4.7: ATTRIBUTE_ALIGNED compatibility issue Message-ID: Hi, In order to be able to build recent JDKs on CentOS 6/RHEL 6 we've discovered that a few fixes would be needed in the codebase. One such instance is being caused by ZGC's use of ATTRIBUTE_ALIGNED: #define ATTRIBUTE_ALIGNED(x) __attribute__((aligned(x+0))) Used as: ZMarkStackList _published ATTRIBUTE_ALIGNED(ZCacheLineSize); This trips up GCC 4.4.7 with: src/hotspot/share/gc/z/zMarkStack.hpp:77: error: requested alignment is not a constant Although, 4.4.7 isn't a supported compiler version, the fix would be this one-liner: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/01/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8220283 Testing: tier1 tests on Linux x86_64, jdk/submit (ongoing) Thoughts? Thanks, Severin From shade at redhat.com Thu Mar 7 17:40:05 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 7 Mar 2019 18:40:05 +0100 Subject: RFR: 8220283: ZGC fails to build on GCC 4.4.7: ATTRIBUTE_ALIGNED compatibility issue In-Reply-To: References: Message-ID: <713dbb3e-805f-0d68-64c1-a30bb9167776@redhat.com> On 3/7/19 6:23 PM, Severin Gehwolf wrote: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/01/webrev/ Some background. After Severin reported this failure, I found this trick by pure chance, fiddling with godbolt: https://godbolt.org/z/OmVQ-B We should extend the "NOTE!" comment with this new knowledge, for instance: // NOTE! The "+0" below is a workaround for a known bug in older GCC versions // (known to fail with 4.6.0, fixed in 4.9.0). This bug affects systems such as // RedHat/Oracle Linux 7.5, which ships with GCC 4.8.5. For more details, see // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55382 and // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53017 // // GCC versions older than 4.6.4 would fail even with "+0", and needs additional // cast to typeof(x) to work around the similar bug. // #define ATTRIBUTE_ALIGNED(x) __attribute__((aligned((typeof(x))x+0))) -Aleksey From sgehwolf at redhat.com Thu Mar 7 18:27:05 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 07 Mar 2019 19:27:05 +0100 Subject: RFR: 8220283: ZGC fails to build on GCC 4.4.7: ATTRIBUTE_ALIGNED compatibility issue In-Reply-To: <713dbb3e-805f-0d68-64c1-a30bb9167776@redhat.com> References: <713dbb3e-805f-0d68-64c1-a30bb9167776@redhat.com> Message-ID: <37fe8668e3497dcca90fa47280865a66d80d7027.camel@redhat.com> On Thu, 2019-03-07 at 18:40 +0100, Aleksey Shipilev wrote: > On 3/7/19 6:23 PM, Severin Gehwolf wrote: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/01/webrev/ > > Some background. After Severin reported this failure, I found this trick by pure chance, fiddling > with godbolt: https://godbolt.org/z/OmVQ-B > > We should extend the "NOTE!" comment with this new knowledge, for instance: > > // NOTE! The "+0" below is a workaround for a known bug in older GCC versions > // (known to fail with 4.6.0, fixed in 4.9.0). This bug affects systems such as > // RedHat/Oracle Linux 7.5, which ships with GCC 4.8.5. For more details, see > // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55382 and > // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53017 > // > // GCC versions older than 4.6.4 would fail even with "+0", and needs additional > // cast to typeof(x) to work around the similar bug. > // > #define ATTRIBUTE_ALIGNED(x) __attribute__((aligned((typeof(x))x+0))) Sure, makes sense. Thanks! Updated webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/02/webrev/ Could I get a second review for this, please? Thanks, Severin From kim.barrett at oracle.com Thu Mar 7 21:09:56 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 7 Mar 2019 16:09:56 -0500 Subject: RFR: 8220283: ZGC fails to build on GCC 4.4.7: ATTRIBUTE_ALIGNED compatibility issue In-Reply-To: <37fe8668e3497dcca90fa47280865a66d80d7027.camel@redhat.com> References: <713dbb3e-805f-0d68-64c1-a30bb9167776@redhat.com> <37fe8668e3497dcca90fa47280865a66d80d7027.camel@redhat.com> Message-ID: <89E8C9BA-5D7D-42B9-AA7F-F7CE9EBF92B2@oracle.com> > On Mar 7, 2019, at 1:27 PM, Severin Gehwolf wrote: > > On Thu, 2019-03-07 at 18:40 +0100, Aleksey Shipilev wrote: >> On 3/7/19 6:23 PM, Severin Gehwolf wrote: >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/01/webrev/ >> >> Some background. After Severin reported this failure, I found this trick by pure chance, fiddling >> with godbolt: https://godbolt.org/z/OmVQ-B >> >> We should extend the "NOTE!" comment with this new knowledge, for instance: >> >> // NOTE! The "+0" below is a workaround for a known bug in older GCC versions >> // (known to fail with 4.6.0, fixed in 4.9.0). This bug affects systems such as >> // RedHat/Oracle Linux 7.5, which ships with GCC 4.8.5. For more details, see >> // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55382 and >> // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53017 >> // >> // GCC versions older than 4.6.4 would fail even with "+0", and needs additional >> // cast to typeof(x) to work around the similar bug. >> // >> #define ATTRIBUTE_ALIGNED(x) __attribute__((aligned((typeof(x))x+0))) > > Sure, makes sense. Thanks! Updated webrev: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/02/webrev/ > > Could I get a second review for this, please? > > Thanks, > Severin Looks okay, but yuck! :( Reviewed. From sgehwolf at redhat.com Fri Mar 8 08:54:22 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 08 Mar 2019 09:54:22 +0100 Subject: RFR: 8220283: ZGC fails to build on GCC 4.4.7: ATTRIBUTE_ALIGNED compatibility issue In-Reply-To: <89E8C9BA-5D7D-42B9-AA7F-F7CE9EBF92B2@oracle.com> References: <713dbb3e-805f-0d68-64c1-a30bb9167776@redhat.com> <37fe8668e3497dcca90fa47280865a66d80d7027.camel@redhat.com> <89E8C9BA-5D7D-42B9-AA7F-F7CE9EBF92B2@oracle.com> Message-ID: <41f9a11e8b41cedf21520d927db8bf1860fa568e.camel@redhat.com> On Thu, 2019-03-07 at 16:09 -0500, Kim Barrett wrote: > > On Mar 7, 2019, at 1:27 PM, Severin Gehwolf wrote: > > > > On Thu, 2019-03-07 at 18:40 +0100, Aleksey Shipilev wrote: > > > On 3/7/19 6:23 PM, Severin Gehwolf wrote: > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/01/webrev/ > > > > > > Some background. After Severin reported this failure, I found this trick by pure chance, fiddling > > > with godbolt: https://godbolt.org/z/OmVQ-B > > > > > > We should extend the "NOTE!" comment with this new knowledge, for instance: > > > > > > // NOTE! The "+0" below is a workaround for a known bug in older GCC versions > > > // (known to fail with 4.6.0, fixed in 4.9.0). This bug affects systems such as > > > // RedHat/Oracle Linux 7.5, which ships with GCC 4.8.5. For more details, see > > > // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55382 and > > > // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53017 > > > // > > > // GCC versions older than 4.6.4 would fail even with "+0", and needs additional > > > // cast to typeof(x) to work around the similar bug. > > > // > > > #define ATTRIBUTE_ALIGNED(x) __attribute__((aligned((typeof(x))x+0))) > > > > Sure, makes sense. Thanks! Updated webrev: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220283/02/webrev/ > > > > Could I get a second review for this, please? > > > > Thanks, > > Severin > > Looks okay, but yuck! :( > > Reviewed. Thanks for the review, Kim! Cheers, Severin From aoqi at loongson.cn Fri Mar 8 11:09:53 2019 From: aoqi at loongson.cn (Ao Qi) Date: Fri, 8 Mar 2019 19:09:53 +0800 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr Message-ID: Hi, I found build is broken when configure with --with-jvm-features=-g1gc,-jfr. There are two problems: 1. g1BarrierSet.hpp is included when g1 is disabled. I think this include can be removed. 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef INCLUDE_G1GC, cardTable.hpp is missed. Bug: https://bugs.openjdk.java.net/browse/JDK-8220344 Webrev: http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ Tested: build test: linux-x86_64-{server, minimal, zero}-{fastdebug, release}, linux-x86_64-{server, minimal, zero}-{fastdebug, release}(configure with -g1gc,-jfr)? jtreg: linux-x86_64-server-release hotspot:tier1 Thanks, Ao Qi From thomas.schatzl at oracle.com Fri Mar 8 11:39:26 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 08 Mar 2019 12:39:26 +0100 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: References: Message-ID: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> Hi, On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: > Hi, > > I found build is broken when configure with > --with-jvm-features=-g1gc,-jfr. There are two problems: > > 1. g1BarrierSet.hpp is included when g1 is disabled. I think this > include can be removed. > 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef > INCLUDE_G1GC, cardTable.hpp is missed. (I noticed that too just yesterday when working on another change - thanks for fixing this). > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8220344 > > Webrev: > http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ > > looks good to me. Thomas From sgehwolf at redhat.com Fri Mar 8 13:03:26 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 08 Mar 2019 14:03:26 +0100 Subject: RFR[XS]: 8220353: [TESTBUG] TestRegisterRestoring uses SafepointALot without UnlockDiagnosticVMOptions Message-ID: Hi, Currently one test in tier1 tests fails for me: TestRegisterRestoring. It fails because its using -XX:+SafepointALot without -XX:+UnlockDiagnosticVMOptions. I assume SafepointALot being diagnostic might have been a recent change. Not sure. Anyway, here is a fix: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220353/01/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8220353 Testing: Test fails prior this fix. Passes after. I intend to push this change under the trivial rules, OK? Thoughts? Thanks, Severin From tobias.hartmann at oracle.com Fri Mar 8 13:10:41 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 8 Mar 2019 14:10:41 +0100 Subject: RFR[XS]: 8220353: [TESTBUG] TestRegisterRestoring uses SafepointALot without UnlockDiagnosticVMOptions In-Reply-To: References: Message-ID: <2fef6e4d-a5aa-312a-0b99-b8c83de7b138@oracle.com> Hi Severin, yes, SafepointALot was made diagnostic by 8219584 [1]. The fix looks good and trivial to me but I think you can then remove -XX:+IgnoreUnrecognizedVMOptions which was there to ignore SafepointALot in product builds when it still was a debug option. Best regards, Tobias [1] http://hg.openjdk.java.net/jdk/jdk/rev/feea57b38a1c#l3.7 On 08.03.19 14:03, Severin Gehwolf wrote: > Hi, > > Currently one test in tier1 tests fails for me: TestRegisterRestoring. > It fails because its using -XX:+SafepointALot without > -XX:+UnlockDiagnosticVMOptions. I assume SafepointALot being diagnostic > might have been a recent change. Not sure. Anyway, here is a fix: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220353/01/webrev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220353 > > Testing: Test fails prior this fix. Passes after. > > I intend to push this change under the trivial rules, OK? > > Thoughts? > > Thanks, > Severin > > From shade at redhat.com Fri Mar 8 13:13:05 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 8 Mar 2019 14:13:05 +0100 Subject: RFR[XS]: 8220353: [TESTBUG] TestRegisterRestoring uses SafepointALot without UnlockDiagnosticVMOptions In-Reply-To: References: Message-ID: On 3/8/19 2:03 PM, Severin Gehwolf wrote: > Currently one test in tier1 tests fails for me: TestRegisterRestoring. > It fails because its using -XX:+SafepointALot without > -XX:+UnlockDiagnosticVMOptions. I assume SafepointALot being diagnostic > might have been a recent change. Not sure. It was introduced by recent push of JDK-8219584, I linked the bug. I see other tests that use SafepointAtLot without unlocking it, are those failing too? runtime/handshake/HandshakeTransitionTest.java It has @requires debug, which probably unlocks the flag automatically. > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220353/01/webrev/ Looks good and trivial to me. > I intend to push this change under the trivial rules, OK? Yes. -Aleksey From sgehwolf at redhat.com Fri Mar 8 13:35:39 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 08 Mar 2019 14:35:39 +0100 Subject: RFR[XS]: 8220353: [TESTBUG] TestRegisterRestoring uses SafepointALot without UnlockDiagnosticVMOptions In-Reply-To: <2fef6e4d-a5aa-312a-0b99-b8c83de7b138@oracle.com> References: <2fef6e4d-a5aa-312a-0b99-b8c83de7b138@oracle.com> Message-ID: <8186a8afdbdf7525fc552443c27bad4d6e3b891e.camel@redhat.com> Hi Tobias, On Fri, 2019-03-08 at 14:10 +0100, Tobias Hartmann wrote: > Hi Severin, > > yes, SafepointALot was made diagnostic by 8219584 [1]. The fix looks good and trivial to me but I > think you can then remove -XX:+IgnoreUnrecognizedVMOptions which was there to ignore SafepointALot > in product builds when it still was a debug option. Thanks. I've removed -XX:+IgnoreUnrecognizedVMOptions: http://hg.openjdk.java.net/jdk/jdk/rev/65a9f034954f Test passes for me without it and added UnlockDiagnosticVMOptions. Thanks, Severin > Best regards, > Tobias > > [1] http://hg.openjdk.java.net/jdk/jdk/rev/feea57b38a1c#l3.7 > > On 08.03.19 14:03, Severin Gehwolf wrote: > > Hi, > > > > Currently one test in tier1 tests fails for me: TestRegisterRestoring. > > It fails because its using -XX:+SafepointALot without > > -XX:+UnlockDiagnosticVMOptions. I assume SafepointALot being diagnostic > > might have been a recent change. Not sure. Anyway, here is a fix: > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220353/01/webrev/ > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220353 > > > > Testing: Test fails prior this fix. Passes after. > > > > I intend to push this change under the trivial rules, OK? > > > > Thoughts? > > > > Thanks, > > Severin > > > > From sgehwolf at redhat.com Fri Mar 8 13:40:13 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 08 Mar 2019 14:40:13 +0100 Subject: RFR[XS]: 8220353: [TESTBUG] TestRegisterRestoring uses SafepointALot without UnlockDiagnosticVMOptions In-Reply-To: References: Message-ID: Hi Aleksey, On Fri, 2019-03-08 at 14:13 +0100, Aleksey Shipilev wrote: > On 3/8/19 2:03 PM, Severin Gehwolf wrote: > > Currently one test in tier1 tests fails for me: TestRegisterRestoring. > > It fails because its using -XX:+SafepointALot without > > -XX:+UnlockDiagnosticVMOptions. I assume SafepointALot being diagnostic > > might have been a recent change. Not sure. > > It was introduced by recent push of JDK-8219584, I linked the bug. > > I see other tests that use SafepointAtLot without unlocking it, are those failing too? > runtime/handshake/HandshakeTransitionTest.java > > It has @requires debug, which probably unlocks the flag automatically. These tests which all use SavepointALot have UnlockDiagnosticVMOptions already: test/hotspot/jtreg/runtime/handshake/HandshakeTransitionTest.java test/hotspot/jtreg/compiler/c2/Test8004741.java test/hotspot/jtreg/compiler/runtime/Test7196199.java test/hotspot/jtreg/runtime/Safepoint/TestAbortVMOnSafepointTimeout.java > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220353/01/webrev/ > > Looks good and trivial to me. Thanks! Pushed: http://hg.openjdk.java.net/jdk/jdk/rev/65a9f034954f Cheers, Severin From thomas.schatzl at oracle.com Fri Mar 8 13:51:20 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 08 Mar 2019 14:51:20 +0100 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) Message-ID: Hi all, can I have reviews for this change the replaces the use of the JNI data type jbyte in the CardTable class by uint8_t via the custom CardValue typedef? This cleanup stems from trying to get rid of JNI data types in VM/GC code; formerly the main reason to use them here has been that the Atomic operations were not available for e.g. uint8_t, but that has been fixed for a long time. While the change itself is a no-op in practice, and testing showed so, I would like non-Oracle platform maintainers to try to compile the change and report problems because of lack of build platforms. CR: https://bugs.openjdk.java.net/browse/JDK-8220301 Webrev: http://cr.openjdk.java.net/~tschatzl/8220301/webrev/ Testing: hs-tier1-5 Thanks, Thomas From tobias.hartmann at oracle.com Fri Mar 8 14:16:36 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 8 Mar 2019 15:16:36 +0100 Subject: [13] RFR(S): 8220341: Class redefinition fails with assert(!is_unloaded()) failed: unloaded method on the stack In-Reply-To: <4ed958d5-729b-9d2b-8d8a-bcfd7bbdea40@oracle.com> References: <4ed958d5-729b-9d2b-8d8a-bcfd7bbdea40@oracle.com> Message-ID: <585eadca-4f71-6cdf-2423-c76329392871@oracle.com> [Switching to hotspot-dev] On 08.03.19 14:34, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8220341 > http://cr.openjdk.java.net/~thartmann/8220341/webrev.00/ > > The assert added by 8163511 [1] is wrong and should be replaced by a check because class > redefinition can encounter unloaded methods in the compile queue during marking (they will be > removed from the queue later). > > Verified fix with failing tests at hs-tier6. Will run more tests. > > Thanks, > Tobias > > [1] https://bugs.openjdk.java.net/browse/JDK-8163511 > From robbin.ehn at oracle.com Fri Mar 8 15:24:38 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 8 Mar 2019 16:24:38 +0100 Subject: RFR(m): 8220351: Cross-modifying code Message-ID: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Hi all, please review. Issue: https://bugs.openjdk.java.net/browse/JDK-8220351 Changeset: http://cr.openjdk.java.net/~rehn/8220351/webrev/ After a JavaThread have been in a safepoint/(handshake) safe state it can start executing updated code. E.g. an oop in the instruction stream can have been updated. Most hardware's require a barrier or that the code cross modified is far away to guarantee that the thread executing the updated instruction stream sees the modification. What far away is and how far an update instruction stream is from a safepoint safe state is not clear. To be compliant with those architectures an instruction stream barrier must be added when leaving the safepoint safe state. There may be crashes today due to this missing barrier. A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe state closer to a nmethod can increase changes of a crash. A few benchmarks will see a performance regression with the extra serializing, such as Octane-Crypto -5% (x86). With a much more elaborate fix, such as changing the local poll to have more than two states (armed/disarmed), it would be possible to only emit such instruction stream barrier when the poll have been armed while the thread was safepoint safe. Passes t1-3 on our platforms, and test built what I can. Thanks, Robbin From aoqi at loongson.cn Fri Mar 8 15:27:18 2019 From: aoqi at loongson.cn (Ao Qi) Date: Fri, 8 Mar 2019 23:27:18 +0800 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> Message-ID: On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl wrote: > > Hi, > > On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: > > Hi, > > > > I found build is broken when configure with > > --with-jvm-features=-g1gc,-jfr. There are two problems: > > > > 1. g1BarrierSet.hpp is included when g1 is disabled. I think this > > include can be removed. > > 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef > > INCLUDE_G1GC, cardTable.hpp is missed. > > (I noticed that too just yesterday when working on another change - > thanks for fixing this). > ;) It seems that build still fails when using --with-jvm-features=-g1gc (without -jfr). There is already a issue [1] for this. > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8220344 > > > > Webrev: > > http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ > > > > > > looks good to me. Thanks Thomas. Could you (or someone) help me to sponsor this? Thanks, Ao Qi [1] https://bugs.openjdk.java.net/browse/JDK-8209802 > > Thomas > From shade at redhat.com Fri Mar 8 16:58:43 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 8 Mar 2019 17:58:43 +0100 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: References: Message-ID: <3c498cf5-19bf-57dc-ca17-99ca25433d83@redhat.com> On 3/8/19 2:51 PM, Thomas Schatzl wrote: > While the change itself is a no-op in practice, and testing showed so, > I would like non-Oracle platform maintainers to try to compile the > change and report problems because of lack of build platforms. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8220301 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8220301/webrev/ Not sure why do we need to drag CardTable::CardValue all over the place, and then STATIC_ASSERT its size, when we could "just" do uint8_t everywhere? Anyway, these configs build fine: linux-aarch64-server-fastdebug linux-arm-server-fastdebug linux-s390x-server-fastdebug linux-x86-server-fastdebug linux-x86_64-minimal-fastdebug linux-x86_64-zero-fastdebug And linux-ppc64le-server-fastdebug fails with: In file included from /home/shade/jdk-jdk/src/hotspot/share/gc/shared/cardTableBarrierSetAssembler.hpp:30:0, from /home/shade/jdk-jdk/src/hotspot/share/gc/shared/cardTableBarrierSet.cpp:26: /home/shade/jdk-jdk/src/hotspot/cpu/ppc/gc/shared/cardTableBarrierSetAssembler_ppc.hpp:37:47: error: ?CardTable? has not been declared void card_table_write(MacroAssembler* masm, CardTable::CardValue* byte_map_base, Register tmp, Register obj); ^~~~~~~~~ It passes with this addendum: diff -r 1ee9ea03b3a5 src/hotspot/share/gc/shared/cardTableBarrierSet.cpp --- a/src/hotspot/share/gc/shared/cardTableBarrierSet.cpp Fri Mar 08 16:44:31 2019 +0100 +++ b/src/hotspot/share/gc/shared/cardTableBarrierSet.cpp Fri Mar 08 17:42:10 2019 +0100 @@ -23,6 +23,7 @@ */ #include "precompiled.hpp" +#include "gc/shared/cardTable.hpp" #include "gc/shared/cardTableBarrierSetAssembler.hpp" #include "gc/shared/cardTableBarrierSet.inline.hpp" #include "gc/shared/collectedHeap.hpp" -Aleksey From aoqi at loongson.cn Sat Mar 9 08:56:33 2019 From: aoqi at loongson.cn (Ao Qi) Date: Sat, 9 Mar 2019 16:56:33 +0800 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> Message-ID: Hi, Could someone help to push this fix? Thanks, Ao Qi On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl wrote: > > Hi, > > On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: > > Hi, > > > > I found build is broken when configure with > > --with-jvm-features=-g1gc,-jfr. There are two problems: > > > > 1. g1BarrierSet.hpp is included when g1 is disabled. I think this > > include can be removed. > > 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef > > INCLUDE_G1GC, cardTable.hpp is missed. > > (I noticed that too just yesterday when working on another change - > thanks for fixing this). > > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8220344 > > > > Webrev: > > http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ > > > > > > looks good to me. > > Thomas > From kim.barrett at oracle.com Sat Mar 9 21:47:47 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 9 Mar 2019 16:47:47 -0500 Subject: RFR: 8217576: C1 atomic access handlers use incorrect decorators Message-ID: <721DCA80-9B21-4196-AE88-382AC94B5D5F@oracle.com> Please review this fix of decorator defaulting in C1 atomic access generators. As noted in the bug, C1 presently ignores the decorators and only generates fully barriered operations, so the incorrect decorators haven't been affecting code generation. CR: https://bugs.openjdk.java.net/browse/JDK-8217576 Webrev: http://cr.openjdk.java.net/~kbarrett/8217576/open.00/ Testing: mach5 tier1. From kim.barrett at oracle.com Sun Mar 10 02:45:34 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 9 Mar 2019 21:45:34 -0500 Subject: RFR(trivial): 8217417: Decorator name typo: C2_TIGHLY_COUPLED_ALLOC Message-ID: <30C106EA-4214-4BE6-ADCE-69BE9D502038@oracle.com> Please review this trivial fix of a typo in a decorator name that has propagated a bit. CR: https://bugs.openjdk.java.net/browse/JDK-8217417 Webrev: http://cr.openjdk.java.net/~kbarrett/8217417/open.00/ Testing: mach5 build on Oracle supported platforms. local (linux-x64-debug) build with shenandoah enabled, since on of the propagated typos is there. From thomas.schatzl at oracle.com Sun Mar 10 11:14:42 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sun, 10 Mar 2019 12:14:42 +0100 Subject: RFR(trivial): 8217417: Decorator name typo: C2_TIGHLY_COUPLED_ALLOC In-Reply-To: <30C106EA-4214-4BE6-ADCE-69BE9D502038@oracle.com> References: <30C106EA-4214-4BE6-ADCE-69BE9D502038@oracle.com> Message-ID: <9ddd24c04681a21560dcdeeed43ab41de3a793d1.camel@oracle.com> Hi, On Sat, 2019-03-09 at 21:45 -0500, Kim Barrett wrote: > Please review this trivial fix of a typo in a decorator name that has > propagated a bit. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8217417 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8217417/open.00/ > > Testing: > mach5 build on Oracle supported platforms. > local (linux-x64-debug) build with shenandoah enabled, since on of > the propagated typos is there. > looks good and trivial. Ship it. Thomas From david.holmes at oracle.com Mon Mar 11 05:05:13 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Mar 2019 15:05:13 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: Hi Robbin, Sorry but I find it hard to understand how we suddenly need to sprinkle instruction stream synchronization operations all around the VM. I can't even see how some of these locations relate to "code modification". Can you please explain in more detail exactly what the problem is. Thanks, David On 9/03/2019 1:24 am, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state it > can start > executing updated code. E.g. an oop in the instruction stream can have been > updated. > > Most hardware's require a barrier or that the code cross modified is far > away to > guarantee that the thread executing the updated instruction stream sees the > modification. > > What far away is and how far an update instruction stream is from a > safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier > must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a > safepoint safe > state closer to a nmethod can increase changes of a crash. > > A few benchmarks will see a performance regression with the extra > serializing, > such as Octane-Crypto -5% (x86). > > With a much more elaborate fix, such as changing the local poll to have > more > than two states (armed/disarmed), it would be possible to only emit such > instruction stream barrier when the poll have been armed while the > thread was > safepoint safe. > > Passes t1-3 on our platforms, and test built what I can. > > Thanks, Robbin From kim.barrett at oracle.com Mon Mar 11 05:56:24 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 11 Mar 2019 01:56:24 -0400 Subject: RFR(trivial): 8217417: Decorator name typo: C2_TIGHLY_COUPLED_ALLOC In-Reply-To: <9ddd24c04681a21560dcdeeed43ab41de3a793d1.camel@oracle.com> References: <30C106EA-4214-4BE6-ADCE-69BE9D502038@oracle.com> <9ddd24c04681a21560dcdeeed43ab41de3a793d1.camel@oracle.com> Message-ID: <26A6E7FC-4076-4B8A-B94B-199B228AB10B@oracle.com> > On Mar 10, 2019, at 7:14 AM, Thomas Schatzl wrote: > > Hi, > > On Sat, 2019-03-09 at 21:45 -0500, Kim Barrett wrote: >> Please review this trivial fix of a typo in a decorator name that has >> propagated a bit. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8217417 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8217417/open.00/ >> >> Testing: >> mach5 build on Oracle supported platforms. >> local (linux-x64-debug) build with shenandoah enabled, since on of >> the propagated typos is there. >> > > looks good and trivial. Ship it. > > Thomas Thanks. From david.holmes at oracle.com Mon Mar 11 07:22:49 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Mar 2019 17:22:49 +1000 Subject: RFR: 8217576: C1 atomic access handlers use incorrect decorators In-Reply-To: <721DCA80-9B21-4196-AE88-382AC94B5D5F@oracle.com> References: <721DCA80-9B21-4196-AE88-382AC94B5D5F@oracle.com> Message-ID: Hi Kim, On 10/03/2019 7:47 am, Kim Barrett wrote: > Please review this fix of decorator defaulting in C1 atomic access > generators. As noted in the bug, C1 presently ignores the decorators > and only generates fully barriered operations, so the incorrect > decorators haven't been affecting code generation. So by fixing this and potentially using weaker memory ordering we could infact expose hitherto hidden bugs. :) Okay. The fix itself looks good. Thanks, David ----- > CR: > https://bugs.openjdk.java.net/browse/JDK-8217576 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8217576/open.00/ > > Testing: > mach5 tier1. > From david.holmes at oracle.com Mon Mar 11 07:35:40 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Mar 2019 17:35:40 +1000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> Message-ID: <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> On 7/03/2019 3:13 am, Baesken, Matthias wrote: > Hello , could I push the latest version of 8219241 ? Sorry based on your earlier email I thought you were looking at doing the restructuring. I was then on vacation most of last week. > A change that moves the coding more into platform/cpu - specific coding (if this is wanted) > could be done when bringing in the AIX virtualization related info in another patch which I plan to do . If that is imminent then okay - I don't like the code as it is. Thanks, David > Thanks, Matthias > > >> -----Original Message----- >> From: Baesken, Matthias >> Sent: Freitag, 1. M?rz 2019 09:09 >> To: 'David Holmes' ; 'hotspot- >> dev at openjdk.java.net' >> Subject: RE: RFR: 8219241: Provide basic virtualization related info in the >> hs_error file on linux/windows x86_64 >> >> Hi David , >> >> I think a clean way would be to have >> >> os::print_virtualization_info(outputStream* st) >> >> with platform and/or if needed platform/cpu - specific coding at the right >> places . >> >> I suggested something similar back then . >> >> This would be also a useful structure to bring in the AIX virtualization >> related info output . >> >> >> Regards, Matthias >> >> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Donnerstag, 28. Februar 2019 22:49 >>> To: Baesken, Matthias ; 'hotspot- >>> dev at openjdk.java.net' >>> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >>> hs_error file on linux/windows x86_64 >>> >>> Hi Matthias, >>> >>> Combining this with 8219746 I have to insist that we use a nice clean >>> OS-CPU abstraction for dealing with this and get rid of the platform >>> ifdefs in os::Linux::print_virtualization_info. What you've added in >>> VM_Version would seem to be the right kind of platform specific >>> functionality and it seems to me that you could get rid of >>> os::Linux::print_virtualization_info and just use >>> VM_Version::print_detected_virtualization in its place. >>> >>> Thanks, >>> David >>> >>> On 1/03/2019 2:04 am, Baesken, Matthias wrote: >>>> Hello, I provided a second version . >>>> The x86_64 assembler code was simplified (thanks to Martin for the >> hint) ; >>> additionally some comments were adjusted . >>>> >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.2/ >>>> >>>> >>>> Best regards , Matthias >>>> >>>> >>>> >>>> Hello , please review the following change. >>>> It provides basic virtualization related information in the hs_error file on >>> linux/windows x86_64. >>>> We use cpuid to get the information (detection of VMWare, Xen, KVM >> and >>> HyperV is included). >>>> >>>> Some info about how to detect the virtualization info can be found here : >>>> https://kb.vmware.com/s/article/1009458 >>>> >>>> >>>> >>>> (it is VMware related but can be applied for other virtualization >>> technologies as well) >>>> >>>> >>>> An overview of the ID strings can be found here as well : >>>> >>>> https://en.wikipedia.org/wiki/CPUID >>>> The following are known ID strings from virtual machines: ... >>>> >>>> >>>> >>>> bug/webrev : >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8219241 >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.1/ >>>> >>>> >>>> >>>> Thanks, Matthias >>>> From david.holmes at oracle.com Mon Mar 11 07:46:24 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Mar 2019 17:46:24 +1000 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> Message-ID: <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> Hi, On 9/03/2019 6:56 pm, Ao Qi wrote: > Hi, > > Could someone help to push this fix? You generally need two reviews for hotspot fixes (only one needs to be from a Reviewer). In this case as you are touching a shenandoah file the shenandoah folk should give the okay - cc'd Aleksey. Thanks, David > Thanks, > Ao Qi > > On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl wrote: >> >> Hi, >> >> On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: >>> Hi, >>> >>> I found build is broken when configure with >>> --with-jvm-features=-g1gc,-jfr. There are two problems: >>> >>> 1. g1BarrierSet.hpp is included when g1 is disabled. I think this >>> include can be removed. >>> 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef >>> INCLUDE_G1GC, cardTable.hpp is missed. >> >> (I noticed that too just yesterday when working on another change - >> thanks for fixing this). >> >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8220344 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ >>> >>> >> >> looks good to me. >> >> Thomas >> From robbin.ehn at oracle.com Mon Mar 11 08:08:39 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 11 Mar 2019 09:08:39 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> Hi David, For simplicity I'll talk about x86, which have instruction cache coherency. Immediate oop example, JavaThread going from native -> java. The JavaThread starts in native, thus safe, and any safepoint or handshake may be executed: store thread_state = _thread_in_native_trans (unsafe) StoreLoad // e.g. lock addl load thread_poll test thread_poll jcc slow_path store thread_state = _thread_in_Java ret return site: load // No data If the is updated we are not guaranteed to see it without mfence. The lock instruction, according to x86 manuals, have no effect on instruction stream. Even if the lock instruction would have effect on instruction stream, an on-going safepoint can update the and finishes between the StoreLoad and test: JavaThread: | VMThread StoreLoad | | update | disarm load thread_poll | This can also happen in the slow path (so there are is a bigger window also). By adding an instruction stream barrier (mfence) *after* the poll we are guaranteed to always see any updates to the instruction stream. Does this help? I notice I forgot to remove some out commented lines, I'll update the webrev in-place. Thanks, Robbin On 2019-03-11 06:05, David Holmes wrote: > Hi Robbin, > > Sorry but I find it hard to understand how we suddenly need to sprinkle > instruction stream synchronization operations all around the VM. I can't even > see how some of these locations relate to "code modification". Can you please > explain in more detail exactly what the problem is. > > Thanks, > David > > On 9/03/2019 1:24 am, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it can start >> executing updated code. E.g. an oop in the instruction stream can have been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is far away to >> guarantee that the thread executing the updated instruction stream sees the >> modification. >> >> What far away is and how far an update instruction stream is from a safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From aoqi at loongson.cn Mon Mar 11 08:11:29 2019 From: aoqi at loongson.cn (Ao Qi) Date: Mon, 11 Mar 2019 16:11:29 +0800 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> Message-ID: On Mon, Mar 11, 2019 at 3:46 PM David Holmes wrote: > > Hi, > > On 9/03/2019 6:56 pm, Ao Qi wrote: > > Hi, > > > > Could someone help to push this fix? > > You generally need two reviews for hotspot fixes (only one needs to be > from a Reviewer). In this case as you are touching a shenandoah file the > shenandoah folk should give the okay - cc'd Aleksey. > Sorry, I did not know this. Thanks David and I will wait to see if there is another review. Thanks, Ao Qi > Thanks, > David > > > Thanks, > > Ao Qi > > > > On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl wrote: > >> > >> Hi, > >> > >> On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: > >>> Hi, > >>> > >>> I found build is broken when configure with > >>> --with-jvm-features=-g1gc,-jfr. There are two problems: > >>> > >>> 1. g1BarrierSet.hpp is included when g1 is disabled. I think this > >>> include can be removed. > >>> 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef > >>> INCLUDE_G1GC, cardTable.hpp is missed. > >> > >> (I noticed that too just yesterday when working on another change - > >> thanks for fixing this). > >> > >>> > >>> Bug: > >>> https://bugs.openjdk.java.net/browse/JDK-8220344 > >>> > >>> Webrev: > >>> http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ > >>> > >>> > >> > >> looks good to me. > >> > >> Thomas > >> From erik.osterlund at oracle.com Mon Mar 11 08:52:30 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 11 Mar 2019 09:52:30 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <518a2092-f4e2-cd39-3eaa-8dcc3df9b04d@oracle.com> Hi Robbin, This looks good to me. I think you have managed to find all the transitions from inactive to active, w.r.t. the safepoint synchronizer. At least I am not aware of the existence of any others. The only thing I'm not quite sure about is the name of the new synchronization primitive (even though I was the one to suggest it... sorry my bad :C). The name "instruction_pipeline" seems a bit implementation specific about what HW architectural features need to be taken care of due to cross-modifying code, which may or may not apply to a given platform. Perhaps cross_modify_fence(), or something along those lines, would be better. That makes it more clear what we are protecting against, as opposed to what HW architectural features that might concern on a given platform. I don't need another webrev for the name change. Thanks, /Erik On 2019-03-08 16:24, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state it > can start > executing updated code. E.g. an oop in the instruction stream can have > been > updated. > > Most hardware's require a barrier or that the code cross modified is > far away to > guarantee that the thread executing the updated instruction stream > sees the > modification. > > What far away is and how far an update instruction stream is from a > safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier > must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a > safepoint safe > state closer to a nmethod can increase changes of a crash. > > A few benchmarks will see a performance regression with the extra > serializing, > such as Octane-Crypto -5% (x86). > > With a much more elaborate fix, such as changing the local poll to > have more > than two states (armed/disarmed), it would be possible to only emit such > instruction stream barrier when the poll have been armed while the > thread was > safepoint safe. > > Passes t1-3 on our platforms, and test built what I can. > > Thanks, Robbin From tobias.hartmann at oracle.com Mon Mar 11 08:55:04 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 11 Mar 2019 09:55:04 +0100 Subject: [13] RFR(S): 8220341: Class redefinition fails with assert(!is_unloaded()) failed: unloaded method on the stack In-Reply-To: References: <4ed958d5-729b-9d2b-8d8a-bcfd7bbdea40@oracle.com> <585eadca-4f71-6cdf-2423-c76329392871@oracle.com> Message-ID: <86ddff52-972e-9181-61f7-55aabb04a206@oracle.com> Thanks Erik. Best regards, Tobias On 11.03.19 09:54, Erik ?sterlund wrote: > Hi Tobias, > > Looks good. > > Thanks, > /Erik > > On 2019-03-08 15:16, Tobias Hartmann wrote: >> [Switching to hotspot-dev] >> >> On 08.03.19 14:34, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following patch: >>> https://bugs.openjdk.java.net/browse/JDK-8220341 >>> http://cr.openjdk.java.net/~thartmann/8220341/webrev.00/ >>> >>> The assert added by 8163511 [1] is wrong and should be replaced by a check because class >>> redefinition can encounter unloaded methods in the compile queue during marking (they will be >>> removed from the queue later). >>> >>> Verified fix with failing tests at hs-tier6. Will run more tests. >>> >>> Thanks, >>> Tobias >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8163511 >>> > From erik.osterlund at oracle.com Mon Mar 11 08:54:15 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 11 Mar 2019 09:54:15 +0100 Subject: [13] RFR(S): 8220341: Class redefinition fails with assert(!is_unloaded()) failed: unloaded method on the stack In-Reply-To: <585eadca-4f71-6cdf-2423-c76329392871@oracle.com> References: <4ed958d5-729b-9d2b-8d8a-bcfd7bbdea40@oracle.com> <585eadca-4f71-6cdf-2423-c76329392871@oracle.com> Message-ID: Hi Tobias, Looks good. Thanks, /Erik On 2019-03-08 15:16, Tobias Hartmann wrote: > [Switching to hotspot-dev] > > On 08.03.19 14:34, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8220341 >> http://cr.openjdk.java.net/~thartmann/8220341/webrev.00/ >> >> The assert added by 8163511 [1] is wrong and should be replaced by a check because class >> redefinition can encounter unloaded methods in the compile queue during marking (they will be >> removed from the queue later). >> >> Verified fix with failing tests at hs-tier6. Will run more tests. >> >> Thanks, >> Tobias >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8163511 >> From robbin.ehn at oracle.com Mon Mar 11 08:56:07 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 11 Mar 2019 09:56:07 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <518a2092-f4e2-cd39-3eaa-8dcc3df9b04d@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <518a2092-f4e2-cd39-3eaa-8dcc3df9b04d@oracle.com> Message-ID: Hi Erik, On 2019-03-11 09:52, Erik ?sterlund wrote: > Hi Robbin, > > This looks good to me. I think you have managed to find all the transitions from > inactive to active, w.r.t. the safepoint synchronizer. At least I am not aware > of the existence of any others. > Thanks! > The only thing I'm not quite sure about is the name of the new synchronization > primitive (even though I was the one to suggest it... sorry my bad :C). The name > "instruction_pipeline" seems a bit implementation specific about what HW > architectural features need to be taken care of due to cross-modifying code, > which may or may not apply to a given platform. Perhaps cross_modify_fence(), or > something along those lines, would be better. That makes it more clear what we > are protecting against, as opposed to what HW architectural features that might > concern on a given platform. Yes sure! But I'll hold off and see if there any better suggestion than cross_modify_fence. /Robbin > > I don't need another webrev for the name change. > > Thanks, > /Erik > > On 2019-03-08 16:24, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it can start >> executing updated code. E.g. an oop in the instruction stream can have been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is far away to >> guarantee that the thread executing the updated instruction stream sees the >> modification. >> >> What far away is and how far an update instruction stream is from a safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin > From stefan.karlsson at oracle.com Mon Mar 11 08:57:57 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 11 Mar 2019 09:57:57 +0100 Subject: RFR: 8220411: Remove ScavengeRootsInCode=0 code Message-ID: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> Hi all, Please review this patch to remove code that checks and acts upon ScavengeRootsInCode=0. http://cr.openjdk.java.net/~stefank/8220411/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8220411 Since a while now, we have not supported -XX:ScavengeRootsInCode=0, and the code actively sets it to 1 if the user passes in 0. This patch removes the actual code, there's another RFE to deal with changing the argument handling to explicitly not support that the flag is set to 0. See: JDK-8144954. My main motivation for this patch is to get rid of some of the usages of CollectedHeap::is_scavengable. The intention is that later patches will completely remove that function. Thanks, StefanK From aph at redhat.com Mon Mar 11 09:34:22 2019 From: aph at redhat.com (Andrew Haley) Date: Mon, 11 Mar 2019 09:34:22 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <92a34e1f-4dac-5e15-cbda-3b6bfe3b4538@redhat.com> On 3/8/19 3:24 PM, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state it can start > executing updated code. E.g. an oop in the instruction stream can have been > updated. > > Most hardware's require a barrier or that the code cross modified is far away to > guarantee that the thread executing the updated instruction stream sees the > modification. > > What far away is and how far an update instruction stream is from a safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe > state closer to a nmethod can increase changes of a crash. What is this hunk for? @@ -2558,12 +2558,9 @@ // VM thread changes sync state to synchronizing and suspends threads for GC. // Thread A is resumed to finish this native method, but doesn't block here since it // didn't see any synchronization is progress, and escapes. - __ movl(Address(r15_thread, JavaThread::thread_state_offset()), _thread_in_native_trans); - - // Force this write out before the read below - __ membar(Assembler::Membar_mask_bits( - Assembler::LoadLoad | Assembler::LoadStore | - Assembler::StoreLoad | Assembler::StoreStore)); + + __ lock(); + __ addl(Address(r15_thread, JavaThread::thread_state_offset()), 1 /* _thread_in_native_trans */); Label after_transition; -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From robbin.ehn at oracle.com Mon Mar 11 10:05:12 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 11 Mar 2019 11:05:12 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <92a34e1f-4dac-5e15-cbda-3b6bfe3b4538@redhat.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <92a34e1f-4dac-5e15-cbda-3b6bfe3b4538@redhat.com> Message-ID: > What is this hunk for? > > @@ -2558,12 +2558,9 @@ > // VM thread changes sync state to synchronizing and suspends threads for GC. > // Thread A is resumed to finish this native method, but doesn't block here since it > // didn't see any synchronization is progress, and escapes. > - __ movl(Address(r15_thread, JavaThread::thread_state_offset()), _thread_in_native_trans); > - > - // Force this write out before the read below > - __ membar(Assembler::Membar_mask_bits( > - Assembler::LoadLoad | Assembler::LoadStore | > - Assembler::StoreLoad | Assembler::StoreStore)); > + > + __ lock(); > + __ addl(Address(r15_thread, JavaThread::thread_state_offset()), 1 /* _thread_in_native_trans */); > > Label after_transition; > Yes, a bit unrelated. I thought: movl thread_state, native_trans lock addl rsp-x, 0 /* dummy */ Seemed a bit silly when we could do: lock addl thread_state, 1 If nothing else saves an instruction. (assuming cache-lines are equally contended) Shall I remove that change ? Thanks, Robbin From aph at redhat.com Mon Mar 11 10:13:50 2019 From: aph at redhat.com (Andrew Haley) Date: Mon, 11 Mar 2019 10:13:50 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <92a34e1f-4dac-5e15-cbda-3b6bfe3b4538@redhat.com> Message-ID: On 3/11/19 10:05 AM, Robbin Ehn wrote: >> What is this hunk for? >> >> @@ -2558,12 +2558,9 @@ >> // VM thread changes sync state to synchronizing and suspends threads for GC. >> // Thread A is resumed to finish this native method, but doesn't block here since it >> // didn't see any synchronization is progress, and escapes. >> - __ movl(Address(r15_thread, JavaThread::thread_state_offset()), _thread_in_native_trans); >> - >> - // Force this write out before the read below >> - __ membar(Assembler::Membar_mask_bits( >> - Assembler::LoadLoad | Assembler::LoadStore | >> - Assembler::StoreLoad | Assembler::StoreStore)); >> + >> + __ lock(); >> + __ addl(Address(r15_thread, JavaThread::thread_state_offset()), 1 /* _thread_in_native_trans */); >> >> Label after_transition; >> > > Yes, a bit unrelated. > I thought: > movl thread_state, native_trans > lock addl rsp-x, 0 /* dummy */ > > Seemed a bit silly when we could do: > lock addl thread_state, 1 > > If nothing else saves an instruction. (assuming cache-lines are equally contended) > > Shall I remove that change ? No, it's fine. It was just rather baffling when i was trying to understand the patch. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From stefan.karlsson at oracle.com Mon Mar 11 10:26:14 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 11 Mar 2019 11:26:14 +0100 Subject: RFR: 8220411: Remove ScavengeRootsInCode=0 code In-Reply-To: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> References: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> Message-ID: <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> Updated webrev after some offline feedback from Erik?: http://cr.openjdk.java.net/~stefank/8220411/webrev.02.delta http://cr.openjdk.java.net/~stefank/8220411/webrev.02 - I've removed can_be_constant and updated the code where it was used. - Removed and replaced check_oop with oopDesc::is_oop_or_null checks. Thanks, StefanK On 2019-03-11 09:57, Stefan Karlsson wrote: > Hi all, > > Please review this patch to remove code that checks and acts upon > ScavengeRootsInCode=0. > > http://cr.openjdk.java.net/~stefank/8220411/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8220411 > > Since a while now, we have not supported -XX:ScavengeRootsInCode=0, and > the code actively sets it to 1 if the user passes in 0. This patch > removes the actual code, there's another RFE to deal with changing the > argument handling to explicitly not support that the flag is set to 0. > See: JDK-8144954. > > My main motivation for this patch is to get rid of some of the usages of > CollectedHeap::is_scavengable. The intention is that later patches will > completely remove that function. > > Thanks, > StefanK From aph at redhat.com Mon Mar 11 10:26:58 2019 From: aph at redhat.com (Andrew Haley) Date: Mon, 11 Mar 2019 10:26:58 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: On 3/8/19 3:24 PM, Robbin Ehn wrote: > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state > it can start executing updated code. E.g. an oop in the instruction > stream can have been updated. Hmm, interesting. I think it works on on AArch64 at present because the segfault trap we take is effectively a full synchronization operation. An AArch64 ISB (instruction synchronization barrier) invalidates the pipeline. It doesn't invalidate the instruction cache, which we'd need to do to see a relocated OOP. If an instruction is modified by some other thread we need to flush the local icache, but to do that we need to know which instructions have been changed. We could create a modification queue, but that seems rather elaborate. What is is that changes OOPs in the instructions during a handshake? Is it just garbage collection, or does it happen at other times too? We could simply move OOPs (and class metadata pointers?) into the constant pool. That would have some performance impact on in-order CPUs, but hopefully not much on out-of-order ones. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From tobias.hartmann at oracle.com Mon Mar 11 10:39:55 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 11 Mar 2019 11:39:55 +0100 Subject: RFR: 8220411: Remove ScavengeRootsInCode=0 code In-Reply-To: <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> References: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> Message-ID: <793f48ae-a64f-0f69-0be2-cb1b2bd6e29a@oracle.com> Hi Stefan, this looks good to me. Best regards, Tobias On 11.03.19 11:26, Stefan Karlsson wrote: > Updated webrev after some offline feedback from Erik?: > > ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02.delta > ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02 > > - I've removed can_be_constant and updated the code where it was used. > - Removed and replaced check_oop with oopDesc::is_oop_or_null checks. > > Thanks, > StefanK > > On 2019-03-11 09:57, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to remove code that checks and acts upon ScavengeRootsInCode=0. >> >> http://cr.openjdk.java.net/~stefank/8220411/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8220411 >> >> Since a while now, we have not supported -XX:ScavengeRootsInCode=0, and the code actively sets it >> to 1 if the user passes in 0. This patch removes the actual code, there's another RFE to deal with >> changing the argument handling to explicitly not support that the flag is set to 0. See: JDK-8144954. >> >> My main motivation for this patch is to get rid of some of the usages of >> CollectedHeap::is_scavengable. The intention is that later patches will completely remove that >> function. >> >> Thanks, >> StefanK From coleen.phillimore at oracle.com Mon Mar 11 13:19:25 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 11 Mar 2019 09:19:25 -0400 Subject: RFR: 8220411: Remove ScavengeRootsInCode=0 code In-Reply-To: <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> References: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> Message-ID: This is nice.? Shouldn't you remove ScavengeRootsInCode from globals.hpp?? As a diagnostic command, I think it goes into the table in arguments.cpp? Thanks, Coleen On 3/11/19 6:26 AM, Stefan Karlsson wrote: > Updated webrev after some offline feedback from Erik?: > > ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02.delta > ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02 > > - I've removed can_be_constant and updated the code where it was used. > - Removed and replaced check_oop with oopDesc::is_oop_or_null checks. > > Thanks, > StefanK > > On 2019-03-11 09:57, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to remove code that checks and acts upon >> ScavengeRootsInCode=0. >> >> http://cr.openjdk.java.net/~stefank/8220411/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8220411 >> >> Since a while now, we have not supported -XX:ScavengeRootsInCode=0, >> and the code actively sets it to 1 if the user passes in 0. This >> patch removes the actual code, there's another RFE to deal with >> changing the argument handling to explicitly not support that the >> flag is set to 0. See: JDK-8144954. >> >> My main motivation for this patch is to get rid of some of the usages >> of CollectedHeap::is_scavengable. The intention is that later patches >> will completely remove that function. >> >> Thanks, >> StefanK From coleen.phillimore at oracle.com Mon Mar 11 13:29:37 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 11 Mar 2019 09:29:37 -0400 Subject: [13] RFR(S): 8220341: Class redefinition fails with assert(!is_unloaded()) failed: unloaded method on the stack In-Reply-To: <86ddff52-972e-9181-61f7-55aabb04a206@oracle.com> References: <4ed958d5-729b-9d2b-8d8a-bcfd7bbdea40@oracle.com> <585eadca-4f71-6cdf-2423-c76329392871@oracle.com> <86ddff52-972e-9181-61f7-55aabb04a206@oracle.com> Message-ID: Looks good, with a reminder not to touch the method and hot_method if the entry in the compile queue is unloaded because they may be deleted if they are old (ie. redefined) Methods. Thanks, Coleen On 3/11/19 4:55 AM, Tobias Hartmann wrote: > Thanks Erik. > > Best regards, > Tobias > > On 11.03.19 09:54, Erik ?sterlund wrote: >> Hi Tobias, >> >> Looks good. >> >> Thanks, >> /Erik >> >> On 2019-03-08 15:16, Tobias Hartmann wrote: >>> [Switching to hotspot-dev] >>> >>> On 08.03.19 14:34, Tobias Hartmann wrote: >>>> Hi, >>>> >>>> please review the following patch: >>>> https://bugs.openjdk.java.net/browse/JDK-8220341 >>>> http://cr.openjdk.java.net/~thartmann/8220341/webrev.00/ >>>> >>>> The assert added by 8163511 [1] is wrong and should be replaced by a check because class >>>> redefinition can encounter unloaded methods in the compile queue during marking (they will be >>>> removed from the queue later). >>>> >>>> Verified fix with failing tests at hs-tier6. Will run more tests. >>>> >>>> Thanks, >>>> Tobias >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8163511 >>>> From stefan.karlsson at oracle.com Mon Mar 11 12:02:52 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 11 Mar 2019 13:02:52 +0100 Subject: RFR: 8220411: Remove ScavengeRootsInCode=0 code In-Reply-To: <793f48ae-a64f-0f69-0be2-cb1b2bd6e29a@oracle.com> References: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> <793f48ae-a64f-0f69-0be2-cb1b2bd6e29a@oracle.com> Message-ID: Thanks, Tobias. StefanK On 2019-03-11 11:39, Tobias Hartmann wrote: > Hi Stefan, > > this looks good to me. > > Best regards, > Tobias > > On 11.03.19 11:26, Stefan Karlsson wrote: >> Updated webrev after some offline feedback from Erik?: >> >> ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02.delta >> ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02 >> >> - I've removed can_be_constant and updated the code where it was used. >> - Removed and replaced check_oop with oopDesc::is_oop_or_null checks. >> >> Thanks, >> StefanK >> >> On 2019-03-11 09:57, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to remove code that checks and acts upon ScavengeRootsInCode=0. >>> >>> http://cr.openjdk.java.net/~stefank/8220411/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8220411 >>> >>> Since a while now, we have not supported -XX:ScavengeRootsInCode=0, and the code actively sets it >>> to 1 if the user passes in 0. This patch removes the actual code, there's another RFE to deal with >>> changing the argument handling to explicitly not support that the flag is set to 0. See: JDK-8144954. >>> >>> My main motivation for this patch is to get rid of some of the usages of >>> CollectedHeap::is_scavengable. The intention is that later patches will completely remove that >>> function. >>> >>> Thanks, >>> StefanK From stefan.karlsson at oracle.com Mon Mar 11 12:06:40 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 11 Mar 2019 13:06:40 +0100 Subject: RFR: 8220411: Remove ScavengeRootsInCode=0 code In-Reply-To: References: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> Message-ID: <7b32db17-07a9-aef5-9b70-ca629b729087@oracle.com> Hi Coleen, Thanks for looking at this. On 2019-03-11 14:19, coleen.phillimore at oracle.com wrote: > > This is nice.? Shouldn't you remove ScavengeRootsInCode from > globals.hpp?? As a diagnostic command, I think it goes into the table in > arguments.cpp? I left that part to be handled by JDK-8144954. Thanks, StefanK > Thanks, > Coleen > > > On 3/11/19 6:26 AM, Stefan Karlsson wrote: >> Updated webrev after some offline feedback from Erik?: >> >> ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02.delta >> ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02 >> >> - I've removed can_be_constant and updated the code where it was used. >> - Removed and replaced check_oop with oopDesc::is_oop_or_null checks. >> >> Thanks, >> StefanK >> >> On 2019-03-11 09:57, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to remove code that checks and acts upon >>> ScavengeRootsInCode=0. >>> >>> http://cr.openjdk.java.net/~stefank/8220411/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8220411 >>> >>> Since a while now, we have not supported -XX:ScavengeRootsInCode=0, >>> and the code actively sets it to 1 if the user passes in 0. This >>> patch removes the actual code, there's another RFE to deal with >>> changing the argument handling to explicitly not support that the >>> flag is set to 0. See: JDK-8144954. >>> >>> My main motivation for this patch is to get rid of some of the usages >>> of CollectedHeap::is_scavengable. The intention is that later patches >>> will completely remove that function. >>> >>> Thanks, >>> StefanK > From coleen.phillimore at oracle.com Mon Mar 11 12:46:41 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 11 Mar 2019 08:46:41 -0400 Subject: RFR: 8220411: Remove ScavengeRootsInCode=0 code In-Reply-To: <7b32db17-07a9-aef5-9b70-ca629b729087@oracle.com> References: <9408aa49-48db-de8a-4f31-95a0a2151959@oracle.com> <3224aed0-6e76-9cfb-6418-765eeb048547@oracle.com> <7b32db17-07a9-aef5-9b70-ca629b729087@oracle.com> Message-ID: <77db397f-909e-c9b4-544c-0789184cdcd7@oracle.com> On 3/11/19 8:06 AM, Stefan Karlsson wrote: > Hi Coleen, > > Thanks for looking at this. > > On 2019-03-11 14:19, coleen.phillimore at oracle.com wrote: >> >> This is nice.? Shouldn't you remove ScavengeRootsInCode from >> globals.hpp?? As a diagnostic command, I think it goes into the table >> in arguments.cpp? > > I left that part to be handled by JDK-8144954. I see.? I guess I should have read the entire email instead of just the patch. thanks, Coleen > > Thanks, > StefanK > >> Thanks, >> Coleen >> >> >> On 3/11/19 6:26 AM, Stefan Karlsson wrote: >>> Updated webrev after some offline feedback from Erik?: >>> >>> ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02.delta >>> ?http://cr.openjdk.java.net/~stefank/8220411/webrev.02 >>> >>> - I've removed can_be_constant and updated the code where it was used. >>> - Removed and replaced check_oop with oopDesc::is_oop_or_null checks. >>> >>> Thanks, >>> StefanK >>> >>> On 2019-03-11 09:57, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to remove code that checks and acts upon >>>> ScavengeRootsInCode=0. >>>> >>>> http://cr.openjdk.java.net/~stefank/8220411/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8220411 >>>> >>>> Since a while now, we have not supported -XX:ScavengeRootsInCode=0, >>>> and the code actively sets it to 1 if the user passes in 0. This >>>> patch removes the actual code, there's another RFE to deal with >>>> changing the argument handling to explicitly not support that the >>>> flag is set to 0. See: JDK-8144954. >>>> >>>> My main motivation for this patch is to get rid of some of the >>>> usages of CollectedHeap::is_scavengable. The intention is that >>>> later patches will completely remove that function. >>>> >>>> Thanks, >>>> StefanK >> From robbin.ehn at oracle.com Mon Mar 11 12:55:40 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 11 Mar 2019 13:55:40 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> On 2019-03-11 11:26, Andrew Haley wrote: > On 3/8/19 3:24 PM, Robbin Ehn wrote: > >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state >> it can start executing updated code. E.g. an oop in the instruction >> stream can have been updated. > > Hmm, interesting. I think it works on on AArch64 at present because > the segfault trap we take is effectively a full synchronization > operation. > > An AArch64 ISB (instruction synchronization barrier) invalidates the > pipeline. It doesn't invalidate the instruction cache, which we'd need > to do to see a relocated OOP. If an instruction is modified by some > other thread we need to flush the local icache, but to do that we need > to know which instructions have been changed. We could create a > modification queue, but that seems rather elaborate. > > What is is that changes OOPs in the instructions during a handshake? > Is it just garbage collection, or does it happen at other times too? They are only changed during a GC safepoint on archs with mustIterateImmediateOopsInCode set to true, AFAIK. Arm and aarch64 have it false. /Robbin > > We could simply move OOPs (and class metadata pointers?) into the > constant pool. That would have some performance impact on in-order > CPUs, but hopefully not much on out-of-order ones. > From zgu at redhat.com Mon Mar 11 13:30:10 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 11 Mar 2019 09:30:10 -0400 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> Message-ID: <6a85b951-0ff7-7f90-63f5-7bb2b0bcc5e9@redhat.com> On 3/11/19 3:46 AM, David Holmes wrote: > Hi, > > On 9/03/2019 6:56 pm, Ao Qi wrote: >> Hi, >> >> Could someone help to push this fix? > > You generally need two reviews for hotspot fixes (only one needs to be > from a Reviewer). In this case as you are touching a shenandoah file the > shenandoah folk should give the okay - cc'd Aleksey. Change looks good. Verified with Shenandoah build. Thanks, -Zhengyu > > Thanks, > David > >> Thanks, >> Ao Qi >> >> On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl >> wrote: >>> >>> Hi, >>> >>> On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: >>>> Hi, >>>> >>>> I found build is broken when configure with >>>> --with-jvm-features=-g1gc,-jfr.? There are two problems: >>>> >>>> 1. g1BarrierSet.hpp is included when g1 is disabled. I think this >>>> include can be removed. >>>> 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef >>>> INCLUDE_G1GC, cardTable.hpp is missed. >>> >>> (I noticed that too just yesterday when working on another change - >>> thanks for fixing this). >>> >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8220344 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ >>>> >>>> >>> >>> ?? looks good to me. >>> >>> Thomas >>> From tobias.hartmann at oracle.com Mon Mar 11 14:12:51 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 11 Mar 2019 15:12:51 +0100 Subject: [13] RFR(S): 8220341: Class redefinition fails with assert(!is_unloaded()) failed: unloaded method on the stack In-Reply-To: References: <4ed958d5-729b-9d2b-8d8a-bcfd7bbdea40@oracle.com> <585eadca-4f71-6cdf-2423-c76329392871@oracle.com> <86ddff52-972e-9181-61f7-55aabb04a206@oracle.com> Message-ID: Hi Coleen, thanks for the review. On 11.03.19 14:29, coleen.phillimore at oracle.com wrote: > Looks good, with a reminder not to touch the method and hot_method if the entry in the compile queue > is unloaded because they may be deleted if they are old (ie. redefined) Methods. Yes, all affected access should be guarded by a is_unloaded check. Best regards, Tobias From thomas.schatzl at oracle.com Mon Mar 11 16:13:13 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 11 Mar 2019 17:13:13 +0100 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: <3c498cf5-19bf-57dc-ca17-99ca25433d83@redhat.com> References: <3c498cf5-19bf-57dc-ca17-99ca25433d83@redhat.com> Message-ID: <0db27460b1ca93b61c07d1310c413a4b2b095f45.camel@oracle.com> Hi, On Fri, 2019-03-08 at 17:58 +0100, Aleksey Shipilev wrote: > On 3/8/19 2:51 PM, Thomas Schatzl wrote: > > While the change itself is a no-op in practice, and testing showed > > so, > > I would like non-Oracle platform maintainers to try to compile the > > change and report problems because of lack of build platforms. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8220301 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8220301/webrev/ > > Not sure why do we need to drag CardTable::CardValue all over the > place, and then STATIC_ASSERT its > size, when we could "just" do uint8_t everywhere? There has been some short internal discussion that preferred a typedef instead of uint8_t right away. I am somewhat preferring the typedef at this point (working in loosely related JDK-8213108) but I am good with a straight uint8_t too. Let's see what others think. The STATIC_ASSERTs are imo better replacements for the existing asserts that do the same as they fail on compile and not during runtime. > > Anyway, these configs build fine: > linux-aarch64-server-fastdebug > linux-arm-server-fastdebug > linux-s390x-server-fastdebug > linux-x86-server-fastdebug > linux-x86_64-minimal-fastdebug > linux-x86_64-zero-fastdebug > > And linux-ppc64le-server-fastdebug fails with: [...] > It passes with this addendum: > Thanks for getting them through these platforms, fixed in http://cr.openjdk.java.net/~tschatzl/8220301/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8220301/webrev.1/ (full) Thanks, Thomas From shade at redhat.com Mon Mar 11 16:25:41 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 11 Mar 2019 17:25:41 +0100 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: <0db27460b1ca93b61c07d1310c413a4b2b095f45.camel@oracle.com> References: <3c498cf5-19bf-57dc-ca17-99ca25433d83@redhat.com> <0db27460b1ca93b61c07d1310c413a4b2b095f45.camel@oracle.com> Message-ID: <3809e317-5a72-e3ba-6117-9bbefe8a204e@redhat.com> On 3/11/19 5:13 PM, Thomas Schatzl wrote: > There has been some short internal discussion that preferred a typedef > instead of uint8_t right away. I am somewhat preferring the typedef at > this point (working in loosely related JDK-8213108) but I am good with > a straight uint8_t too. Let's see what others think. > > The STATIC_ASSERTs are imo better replacements for the existing asserts > that do the same as they fail on compile and not during runtime. Right. I don't have a strong preference. My comment was about encoding ClassValue and then STATIC_ASSERT-ing its size vs. using the type that is guaranteed to be 1-byte. Nit: I missed this earlier, but can/should we do a single STATIC_ASSERT near CardTable::ClassValue definition with the explanation that platform-specific code has this assumption? This would provide additional safeguard from accidentally changing the typedef without changing the commented STATIC_ASSERT, and that STATIC_ASSERT would definitely fire on all platforms since it would be in shared code? > Thanks for getting them through these platforms, fixed in > http://cr.openjdk.java.net/~tschatzl/8220301/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8220301/webrev.1/ (full) Looks okay. -Aleksey From martin.doerr at sap.com Mon Mar 11 17:43:25 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 11 Mar 2019 17:43:25 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> Message-ID: Hi Robbin, > They are only changed during a GC safepoint on archs with > mustIterateImmediateOopsInCode set to true, AFAIK. Only x86 uses Oops directly encoded into the instruction stream. But all platforms have compressed Oops encoded: loadConN nodes in ad files. I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. For example fence() and acquire() already include instruction stream synchronization on PPC. Is that why you implemented the new barrier empty for non-x86 CPUs? Best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of Robbin Ehn Sent: Montag, 11. M?rz 2019 13:56 To: Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code On 2019-03-11 11:26, Andrew Haley wrote: > On 3/8/19 3:24 PM, Robbin Ehn wrote: > >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state >> it can start executing updated code. E.g. an oop in the instruction >> stream can have been updated. > > Hmm, interesting. I think it works on on AArch64 at present because > the segfault trap we take is effectively a full synchronization > operation. > > An AArch64 ISB (instruction synchronization barrier) invalidates the > pipeline. It doesn't invalidate the instruction cache, which we'd need > to do to see a relocated OOP. If an instruction is modified by some > other thread we need to flush the local icache, but to do that we need > to know which instructions have been changed. We could create a > modification queue, but that seems rather elaborate. > > What is is that changes OOPs in the instructions during a handshake? > Is it just garbage collection, or does it happen at other times too? They are only changed during a GC safepoint on archs with mustIterateImmediateOopsInCode set to true, AFAIK. Arm and aarch64 have it false. /Robbin > > We could simply move OOPs (and class metadata pointers?) into the > constant pool. That would have some performance impact on in-order > CPUs, but hopefully not much on out-of-order ones. > From bob.vandette at oracle.com Mon Mar 11 18:26:49 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Mar 2019 14:26:49 -0400 Subject: [Proposal] Better systemd slice memory limit support for OpenJDK In-Reply-To: <7c1eb1cc2345ecf35dc3a5ddc4f69546f1d33df4.camel@redhat.com> References: <4F03B58D-FD8A-4ED1-B14A-C3769C573647@oracle.com> <9D22CF42-A701-48E2-9B27-3D862F187B6C@oracle.com> <7c1eb1cc2345ecf35dc3a5ddc4f69546f1d33df4.camel@redhat.com> Message-ID: Severin, I?m looking at other Docker and Container issues for inclusion in JDK 13 and I?m trying to figure out if we should do anything about the issue you reported. If this problem is going away once the Linux distributions pick up Linux kernel 4.19 or greater, then I?d suggest we just wait. It looks like support for Ubuntu 18.10 ends this July and Ubuntu 19.04 and Fedora 30 are coming out next month (April). Do you agree? Bob. > On Jan 17, 2019, at 2:24 PM, Severin Gehwolf wrote: > > On Thu, 2019-01-17 at 13:11 -0500, Bob Vandette wrote: >> Do you happen to know if this issue impacts Ubuntu 19.04? > > I don't, sorry. > >> Looking at our official OS support list, we?re only supporting Ubuntu 18.04 with JDK 12 which >> is using kernel version 4.15. Even if this gets bumped to 18.10, this is still only using 4.18. > > I can confirm Ubuntu 18.04 doesn't seem to be affected with Kernel > 4.15. RHEL-8 Beta, on the other hand, has 4.18 which is affected. I'm > also trying Ubuntu 18.10 now, which, in theory, should be affected. > >> Go ahead and file a bug while we look into this issue a bit more. I?d like to know the exact >> kernel cutoff and which distros are impacted. If it?s wide-spread on distros getting released >> this year, we should fix it in JDK 13. > > OK. Here you go: https://bugs.openjdk.java.net/browse/JDK-8217338 > > Thanks, > Severin > >> Bob. >> >> >>> On Jan 17, 2019, at 11:16 AM, Severin Gehwolf >>> wrote: >>> >>> Hi Bob, >>> >>> On Thu, 2019-01-17 at 10:59 -0500, Bob Vandette wrote: >>>> I checked a few systems I have access to and they all have >>>> use_hierarchy enabled. When I set a memory >>>> limit the memory.stat hierarchical_memory_limit is identical to >>>> the memory.limit_in_bytes contents. >>> >>> This seems to suggest they're running on older kernels. I know F29, >>> F28 >>> are affected. >>> >>>> Do you have any idea why the kernel behavior changed? >>> >>> No, not really. It could be with the "unified control group >>> hierarchy" >>> work done in 3.16, but that's speculation: >>> https://lwn.net/Articles/601840/ >>> >>> Some more discussion with systemd/kernel folk is here: >>> https://bugzilla.redhat.com/show_bug.cgi?id=1599387 >>> >>>> Is this documented behavior? >>> >>> Yes. Though, in the self-proclaimed very outdated document: >>> https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt >>> >>> Section 6, "Hierarchy support". >>> >>>> I wouldn?t want to add a work-around for a transient kernel bug. >>> >>> FWIW, cgroup-v1 is a legacy interface, so I doubt anything in that >>> code >>> has a great chance of getting "fixed". So far there is no sign of >>> evidence what the expected behaviour should be. >>> >>> Thanks, >>> Severin >>> >>>> Bob. >>>> >>>> >>>>> On Jan 17, 2019, at 9:57 AM, Severin Gehwolf < >>>>> sgehwolf at redhat.com> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Current container awareness for OpenJDK seems to work for >>>>> systemd >>>>> slices too, on some systems. To be precise, this works for >>>>> older >>>>> Kernels e.g. 3.10. However, we've noticed that this breaks for >>>>> newer >>>>> Kernel versions[1] such as the one in F28, currently 4.19.14- >>>>> 200. If >>>>> the container support would also look at hierarchical memory >>>>> limits >>>>> exposed via memory.stat in the cgroup file system, it would >>>>> again >>>>> work[2]. A proof of concept implementation is here: >>>>> >>>>> > http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/webrev/ >>>>> >>>>> This enhancement wouldn't change any existing container memory >>>>> limit >>>>> detection as it only kicks in when all other look-ups >>>>> determined that >>>>> there is no limit in place. I've verified this by running >>>>> Docker >>>>> container tests. The idea is to look for >>>>> hierarchical_memory_limit and >>>>> hierarchical_memsw_limit lines in the memory.stat file of the >>>>> cgroup >>>>> tree. >>>>> >>>>> Would it be possible to consider such an enhancement upstream? >>>>> If so, >>>>> I'll file a bug and propose it for review. >>>>> >>>>> This issue has been originally raised here: >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1509371 >>>>> >>>>> Thanks, >>>>> Severin >>>>> >>>>> [1] Java process gets killed by oom killer. >>>>> See: >>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/before.txt >>>>> [2] Java process throws OutOfMemoryError as expected. >>>>> See: >>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/after.txt >>>>> >>>> >>>> >> >> > From kim.barrett at oracle.com Mon Mar 11 18:37:48 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 11 Mar 2019 14:37:48 -0400 Subject: RFR: 8217576: C1 atomic access handlers use incorrect decorators In-Reply-To: References: <721DCA80-9B21-4196-AE88-382AC94B5D5F@oracle.com> Message-ID: <612E7D43-FBFB-4D84-8EAB-515EB404F8D8@oracle.com> > On Mar 11, 2019, at 3:22 AM, David Holmes wrote: > > Hi Kim, > > On 10/03/2019 7:47 am, Kim Barrett wrote: >> Please review this fix of decorator defaulting in C1 atomic access >> generators. As noted in the bug, C1 presently ignores the decorators >> and only generates fully barriered operations, so the incorrect >> decorators haven't been affecting code generation. > > So by fixing this and potentially using weaker memory ordering we could infact expose hitherto hidden bugs. :) Okay. It's weirder than that. The old code produces a decorator set that is either empty in the MO_ field, or contains two set bits in that field. Neither of those is valid, so who knows what would happen with later changes elsewhere. My guess: either an assertion failure (good) or quietly ignoring an intentional request for a weaker barrier (not so good, might lead to some real head-scratching debugging). > The fix itself looks good. Thanks. > Thanks, > David > ----- > >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8217576 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8217576/open.00/ >> Testing: >> mach5 tier1. From kim.barrett at oracle.com Mon Mar 11 19:29:42 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 11 Mar 2019 15:29:42 -0400 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: References: Message-ID: <3AB4F694-2289-4E4D-8B2B-7DE24E9C4C50@oracle.com> > On Mar 8, 2019, at 8:51 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change the replaces the use of the JNI > data type jbyte in the CardTable class by uint8_t via the custom > CardValue typedef? > > This cleanup stems from trying to get rid of JNI data types in VM/GC > code; formerly the main reason to use them here has been that the > Atomic operations were not available for e.g. uint8_t, but that has > been fixed for a long time. > > While the change itself is a no-op in practice, and testing showed so, > I would like non-Oracle platform maintainers to try to compile the > change and report problems because of lack of build platforms. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8220301 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8220301/webrev/ > Testing: > hs-tier1-5 > > Thanks, > Thomas I'm concerned about changing the signed-ness of card table entries. jbyte is signed. There may be code that expects or assumes signed values. One place of particular concern would be assembly code generators, where we might be generating sign-extending loads. Note that I didn't actually spot any signed-ness problems, but they would be easy to miss. ------------------------------------------------------------------------------ src/hotspot/cpu/aarch64/gc/g1/g1BarrierSetAssembler_aarch64.cpp 44 STATIC_ASSERT(sizeof(CardTable::CardValue) == 1); I think it is better to have assertions (static or otherwise) near the code that depends on them for correctness. The old asserts looked like they were in the right places, this replacement STATIC_ASSERT is not. Similarly in lots of other places. And there are two top-level near the beggining of the file static asserts here: src/hotspot/cpu/arm/gc/g1/g1BarrierSetAssembler_arm.cpp An alternative would be to document and assert it once at the place where CardValue is defined, and just say there's lots of code making that assumption, so change at your peril. That would also address Aleksey's complaint. ------------------------------------------------------------------------------ src/hotspot/share/gc/cms/cmsCardTable.cpp 230 CardValue* last_card_to_check = 231 (CardValue*) MIN2((intptr_t) last_card_of_cur_chunk, 232 (intptr_t) last_card_of_first_obj); pre-existing: Why all the casts here? This would work fine with all the casts removed. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1DirtyCardQueue.hpp 40 protected: 41 typedef CardTable::CardValue CardValue; Not sure why this is being made protected rather than public. Public seems more natural, since it's the declared type of a parameter of a public function. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1HotCardCache.cpp 80 CardValue* previous_ptr = Atomic::cmpxchg(card_ptr, 81 &_hot_cache[masked_index], 82 current_ptr); Argument indentation seems to have gotten messed up. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RemSet.[ch]hpp Seems like there are enough CardTable::CardValue uses in this class to be worth a typedef. ------------------------------------------------------------------------------ From sgehwolf at redhat.com Mon Mar 11 19:46:51 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 11 Mar 2019 20:46:51 +0100 Subject: [Proposal] Better systemd slice memory limit support for OpenJDK In-Reply-To: References: <4F03B58D-FD8A-4ED1-B14A-C3769C573647@oracle.com> <9D22CF42-A701-48E2-9B27-3D862F187B6C@oracle.com> <7c1eb1cc2345ecf35dc3a5ddc4f69546f1d33df4.camel@redhat.com> Message-ID: <4dd64e3748220ea73a6aee50255dc5a94d07a2b1.camel@redhat.com> Hi Bob, On Mon, 2019-03-11 at 14:26 -0400, Bob Vandette wrote: > Severin, > > I?m looking at other Docker and Container issues for inclusion in JDK 13 and > I?m trying to figure out if we should do anything about the issue you reported. > > If this problem is going away once the Linux distributions pick up Linux kernel > 4.19 or greater, then I?d suggest we just wait. What makes you think it does? That comment in Red Hat bugzilla seems wrong to me (if it was that). I'm on 4.20 in F29 and can still reproduce the issue. See attachment. > It looks like support for Ubuntu 18.10 ends this July and Ubuntu 19.04 and Fedora 30 > are coming out next month (April). > > Do you agree? Not quite. I believe the fix is needed. It'll become more prevalent the more distros switch to newer kernels. Heads-up: There is going to be a push for cgroup v2 by default in Fedora[1]. At some point cgroup v2 will need to be supported by the JVM. It's on my plate to work on in the coming months. Thanks, Severin [1] https://fedoraproject.org/wiki/Changes/CGroupsV2 > Bob. > > > > > On Jan 17, 2019, at 2:24 PM, Severin Gehwolf > > wrote: > > > > On Thu, 2019-01-17 at 13:11 -0500, Bob Vandette wrote: > > > Do you happen to know if this issue impacts Ubuntu 19.04? > > > > I don't, sorry. > > > > > Looking at our official OS support list, we?re only supporting > > > Ubuntu 18.04 with JDK 12 which > > > is using kernel version 4.15. Even if this gets bumped to 18.10, > > > this is still only using 4.18. > > > > I can confirm Ubuntu 18.04 doesn't seem to be affected with Kernel > > 4.15. RHEL-8 Beta, on the other hand, has 4.18 which is affected. > > I'm > > also trying Ubuntu 18.10 now, which, in theory, should be affected. > > > > > Go ahead and file a bug while we look into this issue a bit > > > more. I?d like to know the exact > > > kernel cutoff and which distros are impacted. If it?s wide- > > > spread on distros getting released > > > this year, we should fix it in JDK 13. > > > > OK. Here you go: https://bugs.openjdk.java.net/browse/JDK-8217338 > > > > Thanks, > > Severin > > > > > Bob. > > > > > > > > > > On Jan 17, 2019, at 11:16 AM, Severin Gehwolf < > > > > sgehwolf at redhat.com> > > > > wrote: > > > > > > > > Hi Bob, > > > > > > > > On Thu, 2019-01-17 at 10:59 -0500, Bob Vandette wrote: > > > > > I checked a few systems I have access to and they all have > > > > > use_hierarchy enabled. When I set a memory > > > > > limit the memory.stat hierarchical_memory_limit is identical > > > > > to > > > > > the memory.limit_in_bytes contents. > > > > > > > > This seems to suggest they're running on older kernels. I know > > > > F29, > > > > F28 > > > > are affected. > > > > > > > > > Do you have any idea why the kernel behavior changed? > > > > > > > > No, not really. It could be with the "unified control group > > > > hierarchy" > > > > work done in 3.16, but that's speculation: > > > > https://lwn.net/Articles/601840/ > > > > > > > > Some more discussion with systemd/kernel folk is here: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1599387 > > > > > > > > > Is this documented behavior? > > > > > > > > Yes. Though, in the self-proclaimed very outdated document: > > > > https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt > > > > > > > > Section 6, "Hierarchy support". > > > > > > > > > I wouldn?t want to add a work-around for a transient kernel > > > > > bug. > > > > > > > > FWIW, cgroup-v1 is a legacy interface, so I doubt anything in > > > > that > > > > code > > > > has a great chance of getting "fixed". So far there is no sign > > > > of > > > > evidence what the expected behaviour should be. > > > > > > > > Thanks, > > > > Severin > > > > > > > > > Bob. > > > > > > > > > > > > > > > > On Jan 17, 2019, at 9:57 AM, Severin Gehwolf < > > > > > > sgehwolf at redhat.com> wrote: > > > > > > > > > > > > Hi, > > > > > > > > > > > > Current container awareness for OpenJDK seems to work for > > > > > > systemd > > > > > > slices too, on some systems. To be precise, this works for > > > > > > older > > > > > > Kernels e.g. 3.10. However, we've noticed that this breaks > > > > > > for > > > > > > newer > > > > > > Kernel versions[1] such as the one in F28, currently > > > > > > 4.19.14- > > > > > > 200. If > > > > > > the container support would also look at hierarchical > > > > > > memory > > > > > > limits > > > > > > exposed via memory.stat in the cgroup file system, it would > > > > > > again > > > > > > work[2]. A proof of concept implementation is here: > > > > > > > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/webrev/ > > > > > > This enhancement wouldn't change any existing container > > > > > > memory > > > > > > limit > > > > > > detection as it only kicks in when all other look-ups > > > > > > determined that > > > > > > there is no limit in place. I've verified this by running > > > > > > Docker > > > > > > container tests. The idea is to look for > > > > > > hierarchical_memory_limit and > > > > > > hierarchical_memsw_limit lines in the memory.stat file of > > > > > > the > > > > > > cgroup > > > > > > tree. > > > > > > > > > > > > Would it be possible to consider such an enhancement > > > > > > upstream? > > > > > > If so, > > > > > > I'll file a bug and propose it for review. > > > > > > > > > > > > This issue has been originally raised here: > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1509371 > > > > > > > > > > > > Thanks, > > > > > > Severin > > > > > > > > > > > > [1] Java process gets killed by oom killer. > > > > > > See: > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/before.txt > > > > > > [2] Java process throws OutOfMemoryError as expected. > > > > > > See: > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/after.txt > > > > > > -------------- next part -------------- # /usr/lib/jvm/java-11-openjdk/bin/java -Xlog:os+container=trace --version [0.001s][trace][os,container] OSContainer::init: Initializing Container Support [0.002s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.002s][trace][os,container] Memory Limit is: 9223372036854771712 [0.002s][trace][os,container] Memory Limit is: Unlimited [0.002s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us [0.002s][trace][os,container] CPU Quota is: -1 [0.002s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us [0.003s][trace][os,container] CPU Period is: 100000 [0.003s][trace][os,container] Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares [0.003s][trace][os,container] CPU Shares is: 1024 [0.003s][trace][os,container] OSContainer::active_processor_count: 8 [0.003s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us [0.003s][trace][os,container] CPU Quota is: -1 [0.003s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us [0.003s][trace][os,container] CPU Period is: 100000 [0.003s][trace][os,container] Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares [0.003s][trace][os,container] CPU Shares is: 1024 [0.003s][trace][os,container] OSContainer::active_processor_count: 8 [0.003s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.003s][trace][os,container] Memory Limit is: 9223372036854771712 [0.003s][trace][os,container] Memory Limit is: Unlimited [0.003s][debug][os,container] container memory limit unlimited: -1, using host value [0.003s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.003s][trace][os,container] Memory Limit is: 9223372036854771712 [0.003s][trace][os,container] Memory Limit is: Unlimited [0.003s][debug][os,container] container memory limit unlimited: -1, using host value [0.007s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us [0.007s][trace][os,container] CPU Quota is: -1 [0.007s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us [0.007s][trace][os,container] CPU Period is: 100000 [0.007s][trace][os,container] Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares [0.008s][trace][os,container] CPU Shares is: 1024 [0.008s][trace][os,container] OSContainer::active_processor_count: 8 [0.124s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us [0.124s][trace][os,container] CPU Quota is: -1 [0.124s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us [0.124s][trace][os,container] CPU Period is: 100000 [0.124s][trace][os,container] Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares [0.125s][trace][os,container] CPU Shares is: 1024 [0.125s][trace][os,container] OSContainer::active_processor_count: 8 [0.221s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.222s][trace][os,container] Memory Limit is: 9223372036854771712 [0.222s][trace][os,container] Memory Limit is: Unlimited [0.222s][debug][os,container] container memory limit unlimited: -1, using host value [0.224s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.224s][trace][os,container] Memory Limit is: 9223372036854771712 [0.224s][trace][os,container] Memory Limit is: Unlimited [0.224s][debug][os,container] container memory limit unlimited: -1, using host value [0.225s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.225s][trace][os,container] Memory Limit is: 9223372036854771712 [0.225s][trace][os,container] Memory Limit is: Unlimited [0.225s][debug][os,container] container memory limit unlimited: -1, using host value [0.226s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.226s][trace][os,container] Memory Limit is: 9223372036854771712 [0.226s][trace][os,container] Memory Limit is: Unlimited [0.226s][debug][os,container] container memory limit unlimited: -1, using host value [0.226s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.226s][trace][os,container] Memory Limit is: 9223372036854771712 [0.226s][trace][os,container] Memory Limit is: Unlimited [0.226s][debug][os,container] container memory limit unlimited: -1, using host value [0.227s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.227s][trace][os,container] Memory Limit is: 9223372036854771712 [0.227s][trace][os,container] Memory Limit is: Unlimited [0.227s][debug][os,container] container memory limit unlimited: -1, using host value [0.227s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.228s][trace][os,container] Memory Limit is: 9223372036854771712 [0.228s][trace][os,container] Memory Limit is: Unlimited [0.228s][debug][os,container] container memory limit unlimited: -1, using host value [0.229s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.229s][trace][os,container] Memory Limit is: 9223372036854771712 [0.229s][trace][os,container] Memory Limit is: Unlimited [0.229s][debug][os,container] container memory limit unlimited: -1, using host value [0.229s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.229s][trace][os,container] Memory Limit is: 9223372036854771712 [0.229s][trace][os,container] Memory Limit is: Unlimited [0.229s][debug][os,container] container memory limit unlimited: -1, using host value [0.230s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.230s][trace][os,container] Memory Limit is: 9223372036854771712 [0.230s][trace][os,container] Memory Limit is: Unlimited [0.230s][debug][os,container] container memory limit unlimited: -1, using host value [0.231s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.231s][trace][os,container] Memory Limit is: 9223372036854771712 [0.231s][trace][os,container] Memory Limit is: Unlimited [0.231s][debug][os,container] container memory limit unlimited: -1, using host value [0.232s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.232s][trace][os,container] Memory Limit is: 9223372036854771712 [0.232s][trace][os,container] Memory Limit is: Unlimited [0.232s][debug][os,container] container memory limit unlimited: -1, using host value [0.232s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.233s][trace][os,container] Memory Limit is: 9223372036854771712 [0.233s][trace][os,container] Memory Limit is: Unlimited [0.233s][debug][os,container] container memory limit unlimited: -1, using host value [0.233s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.233s][trace][os,container] Memory Limit is: 9223372036854771712 [0.233s][trace][os,container] Memory Limit is: Unlimited [0.233s][debug][os,container] container memory limit unlimited: -1, using host value [0.233s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.233s][trace][os,container] Memory Limit is: 9223372036854771712 [0.233s][trace][os,container] Memory Limit is: Unlimited [0.234s][debug][os,container] container memory limit unlimited: -1, using host value [0.237s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.237s][trace][os,container] Memory Limit is: 9223372036854771712 [0.237s][trace][os,container] Memory Limit is: Unlimited [0.237s][debug][os,container] container memory limit unlimited: -1, using host value [0.238s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.238s][trace][os,container] Memory Limit is: 9223372036854771712 [0.238s][trace][os,container] Memory Limit is: Unlimited [0.238s][debug][os,container] container memory limit unlimited: -1, using host value [0.239s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.239s][trace][os,container] Memory Limit is: 9223372036854771712 [0.239s][trace][os,container] Memory Limit is: Unlimited [0.239s][debug][os,container] container memory limit unlimited: -1, using host value [0.240s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.240s][trace][os,container] Memory Limit is: 9223372036854771712 [0.240s][trace][os,container] Memory Limit is: Unlimited [0.240s][debug][os,container] container memory limit unlimited: -1, using host value [0.243s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.243s][trace][os,container] Memory Limit is: 9223372036854771712 [0.243s][trace][os,container] Memory Limit is: Unlimited [0.243s][debug][os,container] container memory limit unlimited: -1, using host value [0.244s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.245s][trace][os,container] Memory Limit is: 9223372036854771712 [0.245s][trace][os,container] Memory Limit is: Unlimited [0.245s][debug][os,container] container memory limit unlimited: -1, using host value [0.245s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.245s][trace][os,container] Memory Limit is: 9223372036854771712 [0.245s][trace][os,container] Memory Limit is: Unlimited [0.245s][debug][os,container] container memory limit unlimited: -1, using host value [0.246s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.246s][trace][os,container] Memory Limit is: 9223372036854771712 [0.246s][trace][os,container] Memory Limit is: Unlimited [0.246s][debug][os,container] container memory limit unlimited: -1, using host value [0.246s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.247s][trace][os,container] Memory Limit is: 9223372036854771712 [0.247s][trace][os,container] Memory Limit is: Unlimited [0.247s][debug][os,container] container memory limit unlimited: -1, using host value [0.247s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.247s][trace][os,container] Memory Limit is: 9223372036854771712 [0.247s][trace][os,container] Memory Limit is: Unlimited [0.247s][debug][os,container] container memory limit unlimited: -1, using host value [0.248s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.248s][trace][os,container] Memory Limit is: 9223372036854771712 [0.248s][trace][os,container] Memory Limit is: Unlimited [0.248s][debug][os,container] container memory limit unlimited: -1, using host value [0.249s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.249s][trace][os,container] Memory Limit is: 9223372036854771712 [0.249s][trace][os,container] Memory Limit is: Unlimited [0.249s][debug][os,container] container memory limit unlimited: -1, using host value [0.250s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.250s][trace][os,container] Memory Limit is: 9223372036854771712 [0.250s][trace][os,container] Memory Limit is: Unlimited [0.250s][debug][os,container] container memory limit unlimited: -1, using host value [0.257s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.257s][trace][os,container] Memory Limit is: 9223372036854771712 [0.257s][trace][os,container] Memory Limit is: Unlimited [0.257s][debug][os,container] container memory limit unlimited: -1, using host value [0.259s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.259s][trace][os,container] Memory Limit is: 9223372036854771712 [0.259s][trace][os,container] Memory Limit is: Unlimited [0.259s][debug][os,container] container memory limit unlimited: -1, using host value [0.259s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.259s][trace][os,container] Memory Limit is: 9223372036854771712 [0.259s][trace][os,container] Memory Limit is: Unlimited [0.259s][debug][os,container] container memory limit unlimited: -1, using host value [0.259s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.259s][trace][os,container] Memory Limit is: 9223372036854771712 [0.259s][trace][os,container] Memory Limit is: Unlimited [0.259s][debug][os,container] container memory limit unlimited: -1, using host value [0.260s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.260s][trace][os,container] Memory Limit is: 9223372036854771712 [0.260s][trace][os,container] Memory Limit is: Unlimited [0.260s][debug][os,container] container memory limit unlimited: -1, using host value [0.260s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.260s][trace][os,container] Memory Limit is: 9223372036854771712 [0.260s][trace][os,container] Memory Limit is: Unlimited [0.260s][debug][os,container] container memory limit unlimited: -1, using host value [0.260s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.260s][trace][os,container] Memory Limit is: 9223372036854771712 [0.260s][trace][os,container] Memory Limit is: Unlimited [0.260s][debug][os,container] container memory limit unlimited: -1, using host value [0.261s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.261s][trace][os,container] Memory Limit is: 9223372036854771712 [0.261s][trace][os,container] Memory Limit is: Unlimited [0.261s][debug][os,container] container memory limit unlimited: -1, using host value [0.261s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.261s][trace][os,container] Memory Limit is: 9223372036854771712 [0.261s][trace][os,container] Memory Limit is: Unlimited [0.261s][debug][os,container] container memory limit unlimited: -1, using host value [0.261s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.261s][trace][os,container] Memory Limit is: 9223372036854771712 [0.261s][trace][os,container] Memory Limit is: Unlimited [0.261s][debug][os,container] container memory limit unlimited: -1, using host value [0.262s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.262s][trace][os,container] Memory Limit is: 9223372036854771712 [0.262s][trace][os,container] Memory Limit is: Unlimited [0.262s][debug][os,container] container memory limit unlimited: -1, using host value [0.263s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.263s][trace][os,container] Memory Limit is: 9223372036854771712 [0.263s][trace][os,container] Memory Limit is: Unlimited [0.263s][debug][os,container] container memory limit unlimited: -1, using host value [0.263s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.263s][trace][os,container] Memory Limit is: 9223372036854771712 [0.263s][trace][os,container] Memory Limit is: Unlimited [0.263s][debug][os,container] container memory limit unlimited: -1, using host value [0.263s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.263s][trace][os,container] Memory Limit is: 9223372036854771712 [0.263s][trace][os,container] Memory Limit is: Unlimited [0.263s][debug][os,container] container memory limit unlimited: -1, using host value [0.263s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.264s][trace][os,container] Memory Limit is: 9223372036854771712 [0.264s][trace][os,container] Memory Limit is: Unlimited [0.264s][debug][os,container] container memory limit unlimited: -1, using host value [0.264s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.264s][trace][os,container] Memory Limit is: 9223372036854771712 [0.264s][trace][os,container] Memory Limit is: Unlimited [0.264s][debug][os,container] container memory limit unlimited: -1, using host value [0.264s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.264s][trace][os,container] Memory Limit is: 9223372036854771712 [0.264s][trace][os,container] Memory Limit is: Unlimited [0.264s][debug][os,container] container memory limit unlimited: -1, using host value [0.264s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.264s][trace][os,container] Memory Limit is: 9223372036854771712 [0.264s][trace][os,container] Memory Limit is: Unlimited [0.264s][debug][os,container] container memory limit unlimited: -1, using host value [0.264s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.264s][trace][os,container] Memory Limit is: 9223372036854771712 [0.264s][trace][os,container] Memory Limit is: Unlimited [0.264s][debug][os,container] container memory limit unlimited: -1, using host value [0.264s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.264s][trace][os,container] Memory Limit is: 9223372036854771712 [0.264s][trace][os,container] Memory Limit is: Unlimited [0.264s][debug][os,container] container memory limit unlimited: -1, using host value [0.264s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.264s][trace][os,container] Memory Limit is: 9223372036854771712 [0.264s][trace][os,container] Memory Limit is: Unlimited [0.264s][debug][os,container] container memory limit unlimited: -1, using host value [0.265s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.265s][trace][os,container] Memory Limit is: 9223372036854771712 [0.265s][trace][os,container] Memory Limit is: Unlimited [0.265s][debug][os,container] container memory limit unlimited: -1, using host value [0.265s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.265s][trace][os,container] Memory Limit is: 9223372036854771712 [0.265s][trace][os,container] Memory Limit is: Unlimited [0.265s][debug][os,container] container memory limit unlimited: -1, using host value [0.265s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.265s][trace][os,container] Memory Limit is: 9223372036854771712 [0.265s][trace][os,container] Memory Limit is: Unlimited [0.265s][debug][os,container] container memory limit unlimited: -1, using host value [0.265s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.265s][trace][os,container] Memory Limit is: 9223372036854771712 [0.265s][trace][os,container] Memory Limit is: Unlimited [0.265s][debug][os,container] container memory limit unlimited: -1, using host value [0.265s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.265s][trace][os,container] Memory Limit is: 9223372036854771712 [0.265s][trace][os,container] Memory Limit is: Unlimited [0.265s][debug][os,container] container memory limit unlimited: -1, using host value [0.266s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.266s][trace][os,container] Memory Limit is: 9223372036854771712 [0.266s][trace][os,container] Memory Limit is: Unlimited [0.266s][debug][os,container] container memory limit unlimited: -1, using host value [0.267s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.267s][trace][os,container] Memory Limit is: 9223372036854771712 [0.267s][trace][os,container] Memory Limit is: Unlimited [0.267s][debug][os,container] container memory limit unlimited: -1, using host value [0.267s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.267s][trace][os,container] Memory Limit is: 9223372036854771712 [0.267s][trace][os,container] Memory Limit is: Unlimited [0.267s][debug][os,container] container memory limit unlimited: -1, using host value [0.268s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.268s][trace][os,container] Memory Limit is: 9223372036854771712 [0.268s][trace][os,container] Memory Limit is: Unlimited [0.268s][debug][os,container] container memory limit unlimited: -1, using host value [0.268s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.268s][trace][os,container] Memory Limit is: 9223372036854771712 [0.268s][trace][os,container] Memory Limit is: Unlimited [0.268s][debug][os,container] container memory limit unlimited: -1, using host value [0.268s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.268s][trace][os,container] Memory Limit is: 9223372036854771712 [0.268s][trace][os,container] Memory Limit is: Unlimited [0.268s][debug][os,container] container memory limit unlimited: -1, using host value [0.268s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.268s][trace][os,container] Memory Limit is: 9223372036854771712 [0.268s][trace][os,container] Memory Limit is: Unlimited [0.268s][debug][os,container] container memory limit unlimited: -1, using host value [0.268s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.268s][trace][os,container] Memory Limit is: 9223372036854771712 [0.268s][trace][os,container] Memory Limit is: Unlimited [0.268s][debug][os,container] container memory limit unlimited: -1, using host value [0.269s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.269s][trace][os,container] Memory Limit is: 9223372036854771712 [0.269s][trace][os,container] Memory Limit is: Unlimited [0.269s][debug][os,container] container memory limit unlimited: -1, using host value [0.269s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.269s][trace][os,container] Memory Limit is: 9223372036854771712 [0.269s][trace][os,container] Memory Limit is: Unlimited [0.269s][debug][os,container] container memory limit unlimited: -1, using host value [0.269s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.269s][trace][os,container] Memory Limit is: 9223372036854771712 [0.269s][trace][os,container] Memory Limit is: Unlimited [0.269s][debug][os,container] container memory limit unlimited: -1, using host value [0.270s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.270s][trace][os,container] Memory Limit is: 9223372036854771712 [0.270s][trace][os,container] Memory Limit is: Unlimited [0.270s][debug][os,container] container memory limit unlimited: -1, using host value [0.270s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.270s][trace][os,container] Memory Limit is: 9223372036854771712 [0.270s][trace][os,container] Memory Limit is: Unlimited [0.270s][debug][os,container] container memory limit unlimited: -1, using host value openjdk 11.0.2 2019-01-15 OpenJDK Runtime Environment 18.9 (build 11.0.2+7) OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+7, mixed mode, sharing) [0.271s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.limit_in_bytes [0.271s][trace][os,container] Memory Limit is: 9223372036854771712 [0.271s][trace][os,container] Memory Limit is: Unlimited [0.271s][debug][os,container] container memory limit unlimited: -1, using host value # uname -a Linux t580-laptop 4.20.13-200.fc29.x86_64 #1 SMP Wed Feb 27 19:42:55 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux # grep hierarchical /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf0177a78f1d4b33b0c9b4de67082faa.scope/memory.stat hierarchical_memory_limit 10485760 hierarchical_memsw_limit 9223372036854771712 From bob.vandette at oracle.com Mon Mar 11 19:58:09 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Mar 2019 15:58:09 -0400 Subject: [Proposal] Better systemd slice memory limit support for OpenJDK In-Reply-To: <4dd64e3748220ea73a6aee50255dc5a94d07a2b1.camel@redhat.com> References: <4F03B58D-FD8A-4ED1-B14A-C3769C573647@oracle.com> <9D22CF42-A701-48E2-9B27-3D862F187B6C@oracle.com> <7c1eb1cc2345ecf35dc3a5ddc4f69546f1d33df4.camel@redhat.com> <4dd64e3748220ea73a6aee50255dc5a94d07a2b1.camel@redhat.com> Message-ID: <13644592-D61D-4911-8204-E3DC1A99A52C@oracle.com> > On Mar 11, 2019, at 3:46 PM, Severin Gehwolf wrote: > > Hi Bob, > > On Mon, 2019-03-11 at 14:26 -0400, Bob Vandette wrote: >> Severin, >> >> I?m looking at other Docker and Container issues for inclusion in JDK 13 and >> I?m trying to figure out if we should do anything about the issue you reported. >> >> If this problem is going away once the Linux distributions pick up Linux kernel >> 4.19 or greater, then I?d suggest we just wait. > > What makes you think it does? That comment in Red Hat bugzilla seems > wrong to me (if it was that). I'm on 4.20 in F29 and can still > reproduce the issue. See attachment. These comments make it sound like it?s fixed in the Linux kernel version 4.19. Jeremy Cline 2018-10-19 14:47:35 UTC Hey folks, This looks to work as expected in the latest 4.19 kernels. F29 should be rebased to 4.19 in a couple weeks, and since it is actually enforced I don't think it's worth tracking down the commit that fixed this. Jeremy Cline 2018-10-19 15:28:31 UTC > This breaks automatic container detection in language runtimes (e.g. OpenJDK). It sizes it's structures according to the container limit. If it's not able to detect that it's in a container but actually is, then all bets are off. It looks to the user as if some application gets killed seemingly randomly without knowing why. So to that extent, I'm not sure I agree with "it's not worth tracking down the commit that fixed this". I'm happy to backport a patch if you or someone else wants to bisect it, but it's already fixed upstream and it'll be fixed in stable Fedora in a few weeks when we rebase to 4.19. >> It looks like support for Ubuntu 18.10 ends this July and Ubuntu 19.04 and Fedora 30 >> are coming out next month (April). >> >> Do you agree? > > Not quite. I believe the fix is needed. It'll become more prevalent the > more distros switch to newer kernels. > It would be nice to confirm one way or the other. If we do fix this, the Metrics.java file needs to be updated as well. > Heads-up: There is going to be a push for cgroup v2 by default in > Fedora[1]. At some point cgroup v2 will need to be supported by the > JVM. It's on my plate to work on in the coming months. Wow, did they finally complete all of the subsystems that Docker needs or are they planning on mixing v1 with v2? Sounds great, Bob. > > Thanks, > Severin > > [1] https://fedoraproject.org/wiki/Changes/CGroupsV2 > >> Bob. >> >> >> >>> On Jan 17, 2019, at 2:24 PM, Severin Gehwolf >>> wrote: >>> >>> On Thu, 2019-01-17 at 13:11 -0500, Bob Vandette wrote: >>>> Do you happen to know if this issue impacts Ubuntu 19.04? >>> >>> I don't, sorry. >>> >>>> Looking at our official OS support list, we?re only supporting >>>> Ubuntu 18.04 with JDK 12 which >>>> is using kernel version 4.15. Even if this gets bumped to 18.10, >>>> this is still only using 4.18. >>> >>> I can confirm Ubuntu 18.04 doesn't seem to be affected with Kernel >>> 4.15. RHEL-8 Beta, on the other hand, has 4.18 which is affected. >>> I'm >>> also trying Ubuntu 18.10 now, which, in theory, should be affected. >>> >>>> Go ahead and file a bug while we look into this issue a bit >>>> more. I?d like to know the exact >>>> kernel cutoff and which distros are impacted. If it?s wide- >>>> spread on distros getting released >>>> this year, we should fix it in JDK 13. >>> >>> OK. Here you go: https://bugs.openjdk.java.net/browse/JDK-8217338 >>> >>> Thanks, >>> Severin >>> >>>> Bob. >>>> >>>> >>>>> On Jan 17, 2019, at 11:16 AM, Severin Gehwolf < >>>>> sgehwolf at redhat.com> >>>>> wrote: >>>>> >>>>> Hi Bob, >>>>> >>>>> On Thu, 2019-01-17 at 10:59 -0500, Bob Vandette wrote: >>>>>> I checked a few systems I have access to and they all have >>>>>> use_hierarchy enabled. When I set a memory >>>>>> limit the memory.stat hierarchical_memory_limit is identical >>>>>> to >>>>>> the memory.limit_in_bytes contents. >>>>> >>>>> This seems to suggest they're running on older kernels. I know >>>>> F29, >>>>> F28 >>>>> are affected. >>>>> >>>>>> Do you have any idea why the kernel behavior changed? >>>>> >>>>> No, not really. It could be with the "unified control group >>>>> hierarchy" >>>>> work done in 3.16, but that's speculation: >>>>> https://lwn.net/Articles/601840/ >>>>> >>>>> Some more discussion with systemd/kernel folk is here: >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1599387 >>>>> >>>>>> Is this documented behavior? >>>>> >>>>> Yes. Though, in the self-proclaimed very outdated document: >>>>> https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt >>>>> >>>>> Section 6, "Hierarchy support". >>>>> >>>>>> I wouldn?t want to add a work-around for a transient kernel >>>>>> bug. >>>>> >>>>> FWIW, cgroup-v1 is a legacy interface, so I doubt anything in >>>>> that >>>>> code >>>>> has a great chance of getting "fixed". So far there is no sign >>>>> of >>>>> evidence what the expected behaviour should be. >>>>> >>>>> Thanks, >>>>> Severin >>>>> >>>>>> Bob. >>>>>> >>>>>> >>>>>>> On Jan 17, 2019, at 9:57 AM, Severin Gehwolf < >>>>>>> sgehwolf at redhat.com> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Current container awareness for OpenJDK seems to work for >>>>>>> systemd >>>>>>> slices too, on some systems. To be precise, this works for >>>>>>> older >>>>>>> Kernels e.g. 3.10. However, we've noticed that this breaks >>>>>>> for >>>>>>> newer >>>>>>> Kernel versions[1] such as the one in F28, currently >>>>>>> 4.19.14- >>>>>>> 200. If >>>>>>> the container support would also look at hierarchical >>>>>>> memory >>>>>>> limits >>>>>>> exposed via memory.stat in the cgroup file system, it would >>>>>>> again >>>>>>> work[2]. A proof of concept implementation is here: >>>>>>> >>>>>>> >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/webrev/ >>>>>>> This enhancement wouldn't change any existing container >>>>>>> memory >>>>>>> limit >>>>>>> detection as it only kicks in when all other look-ups >>>>>>> determined that >>>>>>> there is no limit in place. I've verified this by running >>>>>>> Docker >>>>>>> container tests. The idea is to look for >>>>>>> hierarchical_memory_limit and >>>>>>> hierarchical_memsw_limit lines in the memory.stat file of >>>>>>> the >>>>>>> cgroup >>>>>>> tree. >>>>>>> >>>>>>> Would it be possible to consider such an enhancement >>>>>>> upstream? >>>>>>> If so, >>>>>>> I'll file a bug and propose it for review. >>>>>>> >>>>>>> This issue has been originally raised here: >>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1509371 >>>>>>> >>>>>>> Thanks, >>>>>>> Severin >>>>>>> >>>>>>> [1] Java process gets killed by oom killer. >>>>>>> See: >>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/before.txt >>>>>>> [2] Java process throws OutOfMemoryError as expected. >>>>>>> See: >>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/after.txt >>>>>>> > From david.holmes at oracle.com Tue Mar 12 01:09:55 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Mar 2019 11:09:55 +1000 Subject: RFR: 8217576: C1 atomic access handlers use incorrect decorators In-Reply-To: <612E7D43-FBFB-4D84-8EAB-515EB404F8D8@oracle.com> References: <721DCA80-9B21-4196-AE88-382AC94B5D5F@oracle.com> <612E7D43-FBFB-4D84-8EAB-515EB404F8D8@oracle.com> Message-ID: On 12/03/2019 4:37 am, Kim Barrett wrote: >> On Mar 11, 2019, at 3:22 AM, David Holmes wrote: >> >> Hi Kim, >> >> On 10/03/2019 7:47 am, Kim Barrett wrote: >>> Please review this fix of decorator defaulting in C1 atomic access >>> generators. As noted in the bug, C1 presently ignores the decorators >>> and only generates fully barriered operations, so the incorrect >>> decorators haven't been affecting code generation. >> >> So by fixing this and potentially using weaker memory ordering we could infact expose hitherto hidden bugs. :) Okay. > > It's weirder than that. The old code produces a decorator set that is > either empty in the MO_ field, or contains two set bits in that > field. Neither of those is valid, so who knows what would happen with > later changes elsewhere. My guess: either an assertion failure (good) > or quietly ignoring an intentional request for a weaker barrier (not > so good, might lead to some real head-scratching debugging). That sounds like we have missing assertions if exactly one-bit must always be set in the decorator! Though putting in a stronger MO than necessary would never be a functional bug. I'd be more concerned if the empty field led to an unexpected relaxed MO - that could be a real head-scratcher. David ----- >> The fix itself looks good. > > Thanks. > >> Thanks, >> David >> ----- >> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8217576 >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8217576/open.00/ >>> Testing: >>> mach5 tier1. > > From aoqi at loongson.cn Tue Mar 12 01:15:43 2019 From: aoqi at loongson.cn (Ao Qi) Date: Tue, 12 Mar 2019 09:15:43 +0800 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: <6a85b951-0ff7-7f90-63f5-7bb2b0bcc5e9@redhat.com> References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> <6a85b951-0ff7-7f90-63f5-7bb2b0bcc5e9@redhat.com> Message-ID: On Mon, Mar 11, 2019 at 9:30 PM Zhengyu Gu wrote: > > On 3/11/19 3:46 AM, David Holmes wrote: > > Hi, > > > > On 9/03/2019 6:56 pm, Ao Qi wrote: > >> Hi, > >> > >> Could someone help to push this fix? > > > > You generally need two reviews for hotspot fixes (only one needs to be > > from a Reviewer). In this case as you are touching a shenandoah file the > > shenandoah folk should give the okay - cc'd Aleksey. > > Change looks good. Verified with Shenandoah build. > Thanks, Zhengyu. I have two reviews now, and I think I need someone help me to push Thanks, Ao Qi > Thanks, > > -Zhengyu > > > > > Thanks, > > David > > > >> Thanks, > >> Ao Qi > >> > >> On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl > >> wrote: > >>> > >>> Hi, > >>> > >>> On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: > >>>> Hi, > >>>> > >>>> I found build is broken when configure with > >>>> --with-jvm-features=-g1gc,-jfr. There are two problems: > >>>> > >>>> 1. g1BarrierSet.hpp is included when g1 is disabled. I think this > >>>> include can be removed. > >>>> 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef > >>>> INCLUDE_G1GC, cardTable.hpp is missed. > >>> > >>> (I noticed that too just yesterday when working on another change - > >>> thanks for fixing this). > >>> > >>>> > >>>> Bug: > >>>> https://bugs.openjdk.java.net/browse/JDK-8220344 > >>>> > >>>> Webrev: > >>>> http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ > >>>> > >>>> > >>> > >>> looks good to me. > >>> > >>> Thomas > >>> From zgu at redhat.com Tue Mar 12 01:25:16 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 11 Mar 2019 21:25:16 -0400 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> <6a85b951-0ff7-7f90-63f5-7bb2b0bcc5e9@redhat.com> Message-ID: On 3/11/19 9:15 PM, Ao Qi wrote: > On Mon, Mar 11, 2019 at 9:30 PM Zhengyu Gu wrote: >> >> On 3/11/19 3:46 AM, David Holmes wrote: >>> Hi, >>> >>> On 9/03/2019 6:56 pm, Ao Qi wrote: >>>> Hi, >>>> >>>> Could someone help to push this fix? >>> >>> You generally need two reviews for hotspot fixes (only one needs to be >>> from a Reviewer). In this case as you are touching a shenandoah file the >>> shenandoah folk should give the okay - cc'd Aleksey. >> >> Change looks good. Verified with Shenandoah build. >> > > Thanks, Zhengyu. I have two reviews now, and I think I need someone > help me to push I can sponsor this patch. -Zhengyu > > Thanks, > Ao Qi > >> Thanks, >> >> -Zhengyu >> >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Ao Qi >>>> >>>> On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: >>>>>> Hi, >>>>>> >>>>>> I found build is broken when configure with >>>>>> --with-jvm-features=-g1gc,-jfr. There are two problems: >>>>>> >>>>>> 1. g1BarrierSet.hpp is included when g1 is disabled. I think this >>>>>> include can be removed. >>>>>> 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef >>>>>> INCLUDE_G1GC, cardTable.hpp is missed. >>>>> >>>>> (I noticed that too just yesterday when working on another change - >>>>> thanks for fixing this). >>>>> >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8220344 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ >>>>>> >>>>>> >>>>> >>>>> looks good to me. >>>>> >>>>> Thomas >>>>> From erik.osterlund at oracle.com Tue Mar 12 06:53:45 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 12 Mar 2019 07:53:45 +0100 Subject: RFR: 8217576: C1 atomic access handlers use incorrect decorators In-Reply-To: References: <721DCA80-9B21-4196-AE88-382AC94B5D5F@oracle.com> <612E7D43-FBFB-4D84-8EAB-515EB404F8D8@oracle.com> Message-ID: Hi Kim, Looks good. Thanks, /Erik > On 12 Mar 2019, at 02:09, David Holmes wrote: > > On 12/03/2019 4:37 am, Kim Barrett wrote: >>> On Mar 11, 2019, at 3:22 AM, David Holmes wrote: >>> >>> Hi Kim, >>> >>>> On 10/03/2019 7:47 am, Kim Barrett wrote: >>>> Please review this fix of decorator defaulting in C1 atomic access >>>> generators. As noted in the bug, C1 presently ignores the decorators >>>> and only generates fully barriered operations, so the incorrect >>>> decorators haven't been affecting code generation. >>> >>> So by fixing this and potentially using weaker memory ordering we could infact expose hitherto hidden bugs. :) Okay. >> It's weirder than that. The old code produces a decorator set that is >> either empty in the MO_ field, or contains two set bits in that >> field. Neither of those is valid, so who knows what would happen with >> later changes elsewhere. My guess: either an assertion failure (good) >> or quietly ignoring an intentional request for a weaker barrier (not >> so good, might lead to some real head-scratching debugging). > > That sounds like we have missing assertions if exactly one-bit must always be set in the decorator! > > Though putting in a stronger MO than necessary would never be a functional bug. I'd be more concerned if the empty field led to an unexpected relaxed MO - that could be a real head-scratcher. > > David > ----- > >>> The fix itself looks good. >> Thanks. >>> Thanks, >>> David >>> ----- >>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8217576 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kbarrett/8217576/open.00/ >>>> Testing: >>>> mach5 tier1. From david.holmes at oracle.com Tue Mar 12 08:00:04 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Mar 2019 18:00:04 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: Hi Robbin, I have comments on the various changes below, but now I've gone through that all I have a meta-comment on the overall approach. AFAICS the only time you have to sync the i-stream is when returning to _thread_in_Java, as that is the only state in which you can execute the modified code - no? Handling this at only those thread-state transitions would seem a lot simpler/cleaner. On 9/03/2019 1:24 am, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ src/hotspot/cpu/x86/assembler_x86.hpp As discussed elsewhere mfence() is no longer the advised mechanism, but instead a serializing instruction like cpuid must be used. --- src/hotspot/cpu/x86/macroAssembler_x86.cpp MacroAssembler::debug32 seems quite bizarre. First, pre-existing: 412 JavaThread* thread = JavaThread::current(); 413 JavaThreadState saved_state = thread->thread_state(); 414 thread->set_thread_state(_thread_in_vm); 415 if (ShowMessageBoxOnError) { 416 JavaThread* thread = JavaThread::current(); 417 JavaThreadState saved_state = thread->thread_state(); 418 thread->set_thread_state(_thread_in_vm); Lines 416-418 are redundant. IIUC you need to serialize after leaving a safepoint but this code cannot enter a safepoint. The state is _thread_in_vm, which prevents a safepoint from being reached and it does not do anything to change state (the ttyLocker locks without a safepoint check and so will not transition to _thread_blocked). So your change seems unnecessary. Further if this code could hit a safepoint check then the place for the new code would be in that return path from the safepoint not in this code. --- src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp 2562 __ lock(); 2563 __ addl(Address(r15_thread, JavaThread::thread_state_offset()), 1 /* _thread_in_native_trans */); This changes seems incorrect. You need a storeload barrier (at least) after the update to thread_state but your lock() prefix would place barriers before I think. In any case I find the existing code clearer and more obviously correct. --- src/hotspot/cpu/x86/stubGenerator_x86_64.cpp src/hotspot/os_cpu/windows_x86/orderAccess_windows_x86.hpp src/hotspot/share/runtime/orderAccess.cpp src/hotspot/share/runtime/stubRoutines.cpp src/hotspot/share/runtime/stubRoutines.hpp I don't understand why we need a stub for this (or for fence() for that matter) ?? --- src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp 1090 __ lock(); 1091 __ addl(Address(thread, JavaThread::thread_state_offset()), 1 /*_thread_in_native_trans*/); Same comment as above. --- src/hotspot/os_cpu/aix_ppc/orderAccess_aix_ppc.hpp src/hotspot/os_cpu/linux_ppc/orderAccess_linux_ppc.hpp PPC has isync for instruction stream synchronization, so I'd expect to see that used in OrderAccess::instruction_pipeline() --- src/hotspot/os_cpu/bsd_x86/orderAccess_bsd_x86.hpp src/hotspot/os_cpu/linux_x86/orderAccess_linux_x86.hpp src/hotspot/os_cpu/solaris_x86/orderAccess_solaris_x86.hpp s/mfence/cpuid --- src/hotspot/os_cpu/linux_aarch64/orderAccess_linux_aarch64.hpp src/hotspot/os_cpu/linux_arm/orderAccess_linux_arm.hpp Aarch64/ARM has ISB for instruction stream sync. --- src/hotspot/os_cpu/linux_s390/orderAccess_linux_s390.hpp Need input from s390 folk. --- src/hotspot/os_cpu/linux_sparc/orderAccess_linux_sparc.hpp src/hotspot/os_cpu/solaris_sparc/orderAccess_solaris_sparc.hpp SPARC needs a FLUSH "H.1.6 Self-Modifying Code If a program includes self-modifying code, it must issue a FLUSH instruction for each modified doubleword of instructions (or a call to supervisor software having an equivalent effect)." [SPARCV9 Architecture manual] --- src/hotspot/share/classfile/classFileParser.cpp Change is unrelated to this bug. --- src/hotspot/share/prims/jvmtiEnv.cpp The changes seem unnecessary. As the code comment states we are going from one safepoint unsafe state to another. There is no way to execute any JIT'd code before returning to Java, and that return is where we should sync if needed. --- src/hotspot/share/prims/jvmtiExport.cpp The change is unnecessary as we are executing in the VMThread. Not only would it be the one doing the code changes at a safepoint, it never executes compiled Java code anyway. --- src/hotspot/share/runtime/interfaceSupport.inline.hpp src/hotspot/share/runtime/safepoint.cpp My initial thought was that the OrderAccess::instruction_pipeline() should be buried in the safepoint blocking code, but in some cases we may not have actually blocked at the safepoint and so (theoretically at least) may need to sync with the i-stream in case an intervening safepoint changed some code. Can't help but worry that unconditionally doing this on native transitions is going to hurt JNI performance. It also looks like we will double up on the instruction_pipeline() in some cases. --- src/hotspot/share/runtime/thread.cpp 1839 OrderAccess::instruction_pipeline(); Can't see why this is necessary here, we should sync when returning to _thread_in_Java. 2471 if (thread_state() != _thread_blocked) { 2472 OrderAccess::instruction_pipeline(); 2473 } Again not sure why this is necessary. The call to java_suspend_self in the not-blocked case comes from JavaThread::handle_special_runtime_exit_condition, which itself is called from the transition code, so I'd expect this to be done in the transition code. thread->set_thread_state(_thread_blocked); thread->java_suspend_self(); thread->set_thread_state(state); + OrderAccess::instruction_pipeline(); Again seems unnecessary - especially given you have one in java_suspend_self() already! (Notwithstanding I think that one should be removed.) Thanks, David ----- > After a JavaThread have been in a safepoint/(handshake) safe state it > can start > executing updated code. E.g. an oop in the instruction stream can have been > updated. > > Most hardware's require a barrier or that the code cross modified is far > away to > guarantee that the thread executing the updated instruction stream sees the > modification. > > What far away is and how far an update instruction stream is from a > safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier > must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a > safepoint safe > state closer to a nmethod can increase changes of a crash. > > A few benchmarks will see a performance regression with the extra > serializing, > such as Octane-Crypto -5% (x86). > > With a much more elaborate fix, such as changing the local poll to have > more > than two states (armed/disarmed), it would be possible to only emit such > instruction stream barrier when the poll have been armed while the > thread was > safepoint safe. > > Passes t1-3 on our platforms, and test built what I can. > > Thanks, Robbin From robbin.ehn at oracle.com Tue Mar 12 09:43:43 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 12 Mar 2019 10:43:43 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> Message-ID: Hi Martin, On 3/11/19 6:43 PM, Doerr, Martin wrote: > Hi Robbin, > >> They are only changed during a GC safepoint on archs with >> mustIterateImmediateOopsInCode set to true, AFAIK. > > Only x86 uses Oops directly encoded into the instruction stream. > But all platforms have compressed Oops encoded: loadConN nodes in ad files. Sorry, forgot about that. Was just looking at the immediate ones. > > I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. > For example fence() and acquire() already include instruction stream synchronization on PPC. > Is that why you implemented the new barrier empty for non-x86 CPUs? Partly because wasn't sure e.g. isync was what you wanted. And maybe someone would choose to gamble to not take the performance hit. Or there may be other considering for that arch which I'm not aware of. (I wasn't aware of arms need to sync I-cache with D-cache, I thought ISB was enough) Thanks, Robbin > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of Robbin Ehn > Sent: Montag, 11. M?rz 2019 13:56 > To: Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > On 2019-03-11 11:26, Andrew Haley wrote: >> On 3/8/19 3:24 PM, Robbin Ehn wrote: >> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> After a JavaThread have been in a safepoint/(handshake) safe state >>> it can start executing updated code. E.g. an oop in the instruction >>> stream can have been updated. >> >> Hmm, interesting. I think it works on on AArch64 at present because >> the segfault trap we take is effectively a full synchronization >> operation. >> >> An AArch64 ISB (instruction synchronization barrier) invalidates the >> pipeline. It doesn't invalidate the instruction cache, which we'd need >> to do to see a relocated OOP. If an instruction is modified by some >> other thread we need to flush the local icache, but to do that we need >> to know which instructions have been changed. We could create a >> modification queue, but that seems rather elaborate. >> >> What is is that changes OOPs in the instructions during a handshake? >> Is it just garbage collection, or does it happen at other times too? > > They are only changed during a GC safepoint on archs with > mustIterateImmediateOopsInCode set to true, AFAIK. > Arm and aarch64 have it false. > > /Robbin > >> >> We could simply move OOPs (and class metadata pointers?) into the >> constant pool. That would have some performance impact on in-order >> CPUs, but hopefully not much on out-of-order ones. >> > > From martin.doerr at sap.com Tue Mar 12 11:47:01 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 12 Mar 2019 11:47:01 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> Message-ID: Hi Robbin, for PPC, isync came into my mind first (also mentioned by David). However, isync is only strictly required for a thread which performs modifications and wants to execute these modified instructions. We already have isync at the patch sites. The normal "fence" = "sync" does the job for cross-modification on PPC, so I don't think we need to execute any additional instructions. The existing barriers look fine for PPC. I believe other non-x86 platforms are also fine with existing fences (s390, haven't checked others). For x86, why don't we just "upgrade" the few existing barriers to include instruction stream synchronization? E.g. sharedRuntime_x86_64.cpp: 2564 templateInterpreterGenerator_x86.cpp: 1094 safepoint.cpp: 822 ... Am I missing anything? Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Dienstag, 12. M?rz 2019 10:44 To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code Hi Martin, On 3/11/19 6:43 PM, Doerr, Martin wrote: > Hi Robbin, > >> They are only changed during a GC safepoint on archs with >> mustIterateImmediateOopsInCode set to true, AFAIK. > > Only x86 uses Oops directly encoded into the instruction stream. > But all platforms have compressed Oops encoded: loadConN nodes in ad files. Sorry, forgot about that. Was just looking at the immediate ones. > > I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. > For example fence() and acquire() already include instruction stream synchronization on PPC. > Is that why you implemented the new barrier empty for non-x86 CPUs? Partly because wasn't sure e.g. isync was what you wanted. And maybe someone would choose to gamble to not take the performance hit. Or there may be other considering for that arch which I'm not aware of. (I wasn't aware of arms need to sync I-cache with D-cache, I thought ISB was enough) Thanks, Robbin > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of Robbin Ehn > Sent: Montag, 11. M?rz 2019 13:56 > To: Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > On 2019-03-11 11:26, Andrew Haley wrote: >> On 3/8/19 3:24 PM, Robbin Ehn wrote: >> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> After a JavaThread have been in a safepoint/(handshake) safe state >>> it can start executing updated code. E.g. an oop in the instruction >>> stream can have been updated. >> >> Hmm, interesting. I think it works on on AArch64 at present because >> the segfault trap we take is effectively a full synchronization >> operation. >> >> An AArch64 ISB (instruction synchronization barrier) invalidates the >> pipeline. It doesn't invalidate the instruction cache, which we'd need >> to do to see a relocated OOP. If an instruction is modified by some >> other thread we need to flush the local icache, but to do that we need >> to know which instructions have been changed. We could create a >> modification queue, but that seems rather elaborate. >> >> What is is that changes OOPs in the instructions during a handshake? >> Is it just garbage collection, or does it happen at other times too? > > They are only changed during a GC safepoint on archs with > mustIterateImmediateOopsInCode set to true, AFAIK. > Arm and aarch64 have it false. > > /Robbin > >> >> We could simply move OOPs (and class metadata pointers?) into the >> constant pool. That would have some performance impact on in-order >> CPUs, but hopefully not much on out-of-order ones. >> > > From robbin.ehn at oracle.com Tue Mar 12 12:56:17 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 12 Mar 2019 13:56:17 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> Message-ID: <3a6f12fd-194b-d5f5-a806-34b8d004b43b@oracle.com> Hi Martin, On 3/12/19 12:47 PM, Doerr, Martin wrote: > Hi Robbin, > > for PPC, isync came into my mind first (also mentioned by David). However, isync is only strictly required for a thread which performs modifications and wants to execute these modified instructions. We already have isync at the patch sites. > > The normal "fence" = "sync" does the job for cross-modification on PPC, so I don't think we need to execute any additional instructions. > The existing barriers look fine for PPC. > > I believe other non-x86 platforms are also fine with existing fences (s390, haven't checked others). > > For x86, why don't we just "upgrade" the few existing barriers to include instruction stream synchronization? > E.g. > sharedRuntime_x86_64.cpp: 2564 > templateInterpreterGenerator_x86.cpp: 1094 > safepoint.cpp: 822 Since at the moment don't have a way to ask for multiple fences in OrderAccess it's only available as a separate method for now. In the transition we have a StoreLoad between storing the unsafe thread state, e.g. native_trans and loading of the poll. But the instruction barrier must happen after poll. Otherwise an oop can be updated after the StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. JavaThread: | VMThread StoreLoad+InstructionPipeline| | update | disarm load thread_poll | Thanks, Robbin > ... > > Am I missing anything? > > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 12. M?rz 2019 10:44 > To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Martin, > > On 3/11/19 6:43 PM, Doerr, Martin wrote: >> Hi Robbin, >> >>> They are only changed during a GC safepoint on archs with >>> mustIterateImmediateOopsInCode set to true, AFAIK. >> >> Only x86 uses Oops directly encoded into the instruction stream. >> But all platforms have compressed Oops encoded: loadConN nodes in ad files. > > Sorry, forgot about that. Was just looking at the immediate ones. > >> >> I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. >> For example fence() and acquire() already include instruction stream synchronization on PPC. >> Is that why you implemented the new barrier empty for non-x86 CPUs? > > Partly because wasn't sure e.g. isync was what you wanted. > And maybe someone would choose to gamble to not take the performance hit. > Or there may be other considering for that arch which I'm not aware of. > (I wasn't aware of arms need to sync I-cache with D-cache, I thought ISB was enough) > > Thanks, Robbin > >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: hotspot-dev On Behalf Of Robbin Ehn >> Sent: Montag, 11. M?rz 2019 13:56 >> To: Andrew Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 2019-03-11 11:26, Andrew Haley wrote: >>> On 3/8/19 3:24 PM, Robbin Ehn wrote: >>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>> Changeset: >>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>> >>>> After a JavaThread have been in a safepoint/(handshake) safe state >>>> it can start executing updated code. E.g. an oop in the instruction >>>> stream can have been updated. >>> >>> Hmm, interesting. I think it works on on AArch64 at present because >>> the segfault trap we take is effectively a full synchronization >>> operation. >>> >>> An AArch64 ISB (instruction synchronization barrier) invalidates the >>> pipeline. It doesn't invalidate the instruction cache, which we'd need >>> to do to see a relocated OOP. If an instruction is modified by some >>> other thread we need to flush the local icache, but to do that we need >>> to know which instructions have been changed. We could create a >>> modification queue, but that seems rather elaborate. >>> >>> What is is that changes OOPs in the instructions during a handshake? >>> Is it just garbage collection, or does it happen at other times too? >> >> They are only changed during a GC safepoint on archs with >> mustIterateImmediateOopsInCode set to true, AFAIK. >> Arm and aarch64 have it false. >> >> /Robbin >> >>> >>> We could simply move OOPs (and class metadata pointers?) into the >>> constant pool. That would have some performance impact on in-order >>> CPUs, but hopefully not much on out-of-order ones. >>> >> >> From sgehwolf at redhat.com Tue Mar 12 13:29:13 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 12 Mar 2019 14:29:13 +0100 Subject: [Proposal] Better systemd slice memory limit support for OpenJDK In-Reply-To: <13644592-D61D-4911-8204-E3DC1A99A52C@oracle.com> References: <4F03B58D-FD8A-4ED1-B14A-C3769C573647@oracle.com> <9D22CF42-A701-48E2-9B27-3D862F187B6C@oracle.com> <7c1eb1cc2345ecf35dc3a5ddc4f69546f1d33df4.camel@redhat.com> <4dd64e3748220ea73a6aee50255dc5a94d07a2b1.camel@redhat.com> <13644592-D61D-4911-8204-E3DC1A99A52C@oracle.com> Message-ID: <7ab54354ffa7d5083472cbbf5f28ccd18b70ab08.camel@redhat.com> On Mon, 2019-03-11 at 15:58 -0400, Bob Vandette wrote: > > On Mar 11, 2019, at 3:46 PM, Severin Gehwolf wrote: > > > > Hi Bob, > > > > On Mon, 2019-03-11 at 14:26 -0400, Bob Vandette wrote: > > > Severin, > > > > > > I?m looking at other Docker and Container issues for inclusion in JDK 13 and > > > I?m trying to figure out if we should do anything about the issue you reported. > > > > > > If this problem is going away once the Linux distributions pick up Linux kernel > > > 4.19 or greater, then I?d suggest we just wait. > > > > What makes you think it does? That comment in Red Hat bugzilla seems > > wrong to me (if it was that). I'm on 4.20 in F29 and can still > > reproduce the issue. See attachment. > > These comments make it sound like it?s fixed in the Linux kernel version 4.19. > > > Jeremy Cline 2018-10-19 14:47:35 UTC > Hey folks, > > This looks to work as expected in the latest 4.19 kernels. F29 should be rebased to 4.19 in a couple weeks, and since it is actually enforced I don't think it's worth tracking down the commit that fixed this. > > > Jeremy Cline 2018-10-19 15:28:31 UTC > > This breaks automatic container detection in language runtimes (e.g. OpenJDK). It sizes it's structures according to the container limit. If it's not able to detect that it's in a container but actually is, then all bets are off. It looks to the user as if some application gets killed seemingly randomly without knowing why. So to that extent, I'm not sure I agree with "it's not worth tracking down the commit that fixed this". > > I'm happy to backport a patch if you or someone else wants to bisect it, but it's already fixed upstream and it'll be fixed in stable Fedora in a few weeks when we rebase to 4.19. Right. That was later refuted by: https://bugzilla.redhat.com/show_bug.cgi?id=1599387#c8 and confirmed it's not fixed by: https://bugzilla.redhat.com/show_bug.cgi?id=1599387#c9 > > > > It looks like support for Ubuntu 18.10 ends this July and Ubuntu 19.04 and Fedora 30 > > > are coming out next month (April). > > > > > > Do you agree? > > > > Not quite. I believe the fix is needed. It'll become more prevalent the > > more distros switch to newer kernels. > > > > It would be nice to confirm one way or the other. If we do fix this, the Metrics.java > file needs to be updated as well. Sure I'll include changes for Metrics.java once we have decided to move forward with this. > > Heads-up: There is going to be a push for cgroup v2 by default in > > Fedora[1]. At some point cgroup v2 will need to be supported by the > > JVM. It's on my plate to work on in the coming months. > > Wow, did they finally complete all of the subsystems that Docker needs or are they planning > on mixing v1 with v2? I don't know about that, sorry. Thanks, Severin > Sounds great, > Bob. > > Thanks, > > Severin > > > > [1] https://fedoraproject.org/wiki/Changes/CGroupsV2 > > > > > Bob. > > > > > > > > > > > > > On Jan 17, 2019, at 2:24 PM, Severin Gehwolf > > > > wrote: > > > > > > > > On Thu, 2019-01-17 at 13:11 -0500, Bob Vandette wrote: > > > > > Do you happen to know if this issue impacts Ubuntu 19.04? > > > > > > > > I don't, sorry. > > > > > > > > > Looking at our official OS support list, we?re only supporting > > > > > Ubuntu 18.04 with JDK 12 which > > > > > is using kernel version 4.15. Even if this gets bumped to 18.10, > > > > > this is still only using 4.18. > > > > > > > > I can confirm Ubuntu 18.04 doesn't seem to be affected with Kernel > > > > 4.15. RHEL-8 Beta, on the other hand, has 4.18 which is affected. > > > > I'm > > > > also trying Ubuntu 18.10 now, which, in theory, should be affected. > > > > > > > > > Go ahead and file a bug while we look into this issue a bit > > > > > more. I?d like to know the exact > > > > > kernel cutoff and which distros are impacted. If it?s wide- > > > > > spread on distros getting released > > > > > this year, we should fix it in JDK 13. > > > > > > > > OK. Here you go: https://bugs.openjdk.java.net/browse/JDK-8217338 > > > > > > > > Thanks, > > > > Severin > > > > > > > > > Bob. > > > > > > > > > > > > > > > > On Jan 17, 2019, at 11:16 AM, Severin Gehwolf < > > > > > > sgehwolf at redhat.com> > > > > > > wrote: > > > > > > > > > > > > Hi Bob, > > > > > > > > > > > > On Thu, 2019-01-17 at 10:59 -0500, Bob Vandette wrote: > > > > > > > I checked a few systems I have access to and they all have > > > > > > > use_hierarchy enabled. When I set a memory > > > > > > > limit the memory.stat hierarchical_memory_limit is identical > > > > > > > to > > > > > > > the memory.limit_in_bytes contents. > > > > > > > > > > > > This seems to suggest they're running on older kernels. I know > > > > > > F29, > > > > > > F28 > > > > > > are affected. > > > > > > > > > > > > > Do you have any idea why the kernel behavior changed? > > > > > > > > > > > > No, not really. It could be with the "unified control group > > > > > > hierarchy" > > > > > > work done in 3.16, but that's speculation: > > > > > > https://lwn.net/Articles/601840/ > > > > > > > > > > > > Some more discussion with systemd/kernel folk is here: > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1599387 > > > > > > > > > > > > > Is this documented behavior? > > > > > > > > > > > > Yes. Though, in the self-proclaimed very outdated document: > > > > > > https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt > > > > > > > > > > > > Section 6, "Hierarchy support". > > > > > > > > > > > > > I wouldn?t want to add a work-around for a transient kernel > > > > > > > bug. > > > > > > > > > > > > FWIW, cgroup-v1 is a legacy interface, so I doubt anything in > > > > > > that > > > > > > code > > > > > > has a great chance of getting "fixed". So far there is no sign > > > > > > of > > > > > > evidence what the expected behaviour should be. > > > > > > > > > > > > Thanks, > > > > > > Severin > > > > > > > > > > > > > Bob. > > > > > > > > > > > > > > > > > > > > > > On Jan 17, 2019, at 9:57 AM, Severin Gehwolf < > > > > > > > > sgehwolf at redhat.com> wrote: > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > Current container awareness for OpenJDK seems to work for > > > > > > > > systemd > > > > > > > > slices too, on some systems. To be precise, this works for > > > > > > > > older > > > > > > > > Kernels e.g. 3.10. However, we've noticed that this breaks > > > > > > > > for > > > > > > > > newer > > > > > > > > Kernel versions[1] such as the one in F28, currently > > > > > > > > 4.19.14- > > > > > > > > 200. If > > > > > > > > the container support would also look at hierarchical > > > > > > > > memory > > > > > > > > limits > > > > > > > > exposed via memory.stat in the cgroup file system, it would > > > > > > > > again > > > > > > > > work[2]. A proof of concept implementation is here: > > > > > > > > > > > > > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/webrev/ > > > > > > > > This enhancement wouldn't change any existing container > > > > > > > > memory > > > > > > > > limit > > > > > > > > detection as it only kicks in when all other look-ups > > > > > > > > determined that > > > > > > > > there is no limit in place. I've verified this by running > > > > > > > > Docker > > > > > > > > container tests. The idea is to look for > > > > > > > > hierarchical_memory_limit and > > > > > > > > hierarchical_memsw_limit lines in the memory.stat file of > > > > > > > > the > > > > > > > > cgroup > > > > > > > > tree. > > > > > > > > > > > > > > > > Would it be possible to consider such an enhancement > > > > > > > > upstream? > > > > > > > > If so, > > > > > > > > I'll file a bug and propose it for review. > > > > > > > > > > > > > > > > This issue has been originally raised here: > > > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1509371 > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Severin > > > > > > > > > > > > > > > > [1] Java process gets killed by oom killer. > > > > > > > > See: > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/before.txt > > > > > > > > [2] Java process throws OutOfMemoryError as expected. > > > > > > > > See: > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/after.txt > > > > > > > > > > From zgu at redhat.com Tue Mar 12 14:01:46 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 12 Mar 2019 10:01:46 -0400 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> <6a85b951-0ff7-7f90-63f5-7bb2b0bcc5e9@redhat.com> Message-ID: <0fbe6289-767d-9711-7c3a-13f51e63f928@redhat.com> >> >> Thanks, Zhengyu. I have two reviews now, and I think I need someone >> help me to push > > I can sponsor this patch. Passed submit and pushed. http://hg.openjdk.java.net/jdk/jdk/rev/ee53a278b9d8 -Zhengyu > > -Zhengyu > > >> >> Thanks, >> Ao Qi >> >>> Thanks, >>> >>> -Zhengyu >>> >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Ao Qi >>>>> >>>>> On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl >>>>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I found build is broken when configure with >>>>>>> --with-jvm-features=-g1gc,-jfr.? There are two problems: >>>>>>> >>>>>>> 1. g1BarrierSet.hpp is included when g1 is disabled. I think this >>>>>>> include can be removed. >>>>>>> 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef >>>>>>> INCLUDE_G1GC, cardTable.hpp is missed. >>>>>> >>>>>> (I noticed that too just yesterday when working on another change - >>>>>> thanks for fixing this). >>>>>> >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8220344 >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ >>>>>>> >>>>>>> >>>>>> >>>>>> ??? looks good to me. >>>>>> >>>>>> Thomas >>>>>> From aoqi at loongson.cn Tue Mar 12 14:24:30 2019 From: aoqi at loongson.cn (Ao Qi) Date: Tue, 12 Mar 2019 22:24:30 +0800 Subject: RFR: JDK-8220344: Build failures when using --with-jvm-features=-g1gc, -jfr In-Reply-To: <0fbe6289-767d-9711-7c3a-13f51e63f928@redhat.com> References: <15cbc76c8d1009abcd1a93c99782980ba1e2a03d.camel@oracle.com> <684552d8-1c52-5675-dfac-1475f525e5da@oracle.com> <6a85b951-0ff7-7f90-63f5-7bb2b0bcc5e9@redhat.com> <0fbe6289-767d-9711-7c3a-13f51e63f928@redhat.com> Message-ID: On Tue, Mar 12, 2019 at 10:01 PM Zhengyu Gu wrote: > > >> > >> Thanks, Zhengyu. I have two reviews now, and I think I need someone > >> help me to push > > > > I can sponsor this patch. > > Passed submit and pushed. > > http://hg.openjdk.java.net/jdk/jdk/rev/ee53a278b9d8 > Thank you, Zhengyu! > -Zhengyu > > > > > -Zhengyu > > > > > >> > >> Thanks, > >> Ao Qi > >> > >>> Thanks, > >>> > >>> -Zhengyu > >>> > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> Thanks, > >>>>> Ao Qi > >>>>> > >>>>> On Fri, Mar 8, 2019 at 7:39 PM Thomas Schatzl > >>>>> wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> On Fri, 2019-03-08 at 19:09 +0800, Ao Qi wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> I found build is broken when configure with > >>>>>>> --with-jvm-features=-g1gc,-jfr. There are two problems: > >>>>>>> > >>>>>>> 1. g1BarrierSet.hpp is included when g1 is disabled. I think this > >>>>>>> include can be removed. > >>>>>>> 2. cardTable.hpp is included in g1CardTable.hpp, and when ifndef > >>>>>>> INCLUDE_G1GC, cardTable.hpp is missed. > >>>>>> > >>>>>> (I noticed that too just yesterday when working on another change - > >>>>>> thanks for fixing this). > >>>>>> > >>>>>>> > >>>>>>> Bug: > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8220344 > >>>>>>> > >>>>>>> Webrev: > >>>>>>> http://cr.openjdk.java.net/~aoqi/8220344/webrev.00/ > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> looks good to me. > >>>>>> > >>>>>> Thomas > >>>>>> From kim.barrett at oracle.com Tue Mar 12 14:52:45 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 12 Mar 2019 10:52:45 -0400 Subject: RFR: 8217576: C1 atomic access handlers use incorrect decorators In-Reply-To: References: <721DCA80-9B21-4196-AE88-382AC94B5D5F@oracle.com> <612E7D43-FBFB-4D84-8EAB-515EB404F8D8@oracle.com> Message-ID: > On Mar 12, 2019, at 2:53 AM, Erik Osterlund wrote: > > Hi Kim, > > Looks good. > > Thanks, > /Erik Thanks. From martin.doerr at sap.com Tue Mar 12 14:57:43 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 12 Mar 2019 14:57:43 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <3a6f12fd-194b-d5f5-a806-34b8d004b43b@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <3a6f12fd-194b-d5f5-a806-34b8d004b43b@oracle.com> Message-ID: Hi Robbin, > In the transition we have a StoreLoad between storing the unsafe thread state, > e.g. native_trans and loading of the poll. But the instruction barrier must > happen after poll. Otherwise an oop can be updated after the > StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. Right, we'd need to make the disarm procedure more complicated to support this. E.g. VM thread could let Java threads which were in_native or in_native_trans disarm themselves with required barrier. I just wanted to share my idea to reduce performance impact. Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Dienstag, 12. M?rz 2019 13:56 To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) Subject: Re: RFR(m): 8220351: Cross-modifying code Hi Martin, On 3/12/19 12:47 PM, Doerr, Martin wrote: > Hi Robbin, > > for PPC, isync came into my mind first (also mentioned by David). However, isync is only strictly required for a thread which performs modifications and wants to execute these modified instructions. We already have isync at the patch sites. > > The normal "fence" = "sync" does the job for cross-modification on PPC, so I don't think we need to execute any additional instructions. > The existing barriers look fine for PPC. > > I believe other non-x86 platforms are also fine with existing fences (s390, haven't checked others). > > For x86, why don't we just "upgrade" the few existing barriers to include instruction stream synchronization? > E.g. > sharedRuntime_x86_64.cpp: 2564 > templateInterpreterGenerator_x86.cpp: 1094 > safepoint.cpp: 822 Since at the moment don't have a way to ask for multiple fences in OrderAccess it's only available as a separate method for now. In the transition we have a StoreLoad between storing the unsafe thread state, e.g. native_trans and loading of the poll. But the instruction barrier must happen after poll. Otherwise an oop can be updated after the StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. JavaThread: | VMThread StoreLoad+InstructionPipeline| | update | disarm load thread_poll | Thanks, Robbin > ... > > Am I missing anything? > > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 12. M?rz 2019 10:44 > To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Martin, > > On 3/11/19 6:43 PM, Doerr, Martin wrote: >> Hi Robbin, >> >>> They are only changed during a GC safepoint on archs with >>> mustIterateImmediateOopsInCode set to true, AFAIK. >> >> Only x86 uses Oops directly encoded into the instruction stream. >> But all platforms have compressed Oops encoded: loadConN nodes in ad files. > > Sorry, forgot about that. Was just looking at the immediate ones. > >> >> I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. >> For example fence() and acquire() already include instruction stream synchronization on PPC. >> Is that why you implemented the new barrier empty for non-x86 CPUs? > > Partly because wasn't sure e.g. isync was what you wanted. > And maybe someone would choose to gamble to not take the performance hit. > Or there may be other considering for that arch which I'm not aware of. > (I wasn't aware of arms need to sync I-cache with D-cache, I thought ISB was enough) > > Thanks, Robbin > >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: hotspot-dev On Behalf Of Robbin Ehn >> Sent: Montag, 11. M?rz 2019 13:56 >> To: Andrew Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 2019-03-11 11:26, Andrew Haley wrote: >>> On 3/8/19 3:24 PM, Robbin Ehn wrote: >>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>> Changeset: >>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>> >>>> After a JavaThread have been in a safepoint/(handshake) safe state >>>> it can start executing updated code. E.g. an oop in the instruction >>>> stream can have been updated. >>> >>> Hmm, interesting. I think it works on on AArch64 at present because >>> the segfault trap we take is effectively a full synchronization >>> operation. >>> >>> An AArch64 ISB (instruction synchronization barrier) invalidates the >>> pipeline. It doesn't invalidate the instruction cache, which we'd need >>> to do to see a relocated OOP. If an instruction is modified by some >>> other thread we need to flush the local icache, but to do that we need >>> to know which instructions have been changed. We could create a >>> modification queue, but that seems rather elaborate. >>> >>> What is is that changes OOPs in the instructions during a handshake? >>> Is it just garbage collection, or does it happen at other times too? >> >> They are only changed during a GC safepoint on archs with >> mustIterateImmediateOopsInCode set to true, AFAIK. >> Arm and aarch64 have it false. >> >> /Robbin >> >>> >>> We could simply move OOPs (and class metadata pointers?) into the >>> constant pool. That would have some performance impact on in-order >>> CPUs, but hopefully not much on out-of-order ones. >>> >> >> From tobias.hartmann at oracle.com Tue Mar 12 15:08:11 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 12 Mar 2019 16:08:11 +0100 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> Message-ID: <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> Hi Harold, so we are checking with StubRoutines::contains() if we are inside StubRoutines::unsafe_arraycopy(), right? What if StubRoutines::contains(pc) returns true but we actually hit a SIGBUS inside another stub? I also noticed that some changed lines have a broken indentation, for example, os_bsd_zero.cpp:197. Best regards, Tobias On 28.02.19 21:22, Harold Seigel wrote: > Hi, > > Please review this fix for JDK-8191278.? The fix uses the GuardUnsafeAccess class in unsafe.cpp so > that the signal handler will know to convert SIGBUS signals to exceptions. > > The compiler related changes and much of the test were written by Jamsheesh (jcm). > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/index.html > > JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8191278 > > The fix was tested on Linux x64, Mac OS, Solaris Sparc, and Windows.? Testing was not done on the > other platforms changed by this fix. > > The fix was regression tested by running Mach5 tiers 1 and 2 tests and builds on Linux x64, Solaris, > Windows, and Mac OS X, by running Mach5 tiers 3 -5 on Linux x64, and by running JCK-13 Lang and VM > tests on Linux x64. > > Thanks, Harold > From thomas.schatzl at oracle.com Tue Mar 12 16:32:00 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 12 Mar 2019 17:32:00 +0100 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: <3AB4F694-2289-4E4D-8B2B-7DE24E9C4C50@oracle.com> References: <3AB4F694-2289-4E4D-8B2B-7DE24E9C4C50@oracle.com> Message-ID: Hi, On Mon, 2019-03-11 at 15:29 -0400, Kim Barrett wrote: > > On Mar 8, 2019, at 8:51 AM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > > > Hi all, > > > > can I have reviews for this change the replaces the use of the JNI > > data type jbyte in the CardTable class by uint8_t via the custom > > CardValue typedef? > > > > This cleanup stems from trying to get rid of JNI data types in > > VM/GC code; formerly the main reason to use them here has been that > > the Atomic operations were not available for e.g. uint8_t, but that > > has been fixed for a long time. > > > > While the change itself is a no-op in practice, and testing showed > > so, I would like non-Oracle platform maintainers to try to compile > > the change and report problems because of lack of build platforms. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8220301 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8220301/webrev/ > > Testing: > > hs-tier1-5 > > > > Thanks, > > Thomas > > I'm concerned about changing the signed-ness of card table entries. > jbyte is signed. There may be code that expects or assumes signed > values. One place of particular concern would be assembly code > generators, where we might be generating sign-extending loads. > > Note that I didn't actually spot any signed-ness problems, but they > would be easy to miss. I haven't spotted anything either. > ------------------------------------------------------------------- > ----------- > src/hotspot/cpu/aarch64/gc/g1/g1BarrierSetAssembler_aarch64.cpp > 44 STATIC_ASSERT(sizeof(CardTable::CardValue) == 1); > > I think it is better to have assertions (static or otherwise) near > the code that depends on them for correctness. The old asserts > looked like they were in the right places, this replacement > STATIC_ASSERT is not. > > Similarly in lots of other places. > > And there are two top-level near the beggining of the file static > asserts here: > src/hotspot/cpu/arm/gc/g1/g1BarrierSetAssembler_arm.cpp > > An alternative would be to document and assert it once at the place > where CardValue is defined, and just say there's lots of code making > that assumption, so change at your peril. That would also address > Aleksey's complaint. I did the latter. I do not see an advantage to repeat the same assert basically everywhere. I added a note about what to look for to find the places easily. > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/cms/cmsCardTable.cpp > 230 CardValue* last_card_to_check = > 231 (CardValue*) MIN2((intptr_t) last_card_of_cur_chunk, > 232 (intptr_t) last_card_of_first_obj); > > pre-existing: Why all the casts here? This would work fine with all > the casts removed. > > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1DirtyCardQueue.hpp > 40 protected: > 41 typedef CardTable::CardValue CardValue; > > Not sure why this is being made protected rather than public. Public > seems more natural, since it's the declared type of a parameter of a > public function. I fixed the visibility here and in several other locations. > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1HotCardCache.cpp > 80 CardValue* previous_ptr = Atomic::cmpxchg(card_ptr, > > 81 &_hot_cache > [masked_index], > > 82 current_ptr > ); > > Argument indentation seems to have gotten messed up. > Fixed. > ------------------------------------------------------------------- > ----------- > src/hotspot/share/gc/g1/g1RemSet.[ch]hpp > > Seems like there are enough CardTable::CardValue uses in this class > to be worth a typedef. Added. http://cr.openjdk.java.net/~tschatzl/8220301/webrev.1_to_2/ (diff) http://cr.openjdk.java.net/~tschatzl/8220301/webrev.2/ (full) webrevs. Thanks, Thomas From kim.barrett at oracle.com Tue Mar 12 20:58:03 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 12 Mar 2019 16:58:03 -0400 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: References: <3AB4F694-2289-4E4D-8B2B-7DE24E9C4C50@oracle.com> Message-ID: <6BB424DC-7693-4271-A818-A490318595EB@oracle.com> > On Mar 12, 2019, at 12:32 PM, Thomas Schatzl wrote: > On Mon, 2019-03-11 at 15:29 -0400, Kim Barrett wrote: >> >> I'm concerned about changing the signed-ness of card table entries. >> jbyte is signed. There may be code that expects or assumes signed >> values. One place of particular concern would be assembly code >> generators, where we might be generating sign-extending loads. >> >> Note that I didn't actually spot any signed-ness problems, but they >> would be easy to miss. > > I haven't spotted anything either. I spent some more time looking for and not finding any. And tests are passing, so I guess I?ll stop worrying. > http://cr.openjdk.java.net/~tschatzl/8220301/webrev.1_to_2/ (diff) > http://cr.openjdk.java.net/~tschatzl/8220301/webrev.2/ (full) > > webrevs. > > Thanks, > Thomas Looks good. From jamsheed.c.m at oracle.com Wed Mar 13 05:48:57 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Wed, 13 Mar 2019 11:18:57 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> Message-ID: <18b4cfab-9304-e076-a94e-459d2ff92530@oracle.com> Hi Tobias On 12/03/19 8:38 PM, Tobias Hartmann wrote: > Hi Harold, > > so we are checking with StubRoutines::contains() if we are inside StubRoutines::unsafe_arraycopy(), > right? What if StubRoutines::contains(pc) returns true but we actually hit a SIGBUS inside another stub? Thank you for the review. from the comments it says "StubRoutines are used by Compiled code and runtime system". and StubRoutines::contains(pc) is expected be checked under code running in Java condition. missed adding is_Java check only in windows. later i had corrected it in fast failure handling code[1], but missed putting it in the code under review. i presumed the stub routine[2] code? sigbus memory access error could happen only in case of Unsafe. but will double check. Best regards, Jamsheed [1]http://cr.openjdk.java.net/~jcm/temp/fastfail.00/src/hotspot/os/windows/os_windows.cpp.udiff.html [2] Java stubRotuines code > > I also noticed that some changed lines have a broken indentation, for example, os_bsd_zero.cpp:197. > > Best regards, > Tobias > > On 28.02.19 21:22, Harold Seigel wrote: >> Hi, >> >> Please review this fix for JDK-8191278.? The fix uses the GuardUnsafeAccess class in unsafe.cpp so >> that the signal handler will know to convert SIGBUS signals to exceptions. >> >> The compiler related changes and much of the test were written by Jamsheesh (jcm). >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/index.html >> >> JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8191278 >> >> The fix was tested on Linux x64, Mac OS, Solaris Sparc, and Windows.? Testing was not done on the >> other platforms changed by this fix. >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests and builds on Linux x64, Solaris, >> Windows, and Mac OS X, by running Mach5 tiers 3 -5 on Linux x64, and by running JCK-13 Lang and VM >> tests on Linux x64. >> >> Thanks, Harold >> From robbin.ehn at oracle.com Wed Mar 13 08:19:15 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 13 Mar 2019 09:19:15 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> Hi David, On 3/12/19 9:00 AM, David Holmes wrote: > Hi Robbin, > > I have comments on the various changes below, but now I've gone through that all > I have a meta-comment on the overall approach. > > AFAICS the only time you have to sync the i-stream is when returning to > _thread_in_Java, as that is the only state in which you can execute the modified > code - no? Handling this at only those thread-state transitions would seem a lot > simpler/cleaner. I disagree, but I'll make such version after I published the update from this. I think the model of always 'refresh' the instruction stream when leaving a safepoint safe state is better. (but because of the S/R protocol and people do 'cheat' transition it becomes a bit clobberish) > > On 9/03/2019 1:24 am, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > src/hotspot/cpu/x86/assembler_x86.hpp > > As discussed elsewhere mfence() is no longer the advised mechanism, but instead > a serializing instruction like cpuid must be used. AMD64 "Upon release, the target thread must then execute a serializing instruction such as CPUID or MFENCE (a locked operation is not sufficient) before proceeding to the modified code to avoid executing a stale view of the instructions which may have been speculatively fetched." Intel "MFENCE does not serialize the instruction stream." CPUID seems to be common ground, changed. > src/hotspot/cpu/x86/macroAssembler_x86.cpp > > MacroAssembler::debug32 seems quite bizarre. First, pre-existing: Reverted. > > --- > > src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp > > 2562?? __ lock(); > 2563?? __ addl(Address(r15_thread, JavaThread::thread_state_offset()), 1 /* > _thread_in_native_trans */); > > This changes seems incorrect. You need a storeload barrier (at least) after the > update to thread_state but your lock() prefix would place barriers before I > think. In any case I find the existing code clearer and more obviously correct. We describe this as . So it is correct. Reverted. > > --- > > src/hotspot/cpu/x86/stubGenerator_x86_64.cpp > src/hotspot/os_cpu/windows_x86/orderAccess_windows_x86.hpp > src/hotspot/share/runtime/orderAccess.cpp > src/hotspot/share/runtime/stubRoutines.cpp > src/hotspot/share/runtime/stubRoutines.hpp > > I don't understand why we need a stub for this (or for fence() for that matter) ?? Visual studio don't support inline assembly on x64. Changing to CPUID. > > --- > > src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp > > 1090?? __ lock(); > 1091?? __ addl(Address(thread, JavaThread::thread_state_offset()), 1 > /*_thread_in_native_trans*/); > > Same comment as above. Reverted. > > --- > > src/hotspot/os_cpu/aix_ppc/orderAccess_aix_ppc.hpp > src/hotspot/os_cpu/linux_ppc/orderAccess_linux_ppc.hpp > > PPC has isync for instruction stream synchronization, so I'd expect to see that > used in OrderAccess::instruction_pipeline() I'll let PPC folks see what they need to do, but yes that was my thought also. > > --- > > src/hotspot/os_cpu/bsd_x86/orderAccess_bsd_x86.hpp > src/hotspot/os_cpu/linux_x86/orderAccess_linux_x86.hpp > src/hotspot/os_cpu/solaris_x86/orderAccess_solaris_x86.hpp > > s/mfence/cpuid Fixed. > > --- > > src/hotspot/os_cpu/linux_aarch64/orderAccess_linux_aarch64.hpp > src/hotspot/os_cpu/linux_arm/orderAccess_linux_arm.hpp > > Aarch64/ARM has ISB for instruction stream sync. As Haley said, arm do not have instruction cache coherency, so ISB is not enough in all cases. > > --- > > src/hotspot/os_cpu/linux_s390/orderAccess_linux_s390.hpp > > Need input from s390 folk. Yes. > > --- > > src/hotspot/os_cpu/linux_sparc/orderAccess_linux_sparc.hpp > src/hotspot/os_cpu/solaris_sparc/orderAccess_solaris_sparc.hpp > > SPARC needs a FLUSH > > "H.1.6 Self-Modifying Code > If a program includes self-modifying code, it must issue a FLUSH instruction for > each modified doubleword of instructions (or a call to supervisor software > having an equivalent effect)." [SPARCV9 Architecture manual] > Yes. > --- > > src/hotspot/share/classfile/classFileParser.cpp > > Change is unrelated to this bug. It's related that without it 32-bit don't compile (without disable-errors was warning). So I need it for test compiling. Want me to revert it? > > --- > > src/hotspot/share/prims/jvmtiEnv.cpp > > The changes seem unnecessary. As the code comment states we are going from one > safepoint unsafe state to another. There is no way to execute any JIT'd code > before returning to Java, and that return is where we should sync if needed. You are correct. > > --- > > src/hotspot/share/prims/jvmtiExport.cpp > > The change is unnecessary as we are executing in the VMThread. Not only would it > be the one doing the code changes at a safepoint, it never executes compiled > Java code anyway. Yes. > > --- > > src/hotspot/share/runtime/interfaceSupport.inline.hpp > src/hotspot/share/runtime/safepoint.cpp > > My initial thought was that the OrderAccess::instruction_pipeline() should be > buried in the safepoint blocking code, but in some cases we may not have > actually blocked at the safepoint and so (theoretically at least) may need to > sync with the i-stream in case an intervening safepoint changed some code. Can't > help but worry that unconditionally doing this on native transitions is going to > hurt JNI performance. It also looks like we will double up on the > instruction_pipeline() in some cases. Yes. I had an idea with passing around a token for it, but it just clobbers the code more. > > --- > > src/hotspot/share/runtime/thread.cpp > > 1839?? OrderAccess::instruction_pipeline(); > > Can't see why this is necessary here, we should sync when returning to > _thread_in_Java. In this changeset we always do the barrier when leaving a safe state. When you are not on threads list you are safe, so you can go new->vm->java just missing the safepoint. > > 2471?? if (thread_state() != _thread_blocked) { > 2472???? OrderAccess::instruction_pipeline(); > 2473?? } > > Again not sure why this is necessary. The call to java_suspend_self in the > not-blocked case comes from JavaThread::handle_special_runtime_exit_condition, > which itself is called from the transition code, so I'd expect this to be done > in the transition code. java_suspend_self is executed in multiple different states. Since you are safe when your have suspend flag set, we must emit the barrier when leaving that safe state. Transitions from unsafe to unsafe e.g. vm to java do not emit the barrier. > > ???? thread->set_thread_state(_thread_blocked); > ???? thread->java_suspend_self(); > ???? thread->set_thread_state(state); > +???? OrderAccess::instruction_pipeline(); > > Again seems unnecessary - especially given you have one in java_suspend_self() > already! (Notwithstanding I think that one should be removed.) The above one is if we are not blocked, in this case we are blocked. The reason for this is that if we are blocked via a real transition, the transition does the instruction barrier. If you don't mind an extra barrier we can emit it unconditionally in java_suspend_self. I think set_thread_state should be private and just the ThreadStateTransistion should be allowed to use set_thread_state() :) Posting update to RFR when completed. /Robbin > > Thanks, > David > ----- > >> After a JavaThread have been in a safepoint/(handshake) safe state it can start >> executing updated code. E.g. an oop in the instruction stream can have been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is far away to >> guarantee that the thread executing the updated instruction stream sees the >> modification. >> >> What far away is and how far an update instruction stream is from a safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From aph at redhat.com Wed Mar 13 09:10:10 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 13 Mar 2019 09:10:10 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> Message-ID: On 3/11/19 5:43 PM, Doerr, Martin wrote: > Hi Robbin, > >> They are only changed during a GC safepoint on archs with >> mustIterateImmediateOopsInCode set to true, AFAIK. > > Only x86 uses Oops directly encoded into the instruction stream. I'm pretty sure that AArch64 uses oops in the instruction stream. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Wed Mar 13 09:30:12 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 13 Mar 2019 09:30:12 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> Message-ID: Hi Andrew, > I'm pretty sure that AArch64 uses oops in the instruction stream. Yes, as compressed oops. (See my previous statement "But all platforms have compressed Oops encoded: loadConN nodes in ad files.".) Best regards, Martin -----Original Message----- From: Andrew Haley Sent: Mittwoch, 13. M?rz 2019 10:10 To: Doerr, Martin ; Robbin Ehn ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code On 3/11/19 5:43 PM, Doerr, Martin wrote: > Hi Robbin, > >> They are only changed during a GC safepoint on archs with >> mustIterateImmediateOopsInCode set to true, AFAIK. > > Only x86 uses Oops directly encoded into the instruction stream. I'm pretty sure that AArch64 uses oops in the instruction stream. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Mar 13 09:39:15 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 13 Mar 2019 09:39:15 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> Message-ID: <6f1c90b6-2b40-01bc-6ce2-7ae6a0d23efc@redhat.com> On 3/13/19 9:30 AM, Doerr, Martin wrote: >> I'm pretty sure that AArch64 uses oops in the instruction stream. > Yes, as compressed oops. Even when compressed oops are off. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From robbin.ehn at oracle.com Wed Mar 13 09:45:15 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 13 Mar 2019 10:45:15 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <6f1c90b6-2b40-01bc-6ce2-7ae6a0d23efc@redhat.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <6f1c90b6-2b40-01bc-6ce2-7ae6a0d23efc@redhat.com> Message-ID: <2e44bcad-e9fe-f474-00d9-110096c606f7@oracle.com> On 2019-03-13 10:39, Andrew Haley wrote: > On 3/13/19 9:30 AM, Doerr, Martin wrote: > >>> I'm pretty sure that AArch64 uses oops in the instruction stream. >> Yes, as compressed oops. > > Even when compressed oops are off. > Since aarch64 have mustIterateImmediateOopsInCode false they are not walked during GC in nmethod::oops_do(), what am I missing ? /Robbin From martin.doerr at sap.com Wed Mar 13 10:19:36 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 13 Mar 2019 10:19:36 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <2e44bcad-e9fe-f474-00d9-110096c606f7@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <6f1c90b6-2b40-01bc-6ce2-7ae6a0d23efc@redhat.com> <2e44bcad-e9fe-f474-00d9-110096c606f7@oracle.com> Message-ID: Hi Robbin, mustIterateImmediateOopsInCode means that nmethods may contain oops in the instruction stream which are not in the nmethod's separately recorded oops section. This only occurs on x86. Other platforms use the oop recorder for such oops so they can be found there, too. So they can be referenced by the nmethod twice: in the separate section and in the instruction stream. This may be the case on aarch64 (not sure). Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Mittwoch, 13. M?rz 2019 10:45 To: Andrew Haley ; Doerr, Martin ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code On 2019-03-13 10:39, Andrew Haley wrote: > On 3/13/19 9:30 AM, Doerr, Martin wrote: > >>> I'm pretty sure that AArch64 uses oops in the instruction stream. >> Yes, as compressed oops. > > Even when compressed oops are off. > Since aarch64 have mustIterateImmediateOopsInCode false they are not walked during GC in nmethod::oops_do(), what am I missing ? /Robbin From aph at redhat.com Wed Mar 13 10:48:40 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 13 Mar 2019 10:48:40 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <6f1c90b6-2b40-01bc-6ce2-7ae6a0d23efc@redhat.com> <2e44bcad-e9fe-f474-00d9-110096c606f7@oracle.com> Message-ID: Hi, On 3/13/19 10:19 AM, Doerr, Martin wrote: > mustIterateImmediateOopsInCode means that nmethods may contain oops > in the instruction stream which are not in the nmethod's separately > recorded oops section. This only occurs on x86. > Other platforms use the oop recorder for such oops so they can be > found there, too. So they can be referenced by the nmethod twice: in > the separate section and in the instruction stream. This may be the > case on aarch64 (not sure). That's right. In more detail, mustIterateImmediateOopsInCode assumes that inline oops can be patched with simple stores, i.e. that an immediate move is an instruction followed by a separately-addressible field that contains the word-sized address. On AArch64, paching of oops is done by void oop_Relocation::fix_oop_relocation() { if (!oop_is_immediate()) { // get the oop from the pool, and re-insert it into the instruction: set_value(value()); | v DataRelocation::set_value | v Relocation::pd_set_data_value | v MacroAssembler::patch_oop -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Wed Mar 13 11:03:31 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 13 Mar 2019 11:03:31 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <3a6f12fd-194b-d5f5-a806-34b8d004b43b@oracle.com> Message-ID: Hi Robbin, I'm not so much concerned about the performance impact on PPC if we have to add isync instructions. But seems like the performance impact on x86 is so significant that we should find a way to move the CPUID instruction out of the most performance critical path. Do you think it would be feasible to do the following? 1. Skip disarm for threads running in_native (or in_native_trans). safepoint.cpp:480 2. Execute instruction fence in JavaThread::check_special_condition_for_native_trans(_and_transition). We should probably skip the rest if the safepoint is over. Best regards, Martin -----Original Message----- From: Doerr, Martin Sent: Dienstag, 12. M?rz 2019 15:58 To: 'Robbin Ehn' ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) Subject: RE: RFR(m): 8220351: Cross-modifying code Hi Robbin, > In the transition we have a StoreLoad between storing the unsafe thread state, > e.g. native_trans and loading of the poll. But the instruction barrier must > happen after poll. Otherwise an oop can be updated after the > StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. Right, we'd need to make the disarm procedure more complicated to support this. E.g. VM thread could let Java threads which were in_native or in_native_trans disarm themselves with required barrier. I just wanted to share my idea to reduce performance impact. Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Dienstag, 12. M?rz 2019 13:56 To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) Subject: Re: RFR(m): 8220351: Cross-modifying code Hi Martin, On 3/12/19 12:47 PM, Doerr, Martin wrote: > Hi Robbin, > > for PPC, isync came into my mind first (also mentioned by David). However, isync is only strictly required for a thread which performs modifications and wants to execute these modified instructions. We already have isync at the patch sites. > > The normal "fence" = "sync" does the job for cross-modification on PPC, so I don't think we need to execute any additional instructions. > The existing barriers look fine for PPC. > > I believe other non-x86 platforms are also fine with existing fences (s390, haven't checked others). > > For x86, why don't we just "upgrade" the few existing barriers to include instruction stream synchronization? > E.g. > sharedRuntime_x86_64.cpp: 2564 > templateInterpreterGenerator_x86.cpp: 1094 > safepoint.cpp: 822 Since at the moment don't have a way to ask for multiple fences in OrderAccess it's only available as a separate method for now. In the transition we have a StoreLoad between storing the unsafe thread state, e.g. native_trans and loading of the poll. But the instruction barrier must happen after poll. Otherwise an oop can be updated after the StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. JavaThread: | VMThread StoreLoad+InstructionPipeline| | update | disarm load thread_poll | Thanks, Robbin > ... > > Am I missing anything? > > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 12. M?rz 2019 10:44 > To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Martin, > > On 3/11/19 6:43 PM, Doerr, Martin wrote: >> Hi Robbin, >> >>> They are only changed during a GC safepoint on archs with >>> mustIterateImmediateOopsInCode set to true, AFAIK. >> >> Only x86 uses Oops directly encoded into the instruction stream. >> But all platforms have compressed Oops encoded: loadConN nodes in ad files. > > Sorry, forgot about that. Was just looking at the immediate ones. > >> >> I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. >> For example fence() and acquire() already include instruction stream synchronization on PPC. >> Is that why you implemented the new barrier empty for non-x86 CPUs? > > Partly because wasn't sure e.g. isync was what you wanted. > And maybe someone would choose to gamble to not take the performance hit. > Or there may be other considering for that arch which I'm not aware of. > (I wasn't aware of arms need to sync I-cache with D-cache, I thought ISB was enough) > > Thanks, Robbin > >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: hotspot-dev On Behalf Of Robbin Ehn >> Sent: Montag, 11. M?rz 2019 13:56 >> To: Andrew Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 2019-03-11 11:26, Andrew Haley wrote: >>> On 3/8/19 3:24 PM, Robbin Ehn wrote: >>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>> Changeset: >>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>> >>>> After a JavaThread have been in a safepoint/(handshake) safe state >>>> it can start executing updated code. E.g. an oop in the instruction >>>> stream can have been updated. >>> >>> Hmm, interesting. I think it works on on AArch64 at present because >>> the segfault trap we take is effectively a full synchronization >>> operation. >>> >>> An AArch64 ISB (instruction synchronization barrier) invalidates the >>> pipeline. It doesn't invalidate the instruction cache, which we'd need >>> to do to see a relocated OOP. If an instruction is modified by some >>> other thread we need to flush the local icache, but to do that we need >>> to know which instructions have been changed. We could create a >>> modification queue, but that seems rather elaborate. >>> >>> What is is that changes OOPs in the instructions during a handshake? >>> Is it just garbage collection, or does it happen at other times too? >> >> They are only changed during a GC safepoint on archs with >> mustIterateImmediateOopsInCode set to true, AFAIK. >> Arm and aarch64 have it false. >> >> /Robbin >> >>> >>> We could simply move OOPs (and class metadata pointers?) into the >>> constant pool. That would have some performance impact on in-order >>> CPUs, but hopefully not much on out-of-order ones. >>> >> >> From robbin.ehn at oracle.com Wed Mar 13 11:07:20 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 13 Mar 2019 12:07:20 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <6f1c90b6-2b40-01bc-6ce2-7ae6a0d23efc@redhat.com> <2e44bcad-e9fe-f474-00d9-110096c606f7@oracle.com> Message-ID: <015dc950-3e9c-1ac6-7bf9-f5448351207f@oracle.com> On 2019-03-13 11:48, Andrew Haley wrote: > Hi, > > On 3/13/19 10:19 AM, Doerr, Martin wrote: > >> mustIterateImmediateOopsInCode means that nmethods may contain oops >> in the instruction stream which are not in the nmethod's separately >> recorded oops section. This only occurs on x86. > >> Other platforms use the oop recorder for such oops so they can be >> found there, too. So they can be referenced by the nmethod twice: in >> the separate section and in the instruction stream. This may be the >> case on aarch64 (not sure). > > That's right. In more detail, mustIterateImmediateOopsInCode assumes > that inline oops can be patched with simple stores, i.e. that an > immediate move is an instruction followed by a separately-addressible > field that contains the word-sized address. Thanks sparc have the same issue as arm, with "flush
". Adding an indirection to these, so we don't have the oops in the instruction stream, as you suggested, seems like the sane thing to do. But costly. /Robbin > > On AArch64, paching of oops is done by > > void oop_Relocation::fix_oop_relocation() { > if (!oop_is_immediate()) { > // get the oop from the pool, and re-insert it into the instruction: > set_value(value()); > > | > v > > DataRelocation::set_value > > | > v > > Relocation::pd_set_data_value > > | > v > > MacroAssembler::patch_oop > From aph at redhat.com Wed Mar 13 11:15:03 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 13 Mar 2019 11:15:03 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <015dc950-3e9c-1ac6-7bf9-f5448351207f@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <6f1c90b6-2b40-01bc-6ce2-7ae6a0d23efc@redhat.com> <2e44bcad-e9fe-f474-00d9-110096c606f7@oracle.com> <015dc950-3e9c-1ac6-7bf9-f5448351207f@oracle.com> Message-ID: On 3/13/19 11:07 AM, Robbin Ehn wrote: > sparc have the same issue as arm, with "flush
". > Adding an indirection to these, so we don't have the oops in the instruction > stream, as you suggested, seems like the sane thing to do. But costly. It depends, I think. In an out-of-order processor with speculation, we shouldn't see the 4- or 5-cycle latency for the load from L1 cache. I guess it would hit cache utilization, though. It'd be an interesting experiment. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.schatzl at oracle.com Wed Mar 13 11:28:09 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 13 Mar 2019 12:28:09 +0100 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: <6BB424DC-7693-4271-A818-A490318595EB@oracle.com> References: <3AB4F694-2289-4E4D-8B2B-7DE24E9C4C50@oracle.com> <6BB424DC-7693-4271-A818-A490318595EB@oracle.com> Message-ID: <69938522998f9978af967eed686d72a64ead3f39.camel@oracle.com> Hi Kim, On Tue, 2019-03-12 at 16:58 -0400, Kim Barrett wrote: > > On Mar 12, 2019, at 12:32 PM, Thomas Schatzl < > > thomas.schatzl at oracle.com> wrote: > > On Mon, 2019-03-11 at 15:29 -0400, Kim Barrett wrote: > > > > > > I'm concerned about changing the signed-ness of card table > > > entries. > > > jbyte is signed. There may be code that expects or assumes > > > signed > > > values. One place of particular concern would be assembly code > > > generators, where we might be generating sign-extending loads. > > > > > > Note that I didn't actually spot any signed-ness problems, but > > > they > > > would be easy to miss. > > > > I haven't spotted anything either. > > I spent some more time looking for and not finding any. And tests > are passing, so I guess I?ll stop worrying. > > > http://cr.openjdk.java.net/~tschatzl/8220301/webrev.1_to_2/ (diff) > > http://cr.openjdk.java.net/~tschatzl/8220301/webrev.2/ (full) > > > > webrevs. > > > > Thanks, > > Thomas > > Looks good. > thanks for your review. Assuming that Aleksey is okay with the changes (there were no substantial ones functionality-wise) I will push today then so that we will see issues before weekend. If it slips for any reason, I will wait until Monday. Thomas From robbin.ehn at oracle.com Wed Mar 13 11:41:58 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 13 Mar 2019 12:41:58 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <3a6f12fd-194b-d5f5-a806-34b8d004b43b@oracle.com> Message-ID: <90746bfd-dc43-66e7-2489-5b22be911e16@oracle.com> Hi Martin, On 2019-03-13 12:03, Doerr, Martin wrote: > Hi Robbin, > > I'm not so much concerned about the performance impact on PPC if we have to add isync instructions. But seems like the performance impact on x86 is so significant that we should find a way to move the CPUID instruction out of the most performance critical path. > > Do you think it would be feasible to do the following? > 1. Skip disarm for threads running in_native (or in_native_trans). safepoint.cpp:480 > 2. Execute instruction fence in JavaThread::check_special_condition_for_native_trans(_and_transition). We should probably skip the rest if the safepoint is over. I think maybe letting the JavaThread disarm itself and check for safepoint/handshake after disarm so we didn't disarm the next safepoint/handshakes. So only the first transition back from native after a safepoint/handshake is 'slow'. And only safepoints/handshakes that modify the instruction stream can leave them armed for native threads. I think that is doable, I'll experiment with that, thanks! /Robbin > > Best regards, > Martin > > > -----Original Message----- > From: Doerr, Martin > Sent: Dienstag, 12. M?rz 2019 15:58 > To: 'Robbin Ehn' ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) > Subject: RE: RFR(m): 8220351: Cross-modifying code > > Hi Robbin, > >> In the transition we have a StoreLoad between storing the unsafe thread state, >> e.g. native_trans and loading of the poll. But the instruction barrier must >> happen after poll. Otherwise an oop can be updated after the >> StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. > > Right, we'd need to make the disarm procedure more complicated to support this. > E.g. VM thread could let Java threads which were in_native or in_native_trans disarm themselves with required barrier. > > I just wanted to share my idea to reduce performance impact. > > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 12. M?rz 2019 13:56 > To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Martin, > > On 3/12/19 12:47 PM, Doerr, Martin wrote: >> Hi Robbin, >> >> for PPC, isync came into my mind first (also mentioned by David). However, isync is only strictly required for a thread which performs modifications and wants to execute these modified instructions. We already have isync at the patch sites. >> >> The normal "fence" = "sync" does the job for cross-modification on PPC, so I don't think we need to execute any additional instructions. >> The existing barriers look fine for PPC. >> >> I believe other non-x86 platforms are also fine with existing fences (s390, haven't checked others). >> >> For x86, why don't we just "upgrade" the few existing barriers to include instruction stream synchronization? >> E.g. >> sharedRuntime_x86_64.cpp: 2564 >> templateInterpreterGenerator_x86.cpp: 1094 >> safepoint.cpp: 822 > > Since at the moment don't have a way to ask for multiple fences in OrderAccess > it's only available as a separate method for now. > > In the transition we have a StoreLoad between storing the unsafe thread state, > e.g. native_trans and loading of the poll. But the instruction barrier must > happen after poll. Otherwise an oop can be updated after the > StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. > > JavaThread: | VMThread > StoreLoad+InstructionPipeline| > | update > | disarm > load thread_poll | > > Thanks, Robbin > >> ... >> >> Am I missing anything? >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Dienstag, 12. M?rz 2019 10:44 >> To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Martin, >> >> On 3/11/19 6:43 PM, Doerr, Martin wrote: >>> Hi Robbin, >>> >>>> They are only changed during a GC safepoint on archs with >>>> mustIterateImmediateOopsInCode set to true, AFAIK. >>> >>> Only x86 uses Oops directly encoded into the instruction stream. >>> But all platforms have compressed Oops encoded: loadConN nodes in ad files. >> >> Sorry, forgot about that. Was just looking at the immediate ones. >> >>> >>> I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. >>> For example fence() and acquire() already include instruction stream synchronization on PPC. >>> Is that why you implemented the new barrier empty for non-x86 CPUs? >> >> Partly because wasn't sure e.g. isync was what you wanted. >> And maybe someone would choose to gamble to not take the performance hit. >> Or there may be other considering for that arch which I'm not aware of. >> (I wasn't aware of arms need to sync I-cache with D-cache, I thought ISB was enough) >> >> Thanks, Robbin >> >>> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: hotspot-dev On Behalf Of Robbin Ehn >>> Sent: Montag, 11. M?rz 2019 13:56 >>> To: Andrew Haley ; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> On 2019-03-11 11:26, Andrew Haley wrote: >>>> On 3/8/19 3:24 PM, Robbin Ehn wrote: >>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>>> Changeset: >>>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>>> >>>>> After a JavaThread have been in a safepoint/(handshake) safe state >>>>> it can start executing updated code. E.g. an oop in the instruction >>>>> stream can have been updated. >>>> >>>> Hmm, interesting. I think it works on on AArch64 at present because >>>> the segfault trap we take is effectively a full synchronization >>>> operation. >>>> >>>> An AArch64 ISB (instruction synchronization barrier) invalidates the >>>> pipeline. It doesn't invalidate the instruction cache, which we'd need >>>> to do to see a relocated OOP. If an instruction is modified by some >>>> other thread we need to flush the local icache, but to do that we need >>>> to know which instructions have been changed. We could create a >>>> modification queue, but that seems rather elaborate. >>>> >>>> What is is that changes OOPs in the instructions during a handshake? >>>> Is it just garbage collection, or does it happen at other times too? >>> >>> They are only changed during a GC safepoint on archs with >>> mustIterateImmediateOopsInCode set to true, AFAIK. >>> Arm and aarch64 have it false. >>> >>> /Robbin >>> >>>> >>>> We could simply move OOPs (and class metadata pointers?) into the >>>> constant pool. That would have some performance impact on in-order >>>> CPUs, but hopefully not much on out-of-order ones. >>>> >>> >>> From martin.doerr at sap.com Wed Mar 13 13:37:55 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 13 Mar 2019 13:37:55 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <90746bfd-dc43-66e7-2489-5b22be911e16@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <0b0a861a-f780-424e-6574-4baf29ef03e0@oracle.com> <3a6f12fd-194b-d5f5-a806-34b8d004b43b@oracle.com> <90746bfd-dc43-66e7-2489-5b22be911e16@oracle.com> Message-ID: Hi Robbin, > I think maybe letting the JavaThread disarm itself and check for > safepoint/handshake after disarm so we didn't disarm the next > safepoint/handshakes Yes, you understand what I mean. I guess one additional C call for each thread which returns from native and didn't notice one or more safepoints should be affordable from performance point of view. If that was a concern, VM thread could perform the disarm for simple handshakes because oops in instruction stream can only get patched when all Java threads are at safepoint AFAIK. Thank you very much for trying this. Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Mittwoch, 13. M?rz 2019 12:42 To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) Subject: Re: RFR(m): 8220351: Cross-modifying code Hi Martin, On 2019-03-13 12:03, Doerr, Martin wrote: > Hi Robbin, > > I'm not so much concerned about the performance impact on PPC if we have to add isync instructions. But seems like the performance impact on x86 is so significant that we should find a way to move the CPUID instruction out of the most performance critical path. > > Do you think it would be feasible to do the following? > 1. Skip disarm for threads running in_native (or in_native_trans). safepoint.cpp:480 > 2. Execute instruction fence in JavaThread::check_special_condition_for_native_trans(_and_transition). We should probably skip the rest if the safepoint is over. I think maybe letting the JavaThread disarm itself and check for safepoint/handshake after disarm so we didn't disarm the next safepoint/handshakes. So only the first transition back from native after a safepoint/handshake is 'slow'. And only safepoints/handshakes that modify the instruction stream can leave them armed for native threads. I think that is doable, I'll experiment with that, thanks! /Robbin > > Best regards, > Martin > > > -----Original Message----- > From: Doerr, Martin > Sent: Dienstag, 12. M?rz 2019 15:58 > To: 'Robbin Ehn' ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) > Subject: RE: RFR(m): 8220351: Cross-modifying code > > Hi Robbin, > >> In the transition we have a StoreLoad between storing the unsafe thread state, >> e.g. native_trans and loading of the poll. But the instruction barrier must >> happen after poll. Otherwise an oop can be updated after the >> StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. > > Right, we'd need to make the disarm procedure more complicated to support this. > E.g. VM thread could let Java threads which were in_native or in_native_trans disarm themselves with required barrier. > > I just wanted to share my idea to reduce performance impact. > > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 12. M?rz 2019 13:56 > To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net; David Holmes (david.holmes at oracle.com) > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Martin, > > On 3/12/19 12:47 PM, Doerr, Martin wrote: >> Hi Robbin, >> >> for PPC, isync came into my mind first (also mentioned by David). However, isync is only strictly required for a thread which performs modifications and wants to execute these modified instructions. We already have isync at the patch sites. >> >> The normal "fence" = "sync" does the job for cross-modification on PPC, so I don't think we need to execute any additional instructions. >> The existing barriers look fine for PPC. >> >> I believe other non-x86 platforms are also fine with existing fences (s390, haven't checked others). >> >> For x86, why don't we just "upgrade" the few existing barriers to include instruction stream synchronization? >> E.g. >> sharedRuntime_x86_64.cpp: 2564 >> templateInterpreterGenerator_x86.cpp: 1094 >> safepoint.cpp: 822 > > Since at the moment don't have a way to ask for multiple fences in OrderAccess > it's only available as a separate method for now. > > In the transition we have a StoreLoad between storing the unsafe thread state, > e.g. native_trans and loading of the poll. But the instruction barrier must > happen after poll. Otherwise an oop can be updated after the > StoreLoad+InstructionPipeline barrier and poll being disarmed before poll check. > > JavaThread: | VMThread > StoreLoad+InstructionPipeline| > | update > | disarm > load thread_poll | > > Thanks, Robbin > >> ... >> >> Am I missing anything? >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Dienstag, 12. M?rz 2019 10:44 >> To: Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Martin, >> >> On 3/11/19 6:43 PM, Doerr, Martin wrote: >>> Hi Robbin, >>> >>>> They are only changed during a GC safepoint on archs with >>>> mustIterateImmediateOopsInCode set to true, AFAIK. >>> >>> Only x86 uses Oops directly encoded into the instruction stream. >>> But all platforms have compressed Oops encoded: loadConN nodes in ad files. >> >> Sorry, forgot about that. Was just looking at the immediate ones. >> >>> >>> I wonder if there are already barriers in place which have the required effect for non-x86 CPUs. >>> For example fence() and acquire() already include instruction stream synchronization on PPC. >>> Is that why you implemented the new barrier empty for non-x86 CPUs? >> >> Partly because wasn't sure e.g. isync was what you wanted. >> And maybe someone would choose to gamble to not take the performance hit. >> Or there may be other considering for that arch which I'm not aware of. >> (I wasn't aware of arms need to sync I-cache with D-cache, I thought ISB was enough) >> >> Thanks, Robbin >> >>> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: hotspot-dev On Behalf Of Robbin Ehn >>> Sent: Montag, 11. M?rz 2019 13:56 >>> To: Andrew Haley ; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> On 2019-03-11 11:26, Andrew Haley wrote: >>>> On 3/8/19 3:24 PM, Robbin Ehn wrote: >>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>>> Changeset: >>>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>>> >>>>> After a JavaThread have been in a safepoint/(handshake) safe state >>>>> it can start executing updated code. E.g. an oop in the instruction >>>>> stream can have been updated. >>>> >>>> Hmm, interesting. I think it works on on AArch64 at present because >>>> the segfault trap we take is effectively a full synchronization >>>> operation. >>>> >>>> An AArch64 ISB (instruction synchronization barrier) invalidates the >>>> pipeline. It doesn't invalidate the instruction cache, which we'd need >>>> to do to see a relocated OOP. If an instruction is modified by some >>>> other thread we need to flush the local icache, but to do that we need >>>> to know which instructions have been changed. We could create a >>>> modification queue, but that seems rather elaborate. >>>> >>>> What is is that changes OOPs in the instructions during a handshake? >>>> Is it just garbage collection, or does it happen at other times too? >>> >>> They are only changed during a GC safepoint on archs with >>> mustIterateImmediateOopsInCode set to true, AFAIK. >>> Arm and aarch64 have it false. >>> >>> /Robbin >>> >>>> >>>> We could simply move OOPs (and class metadata pointers?) into the >>>> constant pool. That would have some performance impact on in-order >>>> CPUs, but hopefully not much on out-of-order ones. >>>> >>> >>> From martin.doerr at sap.com Wed Mar 13 17:56:34 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 13 Mar 2019 17:56:34 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> Message-ID: Hi everybody, while discussing with a colleague the following question about the patch site on x86 came up: nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. I couldn't find a call to AbstractICache::invalidate_range. Does anybody know where it is or if it's missing? Best regards, Martin From shade at redhat.com Wed Mar 13 19:05:59 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 13 Mar 2019 20:05:59 +0100 Subject: RFR (M): 8220301: Remove jbyte use in CardTable (affects all platforms) In-Reply-To: <69938522998f9978af967eed686d72a64ead3f39.camel@oracle.com> References: <3AB4F694-2289-4E4D-8B2B-7DE24E9C4C50@oracle.com> <6BB424DC-7693-4271-A818-A490318595EB@oracle.com> <69938522998f9978af967eed686d72a64ead3f39.camel@oracle.com> Message-ID: On 3/13/19 12:28 PM, Thomas Schatzl wrote: >>> http://cr.openjdk.java.net/~tschatzl/8220301/webrev.1_to_2/ (diff) >>> http://cr.openjdk.java.net/~tschatzl/8220301/webrev.2/ (full) > Assuming that Aleksey is okay with the changes (there were no > substantial ones functionality-wise) I will push today then so that we > will see issues before weekend. If it slips for any reason, I will wait > until Monday. I have no problems with this patch. Spot-checked hotspot still builds on aarch64, ppc64, s390x, x86, x86_64-zero, x86_64-minimal. -Aleksey From robbin.ehn at oracle.com Wed Mar 13 19:29:44 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 13 Mar 2019 20:29:44 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> Message-ID: <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Hi, On 2019-03-13 18:56, Doerr, Martin wrote: > Hi everybody, > > while discussing with a colleague the following question about the patch site on x86 came up: > > nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. > I couldn't find a call to AbstractICache::invalidate_range. Look for ICache::invalidate_range, I see 11 calls on x64. And one call to AbstractICache/ICache::invalidate_word. Btw I have spent some time looking at x86/icache_x86.cpp... But that is whole new subject and let's keep that separate from this :) I'll create issues for some of these topics, which if nothing else should be discussed. /Robbin > > Does anybody know where it is or if it's missing? > > Best regards, > Martin > From kim.barrett at oracle.com Wed Mar 13 20:07:56 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 13 Mar 2019 16:07:56 -0400 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs Message-ID: Please review this change to the JNI specification. The specified behavior of Weak Global References, and in particular their relationship to Java PhantomReference, is being updated to account for a change to PhantomReference by JDK-8071507. CR: https://bugs.openjdk.java.net/browse/JDK-8188066 CSR: https://bugs.openjdk.java.net/browse/JDK-8220617 Changes: http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1.html specdiff for the changes. In addition to updating the section describing Weak Global References, the description of NewGlobalRef is updated, as well as the copyright footer. http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.md Copies of the before and after text for the JNI Weak Global References section; easier to read than diffs or specdiff. From kim.barrett at oracle.com Wed Mar 13 20:29:43 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 13 Mar 2019 16:29:43 -0400 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: References: Message-ID: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> > On Mar 13, 2019, at 4:07 PM, Kim Barrett wrote: > > Please review this change to the JNI specification. The specified > behavior of Weak Global References, and in particular their > relationship to Java PhantomReference, is being updated to account > for a change to PhantomReference by JDK-8071507. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8188066 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8220617 > > Changes: > http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1.html > specdiff for the changes. In addition to updating the section > describing Weak Global References, the description of NewGlobalRef > is updated, as well as the copyright footer. > > http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.md > Copies of the before and after text for the JNI Weak Global > References section; easier to read than diffs or specdiff. I botched the upload of the changes to cr.openjdk.java.net. The corrected URLS are: http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1/diff.html http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.html From mandy.chung at oracle.com Wed Mar 13 21:00:06 2019 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 13 Mar 2019 14:00:06 -0700 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> References: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> Message-ID: <7a6f0e8a-801f-8976-1d65-986b36a3af7d@oracle.com> On 3/13/19 1:29 PM, Kim Barrett wrote: >> On Mar 13, 2019, at 4:07 PM, Kim Barrett wrote: >> >> Please review this change to the JNI specification. The specified >> behavior of Weak Global References, and in particular their >> relationship to Java PhantomReference, is being updated to account >> for a change to PhantomReference by JDK-8071507. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8188066 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8220617 Reviewed. Please either attach the spec diff or inline the spec in the CSR for archiving purpose. This spec update aligning the weak global refs with phantom reference is great and thanks for rewriting spec. Reads well. Mandy From per.liden at oracle.com Wed Mar 13 21:32:09 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 13 Mar 2019 22:32:09 +0100 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> References: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> Message-ID: On 03/13/2019 09:29 PM, Kim Barrett wrote: >> On Mar 13, 2019, at 4:07 PM, Kim Barrett wrote: >> >> Please review this change to the JNI specification. The specified >> behavior of Weak Global References, and in particular their >> relationship to Java PhantomReference, is being updated to account >> for a change to PhantomReference by JDK-8071507. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8188066 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8220617 >> >> Changes: >> http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1.html >> specdiff for the changes. In addition to updating the section >> describing Weak Global References, the description of NewGlobalRef >> is updated, as well as the copyright footer. >> >> http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.md >> Copies of the before and after text for the JNI Weak Global >> References section; easier to read than diffs or specdiff. Clear and unambiguous, nice! Reviewed. /Per From kim.barrett at oracle.com Wed Mar 13 23:02:13 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 13 Mar 2019 19:02:13 -0400 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> References: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> Message-ID: <6F6648D3-2536-4047-B73B-0A0411960F49@oracle.com> > On Mar 13, 2019, at 4:29 PM, Kim Barrett wrote: > >> On Mar 13, 2019, at 4:07 PM, Kim Barrett wrote: >> >> Please review this change to the JNI specification. The specified >> behavior of Weak Global References, and in particular their >> relationship to Java PhantomReference, is being updated to account >> for a change to PhantomReference by JDK-8071507. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8188066 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8220617 >> >> Changes: >> http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1.html >> specdiff for the changes. In addition to updating the section >> describing Weak Global References, the description of NewGlobalRef >> is updated, as well as the copyright footer. >> >> http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.md >> Copies of the before and after text for the JNI Weak Global >> References section; easier to read than diffs or specdiff. > > I botched the upload of the changes to cr.openjdk.java.net. The corrected URLS are: > > http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1/diff.html > http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.html Stefan Karlsson suggested (offline) a couple of wording changes in the description of Weak Global References. (1) Replace "destroyed" with "deleted" in two places, to align with the names of DeleteGlobalRef and the like. (2) Reword the second sentence of the fourth paragraph, as there is some potential for ambiguity around "If the references are the same". Here's the diff between rev1 and those changes. I'll post an updated specdiff later, once the dust settles. diff -r 76b82c006509 -r 6a2181c4fee2 closed/src/java.se/share/specs/jni/functions.md --- a/closed/src/java.se/share/specs/jni/functions.md Wed Mar 13 15:40:33 2019 -0400 +++ b/closed/src/java.se/share/specs/jni/functions.md Wed Mar 13 17:06:23 2019 -0400 @@ -1177,9 +1177,9 @@ to `NULL` without notice. `IsSameObject` can be used to compare a weak global reference to a non-`NULL` -local or global reference. If the references are the same, the weak global -reference's underlying object will not be freed so long as the other reference -has not been destroyed. +local or global reference. If the objects are the same, the weak global +reference won't become functionally equivalent to `NULL` so long as the other +reference has not been deleted. `IsSameObject` can also be used to compare a weak global reference to `NULL` to determine whether the underlying object has been freed. @@ -1193,7 +1193,7 @@ `NewGlobalRef`. These functions will return `NULL` if the object has been freed. Otherwise, the new reference will prevent the underlying object from being freed. The new reference, if non-`NULL`, can then be used to access the -underlying object, and destroyed when such access is no longer needed. +underlying object, and deleted when such access is no longer needed. ------------------------------------------------------------------------------- From kim.barrett at oracle.com Wed Mar 13 23:03:00 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 13 Mar 2019 19:03:00 -0400 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: <7a6f0e8a-801f-8976-1d65-986b36a3af7d@oracle.com> References: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> <7a6f0e8a-801f-8976-1d65-986b36a3af7d@oracle.com> Message-ID: > On Mar 13, 2019, at 5:00 PM, Mandy Chung wrote: > > > > On 3/13/19 1:29 PM, Kim Barrett wrote: >>> On Mar 13, 2019, at 4:07 PM, Kim Barrett wrote: >>> >>> Please review this change to the JNI specification. The specified >>> behavior of Weak Global References, and in particular their >>> relationship to Java PhantomReference, is being updated to account >>> for a change to PhantomReference by JDK-8071507. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8188066 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8220617 > > Reviewed. Please either attach the spec diff or inline the spec > in the CSR for archiving purpose. > > This spec update aligning the weak global refs with phantom reference > is great and thanks for rewriting spec. Reads well. > > Mandy Thanks. I?ll attach the final specdiff to the CSR once the review is complete. From kim.barrett at oracle.com Wed Mar 13 23:03:14 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 13 Mar 2019 19:03:14 -0400 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: References: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> Message-ID: > On Mar 13, 2019, at 5:32 PM, Per Liden wrote: > > > On 03/13/2019 09:29 PM, Kim Barrett wrote: >>> On Mar 13, 2019, at 4:07 PM, Kim Barrett wrote: >>> >>> Please review this change to the JNI specification. The specified >>> behavior of Weak Global References, and in particular their >>> relationship to Java PhantomReference, is being updated to account >>> for a change to PhantomReference by JDK-8071507. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8188066 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8220617 >>> >>> Changes: >>> http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1.html >>> specdiff for the changes. In addition to updating the section >>> describing Weak Global References, the description of NewGlobalRef >>> is updated, as well as the copyright footer. >>> >>> http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.md >>> Copies of the before and after text for the JNI Weak Global >>> References section; easier to read than diffs or specdiff. > > Clear and unambiguous, nice! > > Reviewed. > > /Per Thanks. From kim.barrett at oracle.com Wed Mar 13 23:07:32 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 13 Mar 2019 19:07:32 -0400 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: References: Message-ID: > On Mar 13, 2019, at 6:19 PM, Hans Boehm wrote: > > Looks great to me! Thanks. > Just to be clear, this is intended to guarantee that it's not possible to promote a weak global reference that indirectly points to A after a PhantomReference or a WeakGlobalReference to A has been cleared? Thus this reference clearing must happen more or less atomically. Yes. That?s the point of saying the jweak becomes functionally equivalent to NULL at the same time as a PhantomReference for the same object would be cleared. > I think this property has always been needed for the intended use of PhantomReferences, since otherwise the referent of a cleared PhantomReference could be resurrected. It's great to get it in the spec. Exactly. That?s the ?important safety invariant? mentioned in the CSR. From jamsheed.c.m at oracle.com Thu Mar 14 01:29:29 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Thu, 14 Mar 2019 06:59:29 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> Message-ID: Hi Tobias, i will try to limit my checks to array copy stubs. will make changes and send it for review. Best regards, Jamsheed On 12/03/19 8:38 PM, Tobias Hartmann wrote: > Hi Harold, > > so we are checking with StubRoutines::contains() if we are inside StubRoutines::unsafe_arraycopy(), > right? What if StubRoutines::contains(pc) returns true but we actually hit a SIGBUS inside another stub? > > I also noticed that some changed lines have a broken indentation, for example, os_bsd_zero.cpp:197. > > Best regards, > Tobias > > On 28.02.19 21:22, Harold Seigel wrote: >> Hi, >> >> Please review this fix for JDK-8191278.? The fix uses the GuardUnsafeAccess class in unsafe.cpp so >> that the signal handler will know to convert SIGBUS signals to exceptions. >> >> The compiler related changes and much of the test were written by Jamsheesh (jcm). >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8191278/webrev/index.html >> >> JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8191278 >> >> The fix was tested on Linux x64, Mac OS, Solaris Sparc, and Windows.? Testing was not done on the >> other platforms changed by this fix. >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests and builds on Linux x64, Solaris, >> Windows, and Mac OS X, by running Mach5 tiers 3 -5 on Linux x64, and by running JCK-13 Lang and VM >> tests on Linux x64. >> >> Thanks, Harold >> From david.holmes at oracle.com Thu Mar 14 04:40:21 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Mar 2019 14:40:21 +1000 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: <6F6648D3-2536-4047-B73B-0A0411960F49@oracle.com> References: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> <6F6648D3-2536-4047-B73B-0A0411960F49@oracle.com> Message-ID: <4a589284-4e4d-5d4c-7441-6d00a9725fe3@oracle.com> Looks good. Thanks Kim! David ----- On 14/03/2019 9:02 am, Kim Barrett wrote: >> On Mar 13, 2019, at 4:29 PM, Kim Barrett wrote: >> >>> On Mar 13, 2019, at 4:07 PM, Kim Barrett wrote: >>> >>> Please review this change to the JNI specification. The specified >>> behavior of Weak Global References, and in particular their >>> relationship to Java PhantomReference, is being updated to account >>> for a change to PhantomReference by JDK-8071507. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8188066 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8220617 >>> >>> Changes: >>> http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1.html >>> specdiff for the changes. In addition to updating the section >>> describing Weak Global References, the description of NewGlobalRef >>> is updated, as well as the copyright footer. >>> >>> http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.md >>> Copies of the before and after text for the JNI Weak Global >>> References section; easier to read than diffs or specdiff. >> >> I botched the upload of the changes to cr.openjdk.java.net. The corrected URLS are: >> >> http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1/diff.html >> http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.html > > Stefan Karlsson suggested (offline) a couple of wording changes in the > description of Weak Global References. > > (1) Replace "destroyed" with "deleted" in two places, to align with the > names of DeleteGlobalRef and the like. > > (2) Reword the second sentence of the fourth paragraph, as there is some > potential for ambiguity around "If the references are the same". > > Here's the diff between rev1 and those changes. I'll post an updated > specdiff later, once the dust settles. > > diff -r 76b82c006509 -r 6a2181c4fee2 closed/src/java.se/share/specs/jni/functions.md > --- a/closed/src/java.se/share/specs/jni/functions.md Wed Mar 13 15:40:33 2019 -0400 > +++ b/closed/src/java.se/share/specs/jni/functions.md Wed Mar 13 17:06:23 2019 -0400 > @@ -1177,9 +1177,9 @@ > to `NULL` without notice. > > `IsSameObject` can be used to compare a weak global reference to a non-`NULL` > -local or global reference. If the references are the same, the weak global > -reference's underlying object will not be freed so long as the other reference > -has not been destroyed. > +local or global reference. If the objects are the same, the weak global > +reference won't become functionally equivalent to `NULL` so long as the other > +reference has not been deleted. > > `IsSameObject` can also be used to compare a weak global reference to `NULL` > to determine whether the underlying object has been freed. > @@ -1193,7 +1193,7 @@ > `NewGlobalRef`. These functions will return `NULL` if the object has been > freed. Otherwise, the new reference will prevent the underlying object from > being freed. The new reference, if non-`NULL`, can then be used to access the > -underlying object, and destroyed when such access is no longer needed. > +underlying object, and deleted when such access is no longer needed. > > ------------------------------------------------------------------------------- > > From david.holmes at oracle.com Thu Mar 14 04:48:36 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Mar 2019 14:48:36 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> Message-ID: <050f8f40-2a6a-9a5e-5920-70498b2d109e@oracle.com> On 13/03/2019 6:19 pm, Robbin Ehn wrote: > Hi David, > > On 3/12/19 9:00 AM, David Holmes wrote: >> Hi Robbin, >> >> I have comments on the various changes below, but now I've gone >> through that all I have a meta-comment on the overall approach. >> >> AFAICS the only time you have to sync the i-stream is when returning >> to _thread_in_Java, as that is the only state in which you can execute >> the modified code - no? Handling this at only those thread-state >> transitions would seem a lot simpler/cleaner. > > I disagree, but I'll make such version after I published the update from > this. > I think the model of always 'refresh' the instruction stream when leaving a > safepoint safe state is better. (but because of the S/R protocol and > people do 'cheat' transition it becomes a bit clobberish) I don't see how it is better to cover cases that do not need to be covered. _thread_blocked is safepoint-safe while _thread_in_VM is not, but you don't need to sync with the i-stream when unblocking and returning to the VM! Only transitions back to Java need to do this. You then want to optimize things so that you only sync when needed. But lets see how this plays out after you disarm experiments Thanks, David >> >> On 9/03/2019 1:24 am, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> src/hotspot/cpu/x86/assembler_x86.hpp >> >> As discussed elsewhere mfence() is no longer the advised mechanism, >> but instead a serializing instruction like cpuid must be used. > > AMD64 > "Upon release, the target thread must then execute a serializing > instruction > such as CPUID or MFENCE (a locked operation is not sufficient) before > proceeding > to the modified code to avoid executing a stale view of the instructions > which > may have been speculatively fetched." > > Intel > "MFENCE does not serialize the instruction stream." > > CPUID seems to be common ground, changed. > >> src/hotspot/cpu/x86/macroAssembler_x86.cpp >> >> MacroAssembler::debug32 seems quite bizarre. First, pre-existing: > > Reverted. > > >> >> --- >> >> src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp >> >> 2562?? __ lock(); >> 2563?? __ addl(Address(r15_thread, JavaThread::thread_state_offset()), >> 1 /* _thread_in_native_trans */); >> >> This changes seems incorrect. You need a storeload barrier (at least) >> after the update to thread_state but your lock() prefix would place >> barriers before I think. In any case I find the existing code clearer >> and more obviously correct. > > We describe this as . > So it is correct. > > Reverted. > > >> >> --- >> >> src/hotspot/cpu/x86/stubGenerator_x86_64.cpp >> src/hotspot/os_cpu/windows_x86/orderAccess_windows_x86.hpp >> src/hotspot/share/runtime/orderAccess.cpp >> src/hotspot/share/runtime/stubRoutines.cpp >> src/hotspot/share/runtime/stubRoutines.hpp >> >> I don't understand why we need a stub for this (or for fence() for >> that matter) ?? > > Visual studio don't support inline assembly on x64. > Changing to CPUID. > >> >> --- >> >> src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp >> >> 1090?? __ lock(); >> 1091?? __ addl(Address(thread, JavaThread::thread_state_offset()), 1 >> /*_thread_in_native_trans*/); >> >> Same comment as above. > > Reverted. > >> >> --- >> >> src/hotspot/os_cpu/aix_ppc/orderAccess_aix_ppc.hpp >> src/hotspot/os_cpu/linux_ppc/orderAccess_linux_ppc.hpp >> >> PPC has isync for instruction stream synchronization, so I'd expect to >> see that used in OrderAccess::instruction_pipeline() > > I'll let PPC folks see what they need to do, but yes that was my thought > also. > >> >> --- >> >> src/hotspot/os_cpu/bsd_x86/orderAccess_bsd_x86.hpp >> src/hotspot/os_cpu/linux_x86/orderAccess_linux_x86.hpp >> src/hotspot/os_cpu/solaris_x86/orderAccess_solaris_x86.hpp >> >> s/mfence/cpuid > > Fixed. > >> >> --- >> >> src/hotspot/os_cpu/linux_aarch64/orderAccess_linux_aarch64.hpp >> src/hotspot/os_cpu/linux_arm/orderAccess_linux_arm.hpp >> >> Aarch64/ARM has ISB for instruction stream sync. > > As Haley said, arm do not have instruction cache coherency, so ISB is > not enough in all cases. > >> >> --- >> >> src/hotspot/os_cpu/linux_s390/orderAccess_linux_s390.hpp >> >> Need input from s390 folk. > > Yes. > >> >> --- >> >> src/hotspot/os_cpu/linux_sparc/orderAccess_linux_sparc.hpp >> src/hotspot/os_cpu/solaris_sparc/orderAccess_solaris_sparc.hpp >> >> SPARC needs a FLUSH >> >> "H.1.6 Self-Modifying Code >> If a program includes self-modifying code, it must issue a FLUSH >> instruction for each modified doubleword of instructions (or a call to >> supervisor software having an equivalent effect)." [SPARCV9 >> Architecture manual] >> > > Yes. > >> --- >> >> src/hotspot/share/classfile/classFileParser.cpp >> >> Change is unrelated to this bug. > > It's related that without it 32-bit don't compile (without > disable-errors was warning). So I need it for test compiling. Want me to > revert it? > >> >> --- >> >> src/hotspot/share/prims/jvmtiEnv.cpp >> >> The changes seem unnecessary. As the code comment states we are going >> from one safepoint unsafe state to another. There is no way to execute >> any JIT'd code before returning to Java, and that return is where we >> should sync if needed. > > You are correct. > >> >> --- >> >> src/hotspot/share/prims/jvmtiExport.cpp >> >> The change is unnecessary as we are executing in the VMThread. Not >> only would it be the one doing the code changes at a safepoint, it >> never executes compiled Java code anyway. > > Yes. > >> >> --- >> >> src/hotspot/share/runtime/interfaceSupport.inline.hpp >> src/hotspot/share/runtime/safepoint.cpp >> >> My initial thought was that the OrderAccess::instruction_pipeline() >> should be buried in the safepoint blocking code, but in some cases we >> may not have actually blocked at the safepoint and so (theoretically >> at least) may need to sync with the i-stream in case an intervening >> safepoint changed some code. Can't help but worry that unconditionally >> doing this on native transitions is going to hurt JNI performance. It >> also looks like we will double up on the instruction_pipeline() in >> some cases. > > Yes. I had an idea with passing around a token for it, but it just > clobbers the code more. > >> >> --- >> >> src/hotspot/share/runtime/thread.cpp >> >> 1839?? OrderAccess::instruction_pipeline(); >> >> Can't see why this is necessary here, we should sync when returning to >> _thread_in_Java. > > In this changeset we always do the barrier when leaving a safe state. > When you are not on threads list you are safe, so you can go > new->vm->java just missing the safepoint. > >> >> 2471?? if (thread_state() != _thread_blocked) { >> 2472???? OrderAccess::instruction_pipeline(); >> 2473?? } >> >> Again not sure why this is necessary. The call to java_suspend_self in >> the not-blocked case comes from >> JavaThread::handle_special_runtime_exit_condition, which itself is >> called from the transition code, so I'd expect this to be done in the >> transition code. > > java_suspend_self is executed in multiple different states. > Since you are safe when your have suspend flag set, we must emit the > barrier > when leaving that safe state. Transitions from unsafe to unsafe e.g. vm > to java > do not emit the barrier. > >> >> ????? thread->set_thread_state(_thread_blocked); >> ????? thread->java_suspend_self(); >> ????? thread->set_thread_state(state); >> +???? OrderAccess::instruction_pipeline(); >> >> Again seems unnecessary - especially given you have one in >> java_suspend_self() already! (Notwithstanding I think that one should >> be removed.) > > The above one is if we are not blocked, in this case we are blocked. > The reason for this is that if we are blocked via a real transition, the > transition does the instruction barrier. If you don't mind an extra > barrier we can emit it unconditionally in java_suspend_self. > > I think set_thread_state should be private and just the > ThreadStateTransistion should be allowed to use set_thread_state() :) > > Posting update to RFR when completed. > > /Robbin > >> >> Thanks, >> David >> ----- >> >>> After a JavaThread have been in a safepoint/(handshake) safe state it >>> can start >>> executing updated code. E.g. an oop in the instruction stream can >>> have been >>> updated. >>> >>> Most hardware's require a barrier or that the code cross modified is >>> far away to >>> guarantee that the thread executing the updated instruction stream >>> sees the >>> modification. >>> >>> What far away is and how far an update instruction stream is from a >>> safepoint >>> safe state is not clear. >>> >>> To be compliant with those architectures an instruction stream >>> barrier must be >>> added when leaving the safepoint safe state. >>> >>> There may be crashes today due to this missing barrier. >>> A new CPU with deeper pipeline or changes to the VM which moves a >>> safepoint safe >>> state closer to a nmethod can increase changes of a crash. >>> >>> A few benchmarks will see a performance regression with the extra >>> serializing, >>> such as Octane-Crypto -5% (x86). >>> >>> With a much more elaborate fix, such as changing the local poll to >>> have more >>> than two states (armed/disarmed), it would be possible to only emit such >>> instruction stream barrier when the poll have been armed while the >>> thread was >>> safepoint safe. >>> >>> Passes t1-3 on our platforms, and test built what I can. >>> >>> Thanks, Robbin From robbin.ehn at oracle.com Thu Mar 14 08:24:51 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 14 Mar 2019 09:24:51 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <050f8f40-2a6a-9a5e-5920-70498b2d109e@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <050f8f40-2a6a-9a5e-5920-70498b2d109e@oracle.com> Message-ID: Hi David, On 3/14/19 5:48 AM, David Holmes wrote: > I don't see how it is better to cover cases that do not need to be covered. > _thread_blocked is safepoint-safe while _thread_in_VM is not, but you don't need > to sync with the i-stream when unblocking and returning to the VM! Only > transitions back to Java need to do this. You then want to optimize things so > that you only sync when needed. I see it the other way around. If you transition java->vm->java, you are never safe so you don't need the barrier. Introducing more cost to the blocked state (and the orthogonal suspended state) I don't see any problems with and don't need any optimizations. While the java->vm->java would need such. /Robbin > > But lets see how this plays out after you disarm experiments > > Thanks, > David > >>> >>> On 9/03/2019 1:24 am, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>> Changeset: >>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> src/hotspot/cpu/x86/assembler_x86.hpp >>> >>> As discussed elsewhere mfence() is no longer the advised mechanism, but >>> instead a serializing instruction like cpuid must be used. >> >> AMD64 >> "Upon release, the target thread must then execute a serializing instruction >> such as CPUID or MFENCE (a locked operation is not sufficient) before proceeding >> to the modified code to avoid executing a stale view of the instructions which >> may have been speculatively fetched." >> >> Intel >> "MFENCE does not serialize the instruction stream." >> >> CPUID seems to be common ground, changed. >> >>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>> >>> MacroAssembler::debug32 seems quite bizarre. First, pre-existing: >> >> Reverted. >> >> >>> >>> --- >>> >>> src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp >>> >>> 2562?? __ lock(); >>> 2563?? __ addl(Address(r15_thread, JavaThread::thread_state_offset()), 1 /* >>> _thread_in_native_trans */); >>> >>> This changes seems incorrect. You need a storeload barrier (at least) after >>> the update to thread_state but your lock() prefix would place barriers before >>> I think. In any case I find the existing code clearer and more obviously >>> correct. >> >> We describe this as . >> So it is correct. >> >> Reverted. >> >> >>> >>> --- >>> >>> src/hotspot/cpu/x86/stubGenerator_x86_64.cpp >>> src/hotspot/os_cpu/windows_x86/orderAccess_windows_x86.hpp >>> src/hotspot/share/runtime/orderAccess.cpp >>> src/hotspot/share/runtime/stubRoutines.cpp >>> src/hotspot/share/runtime/stubRoutines.hpp >>> >>> I don't understand why we need a stub for this (or for fence() for that >>> matter) ?? >> >> Visual studio don't support inline assembly on x64. >> Changing to CPUID. >> >>> >>> --- >>> >>> src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp >>> >>> 1090?? __ lock(); >>> 1091?? __ addl(Address(thread, JavaThread::thread_state_offset()), 1 >>> /*_thread_in_native_trans*/); >>> >>> Same comment as above. >> >> Reverted. >> >>> >>> --- >>> >>> src/hotspot/os_cpu/aix_ppc/orderAccess_aix_ppc.hpp >>> src/hotspot/os_cpu/linux_ppc/orderAccess_linux_ppc.hpp >>> >>> PPC has isync for instruction stream synchronization, so I'd expect to see >>> that used in OrderAccess::instruction_pipeline() >> >> I'll let PPC folks see what they need to do, but yes that was my thought also. >> >>> >>> --- >>> >>> src/hotspot/os_cpu/bsd_x86/orderAccess_bsd_x86.hpp >>> src/hotspot/os_cpu/linux_x86/orderAccess_linux_x86.hpp >>> src/hotspot/os_cpu/solaris_x86/orderAccess_solaris_x86.hpp >>> >>> s/mfence/cpuid >> >> Fixed. >> >>> >>> --- >>> >>> src/hotspot/os_cpu/linux_aarch64/orderAccess_linux_aarch64.hpp >>> src/hotspot/os_cpu/linux_arm/orderAccess_linux_arm.hpp >>> >>> Aarch64/ARM has ISB for instruction stream sync. >> >> As Haley said, arm do not have instruction cache coherency, so ISB is not >> enough in all cases. >> >>> >>> --- >>> >>> src/hotspot/os_cpu/linux_s390/orderAccess_linux_s390.hpp >>> >>> Need input from s390 folk. >> >> Yes. >> >>> >>> --- >>> >>> src/hotspot/os_cpu/linux_sparc/orderAccess_linux_sparc.hpp >>> src/hotspot/os_cpu/solaris_sparc/orderAccess_solaris_sparc.hpp >>> >>> SPARC needs a FLUSH >>> >>> "H.1.6 Self-Modifying Code >>> If a program includes self-modifying code, it must issue a FLUSH instruction >>> for each modified doubleword of instructions (or a call to supervisor >>> software having an equivalent effect)." [SPARCV9 Architecture manual] >>> >> >> Yes. >> >>> --- >>> >>> src/hotspot/share/classfile/classFileParser.cpp >>> >>> Change is unrelated to this bug. >> >> It's related that without it 32-bit don't compile (without disable-errors was >> warning). So I need it for test compiling. Want me to revert it? >> >>> >>> --- >>> >>> src/hotspot/share/prims/jvmtiEnv.cpp >>> >>> The changes seem unnecessary. As the code comment states we are going from >>> one safepoint unsafe state to another. There is no way to execute any JIT'd >>> code before returning to Java, and that return is where we should sync if >>> needed. >> >> You are correct. >> >>> >>> --- >>> >>> src/hotspot/share/prims/jvmtiExport.cpp >>> >>> The change is unnecessary as we are executing in the VMThread. Not only would >>> it be the one doing the code changes at a safepoint, it never executes >>> compiled Java code anyway. >> >> Yes. >> >>> >>> --- >>> >>> src/hotspot/share/runtime/interfaceSupport.inline.hpp >>> src/hotspot/share/runtime/safepoint.cpp >>> >>> My initial thought was that the OrderAccess::instruction_pipeline() should be >>> buried in the safepoint blocking code, but in some cases we may not have >>> actually blocked at the safepoint and so (theoretically at least) may need to >>> sync with the i-stream in case an intervening safepoint changed some code. >>> Can't help but worry that unconditionally doing this on native transitions is >>> going to hurt JNI performance. It also looks like we will double up on the >>> instruction_pipeline() in some cases. >> >> Yes. I had an idea with passing around a token for it, but it just clobbers >> the code more. >> >>> >>> --- >>> >>> src/hotspot/share/runtime/thread.cpp >>> >>> 1839?? OrderAccess::instruction_pipeline(); >>> >>> Can't see why this is necessary here, we should sync when returning to >>> _thread_in_Java. >> >> In this changeset we always do the barrier when leaving a safe state. >> When you are not on threads list you are safe, so you can go new->vm->java >> just missing the safepoint. >> >>> >>> 2471?? if (thread_state() != _thread_blocked) { >>> 2472???? OrderAccess::instruction_pipeline(); >>> 2473?? } >>> >>> Again not sure why this is necessary. The call to java_suspend_self in the >>> not-blocked case comes from >>> JavaThread::handle_special_runtime_exit_condition, which itself is called >>> from the transition code, so I'd expect this to be done in the transition code. >> >> java_suspend_self is executed in multiple different states. >> Since you are safe when your have suspend flag set, we must emit the barrier >> when leaving that safe state. Transitions from unsafe to unsafe e.g. vm to java >> do not emit the barrier. >> >>> >>> ????? thread->set_thread_state(_thread_blocked); >>> ????? thread->java_suspend_self(); >>> ????? thread->set_thread_state(state); >>> +???? OrderAccess::instruction_pipeline(); >>> >>> Again seems unnecessary - especially given you have one in >>> java_suspend_self() already! (Notwithstanding I think that one should be >>> removed.) >> >> The above one is if we are not blocked, in this case we are blocked. >> The reason for this is that if we are blocked via a real transition, the >> transition does the instruction barrier. If you don't mind an extra barrier we >> can emit it unconditionally in java_suspend_self. >> >> I think set_thread_state should be private and just the ThreadStateTransistion >> should be allowed to use set_thread_state() :) >> >> Posting update to RFR when completed. >> >> /Robbin >> >>> >>> Thanks, >>> David >>> ----- >>> >>>> After a JavaThread have been in a safepoint/(handshake) safe state it can start >>>> executing updated code. E.g. an oop in the instruction stream can have been >>>> updated. >>>> >>>> Most hardware's require a barrier or that the code cross modified is far >>>> away to >>>> guarantee that the thread executing the updated instruction stream sees the >>>> modification. >>>> >>>> What far away is and how far an update instruction stream is from a safepoint >>>> safe state is not clear. >>>> >>>> To be compliant with those architectures an instruction stream barrier must be >>>> added when leaving the safepoint safe state. >>>> >>>> There may be crashes today due to this missing barrier. >>>> A new CPU with deeper pipeline or changes to the VM which moves a safepoint >>>> safe >>>> state closer to a nmethod can increase changes of a crash. >>>> >>>> A few benchmarks will see a performance regression with the extra serializing, >>>> such as Octane-Crypto -5% (x86). >>>> >>>> With a much more elaborate fix, such as changing the local poll to have more >>>> than two states (armed/disarmed), it would be possible to only emit such >>>> instruction stream barrier when the poll have been armed while the thread was >>>> safepoint safe. >>>> >>>> Passes t1-3 on our platforms, and test built what I can. >>>> >>>> Thanks, Robbin From tobias.hartmann at oracle.com Thu Mar 14 08:26:13 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 14 Mar 2019 09:26:13 +0100 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <18b4cfab-9304-e076-a94e-459d2ff92530@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> <18b4cfab-9304-e076-a94e-459d2ff92530@oracle.com> Message-ID: <00ccb069-f213-63ff-4501-9992fc2a4229@oracle.com> Hi Jamsheed, On 13.03.19 06:48, Jamsheed wrote: > Thank you for the review. > > from the comments it says "StubRoutines are used by Compiled code and runtime system". and > StubRoutines::contains(pc) is expected be checked under code running in Java condition. > > missed adding is_Java check only in windows. later i had corrected it in fast failure handling > code[1], but missed putting it in the code under review. > > i presumed the stub routine[2] code? sigbus memory access error could happen only in case of Unsafe. > but will double check. If there's a bug in one of the other stubs, we could hit a SIGBUS there as well right? With your fix such a failure would be converted to an exception and therefore might go through unnoticed. I think the same applies to methods with has_unsafe_access() already but in this case we don't even check for unsafe access (because we would need to check the caller method of the stub). Best regards, Tobias From tobias.hartmann at oracle.com Thu Mar 14 08:28:54 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 14 Mar 2019 09:28:54 +0100 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> Message-ID: Hi Jamsheed, On 14.03.19 02:29, Jamsheed wrote: > i will try to limit my checks to array copy stubs. will make changes and send it for review. Okay, if there's a simple way to do that. Otherwise, I'm fine with the current version of the fix. Best regards, Tobias From david.holmes at oracle.com Thu Mar 14 09:02:16 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Mar 2019 19:02:16 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <050f8f40-2a6a-9a5e-5920-70498b2d109e@oracle.com> Message-ID: <317a426f-6fa2-2c36-415e-92792ff6e441@oracle.com> On 14/03/2019 6:24 pm, Robbin Ehn wrote: > Hi David, > > On 3/14/19 5:48 AM, David Holmes wrote: >> I don't see how it is better to cover cases that do not need to be >> covered. _thread_blocked is safepoint-safe while _thread_in_VM is not, >> but you don't need to sync with the i-stream when unblocking and >> returning to the VM! Only transitions back to Java need to do this. >> You then want to optimize things so that you only sync when needed. > > I see it the other way around. > If you transition java->vm->java, you are never safe so you don't need > the barrier. True. I think it would be interesting to flesh out the complete matrix to see which variation requires the fewest checks. There are obviously cases for each approach that don't need the barrier. But given semantically it is the execution of Java code that requires the barrier, it makes more sense to me to at least initially start with the -return-to-java states and then from there show there are optimisations possible that allow you place the barriers more strategically in some of the safe->unsafe transitions. > Introducing more cost to the blocked state (and the orthogonal suspended > state) I don't see any problems with and don't need any optimizations. If you're arguing that "we're blocked anyway so cost doesn't matter" that is not always true. If you think about thread-startup handshakes during VM initialization blocking longer than needed can add to startup time. David > While the java->vm->java would need such. > > /Robbin > >> >> But lets see how this plays out after you disarm experiments >> >> Thanks, >> David >> >>>> >>>> On 9/03/2019 1:24 am, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>>> Changeset: >>>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>> >>>> src/hotspot/cpu/x86/assembler_x86.hpp >>>> >>>> As discussed elsewhere mfence() is no longer the advised mechanism, >>>> but instead a serializing instruction like cpuid must be used. >>> >>> AMD64 >>> "Upon release, the target thread must then execute a serializing >>> instruction >>> such as CPUID or MFENCE (a locked operation is not sufficient) before >>> proceeding >>> to the modified code to avoid executing a stale view of the >>> instructions which >>> may have been speculatively fetched." >>> >>> Intel >>> "MFENCE does not serialize the instruction stream." >>> >>> CPUID seems to be common ground, changed. >>> >>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>> >>>> MacroAssembler::debug32 seems quite bizarre. First, pre-existing: >>> >>> Reverted. >>> >>> >>>> >>>> --- >>>> >>>> src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp >>>> >>>> 2562?? __ lock(); >>>> 2563?? __ addl(Address(r15_thread, >>>> JavaThread::thread_state_offset()), 1 /* _thread_in_native_trans */); >>>> >>>> This changes seems incorrect. You need a storeload barrier (at >>>> least) after the update to thread_state but your lock() prefix would >>>> place barriers before I think. In any case I find the existing code >>>> clearer and more obviously correct. >>> >>> We describe this as . >>> So it is correct. >>> >>> Reverted. >>> >>> >>>> >>>> --- >>>> >>>> src/hotspot/cpu/x86/stubGenerator_x86_64.cpp >>>> src/hotspot/os_cpu/windows_x86/orderAccess_windows_x86.hpp >>>> src/hotspot/share/runtime/orderAccess.cpp >>>> src/hotspot/share/runtime/stubRoutines.cpp >>>> src/hotspot/share/runtime/stubRoutines.hpp >>>> >>>> I don't understand why we need a stub for this (or for fence() for >>>> that matter) ?? >>> >>> Visual studio don't support inline assembly on x64. >>> Changing to CPUID. >>> >>>> >>>> --- >>>> >>>> src/hotspot/cpu/x86/templateInterpreterGenerator_x86.cpp >>>> >>>> 1090?? __ lock(); >>>> 1091?? __ addl(Address(thread, JavaThread::thread_state_offset()), 1 >>>> /*_thread_in_native_trans*/); >>>> >>>> Same comment as above. >>> >>> Reverted. >>> >>>> >>>> --- >>>> >>>> src/hotspot/os_cpu/aix_ppc/orderAccess_aix_ppc.hpp >>>> src/hotspot/os_cpu/linux_ppc/orderAccess_linux_ppc.hpp >>>> >>>> PPC has isync for instruction stream synchronization, so I'd expect >>>> to see that used in OrderAccess::instruction_pipeline() >>> >>> I'll let PPC folks see what they need to do, but yes that was my >>> thought also. >>> >>>> >>>> --- >>>> >>>> src/hotspot/os_cpu/bsd_x86/orderAccess_bsd_x86.hpp >>>> src/hotspot/os_cpu/linux_x86/orderAccess_linux_x86.hpp >>>> src/hotspot/os_cpu/solaris_x86/orderAccess_solaris_x86.hpp >>>> >>>> s/mfence/cpuid >>> >>> Fixed. >>> >>>> >>>> --- >>>> >>>> src/hotspot/os_cpu/linux_aarch64/orderAccess_linux_aarch64.hpp >>>> src/hotspot/os_cpu/linux_arm/orderAccess_linux_arm.hpp >>>> >>>> Aarch64/ARM has ISB for instruction stream sync. >>> >>> As Haley said, arm do not have instruction cache coherency, so ISB is >>> not enough in all cases. >>> >>>> >>>> --- >>>> >>>> src/hotspot/os_cpu/linux_s390/orderAccess_linux_s390.hpp >>>> >>>> Need input from s390 folk. >>> >>> Yes. >>> >>>> >>>> --- >>>> >>>> src/hotspot/os_cpu/linux_sparc/orderAccess_linux_sparc.hpp >>>> src/hotspot/os_cpu/solaris_sparc/orderAccess_solaris_sparc.hpp >>>> >>>> SPARC needs a FLUSH >>>> >>>> "H.1.6 Self-Modifying Code >>>> If a program includes self-modifying code, it must issue a FLUSH >>>> instruction for each modified doubleword of instructions (or a call >>>> to supervisor software having an equivalent effect)." [SPARCV9 >>>> Architecture manual] >>>> >>> >>> Yes. >>> >>>> --- >>>> >>>> src/hotspot/share/classfile/classFileParser.cpp >>>> >>>> Change is unrelated to this bug. >>> >>> It's related that without it 32-bit don't compile (without >>> disable-errors was warning). So I need it for test compiling. Want me >>> to revert it? >>> >>>> >>>> --- >>>> >>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>> >>>> The changes seem unnecessary. As the code comment states we are >>>> going from one safepoint unsafe state to another. There is no way to >>>> execute any JIT'd code before returning to Java, and that return is >>>> where we should sync if needed. >>> >>> You are correct. >>> >>>> >>>> --- >>>> >>>> src/hotspot/share/prims/jvmtiExport.cpp >>>> >>>> The change is unnecessary as we are executing in the VMThread. Not >>>> only would it be the one doing the code changes at a safepoint, it >>>> never executes compiled Java code anyway. >>> >>> Yes. >>> >>>> >>>> --- >>>> >>>> src/hotspot/share/runtime/interfaceSupport.inline.hpp >>>> src/hotspot/share/runtime/safepoint.cpp >>>> >>>> My initial thought was that the OrderAccess::instruction_pipeline() >>>> should be buried in the safepoint blocking code, but in some cases >>>> we may not have actually blocked at the safepoint and so >>>> (theoretically at least) may need to sync with the i-stream in case >>>> an intervening safepoint changed some code. Can't help but worry >>>> that unconditionally doing this on native transitions is going to >>>> hurt JNI performance. It also looks like we will double up on the >>>> instruction_pipeline() in some cases. >>> >>> Yes. I had an idea with passing around a token for it, but it just >>> clobbers the code more. >>> >>>> >>>> --- >>>> >>>> src/hotspot/share/runtime/thread.cpp >>>> >>>> 1839?? OrderAccess::instruction_pipeline(); >>>> >>>> Can't see why this is necessary here, we should sync when returning >>>> to _thread_in_Java. >>> >>> In this changeset we always do the barrier when leaving a safe state. >>> When you are not on threads list you are safe, so you can go >>> new->vm->java just missing the safepoint. >>> >>>> >>>> 2471?? if (thread_state() != _thread_blocked) { >>>> 2472???? OrderAccess::instruction_pipeline(); >>>> 2473?? } >>>> >>>> Again not sure why this is necessary. The call to java_suspend_self >>>> in the not-blocked case comes from >>>> JavaThread::handle_special_runtime_exit_condition, which itself is >>>> called from the transition code, so I'd expect this to be done in >>>> the transition code. >>> >>> java_suspend_self is executed in multiple different states. >>> Since you are safe when your have suspend flag set, we must emit the >>> barrier >>> when leaving that safe state. Transitions from unsafe to unsafe e.g. >>> vm to java >>> do not emit the barrier. >>> >>>> >>>> ????? thread->set_thread_state(_thread_blocked); >>>> ????? thread->java_suspend_self(); >>>> ????? thread->set_thread_state(state); >>>> +???? OrderAccess::instruction_pipeline(); >>>> >>>> Again seems unnecessary - especially given you have one in >>>> java_suspend_self() already! (Notwithstanding I think that one >>>> should be removed.) >>> >>> The above one is if we are not blocked, in this case we are blocked. >>> The reason for this is that if we are blocked via a real transition, >>> the transition does the instruction barrier. If you don't mind an >>> extra barrier we can emit it unconditionally in java_suspend_self. >>> >>> I think set_thread_state should be private and just the >>> ThreadStateTransistion should be allowed to use set_thread_state() :) >>> >>> Posting update to RFR when completed. >>> >>> /Robbin >>> >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> After a JavaThread have been in a safepoint/(handshake) safe state >>>>> it can start >>>>> executing updated code. E.g. an oop in the instruction stream can >>>>> have been >>>>> updated. >>>>> >>>>> Most hardware's require a barrier or that the code cross modified >>>>> is far away to >>>>> guarantee that the thread executing the updated instruction stream >>>>> sees the >>>>> modification. >>>>> >>>>> What far away is and how far an update instruction stream is from a >>>>> safepoint >>>>> safe state is not clear. >>>>> >>>>> To be compliant with those architectures an instruction stream >>>>> barrier must be >>>>> added when leaving the safepoint safe state. >>>>> >>>>> There may be crashes today due to this missing barrier. >>>>> A new CPU with deeper pipeline or changes to the VM which moves a >>>>> safepoint safe >>>>> state closer to a nmethod can increase changes of a crash. >>>>> >>>>> A few benchmarks will see a performance regression with the extra >>>>> serializing, >>>>> such as Octane-Crypto -5% (x86). >>>>> >>>>> With a much more elaborate fix, such as changing the local poll to >>>>> have more >>>>> than two states (armed/disarmed), it would be possible to only emit >>>>> such >>>>> instruction stream barrier when the poll have been armed while the >>>>> thread was >>>>> safepoint safe. >>>>> >>>>> Passes t1-3 on our platforms, and test built what I can. >>>>> >>>>> Thanks, Robbin From martin.doerr at sap.com Thu Mar 14 09:25:41 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 14 Mar 2019 09:25:41 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: Hi Robbin, > But that is whole new subject and let's keep that separate from this :) Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: orderAccess_aix/linux_ppc.hpp: inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } orderAccess_linux_s390.hpp: inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } Thanks and best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Mittwoch, 13. M?rz 2019 20:30 To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net Cc: Reingruber, Richard Subject: Re: RFR(m): 8220351: Cross-modifying code Hi, On 2019-03-13 18:56, Doerr, Martin wrote: > Hi everybody, > > while discussing with a colleague the following question about the patch site on x86 came up: > > nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. > I couldn't find a call to AbstractICache::invalidate_range. Look for ICache::invalidate_range, I see 11 calls on x64. And one call to AbstractICache/ICache::invalidate_word. Btw I have spent some time looking at x86/icache_x86.cpp... But that is whole new subject and let's keep that separate from this :) I'll create issues for some of these topics, which if nothing else should be discussed. /Robbin > > Does anybody know where it is or if it's missing? > > Best regards, > Martin > From robbin.ehn at oracle.com Thu Mar 14 09:33:34 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 14 Mar 2019 10:33:34 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: Hi Martin, On 3/14/19 10:25 AM, Doerr, Martin wrote: > Hi Robbin, > >> But that is whole new subject and let's keep that separate from this :) > Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? > > > Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: > > orderAccess_aix/linux_ppc.hpp: > inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } > > orderAccess_linux_s390.hpp: > inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } > Thanks! >> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >> I couldn't find a call to AbstractICache::invalidate_range. I think I miss-understood you here, the reason I heard for not doing invalidate_range/word in nmethod::oops_do is that it was deemed that a safepoint is 'long enough'. /Robbin > > Look for ICache::invalidate_range, I see 11 calls on x64. > And one call to AbstractICache/ICache::invalidate_word. > Btw I have spent some time looking at x86/icache_x86.cpp... > But that is whole new subject and let's keep that separate from this :) > > I'll create issues for some of these topics, which if nothing else should be > discussed. > > /Robbin > >> >> Does anybody know where it is or if it's missing? >> >> Best regards, >> Martin >> From fujie at loongson.cn Thu Mar 14 10:00:08 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 14 Mar 2019 18:00:08 +0800 Subject: RFR: 8220658: Improve the readability of container information in the error log Message-ID: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> Hi all, Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ In the VM's error log, the container information was dumpped as: --------------------------------- container (cgroup) information: container_type: cgroupv1 cpu_cpuset_cpus: 0-11 cpu_memory_nodes: 0 active_processor_count: 12 cpu_quota: -1 cpu_period: 100000 cpu_shares: -1 memory_limit_in_bytes: -1 memory_and_swap_limit_in_bytes: -2 memory_soft_limit_in_bytes: -1 memory_usage_in_bytes: 11690164224 memory_max_usage_in_bytes: 1806413824 --------------------------------- Items such as "memory_limit_in_bytes" and "memory_and_swap_limit_in_bytes" are difficult to understand. It might be more human-readable if we could dump it like this: --------------------------------- container (cgroup) information: container_type: cgroupv1 cpu_cpuset_cpus: 0-11 cpu_memory_nodes: 0 active_processor_count: 12 cpu_quota: no quota cpu_period: 100000 cpu_shares: no share memory_limit_in_bytes: unlimited memory_and_swap_limit_in_bytes: failed memory_soft_limit_in_bytes: unlimited memory_usage_in_bytes: 11690164224 memory_max_usage_in_bytes: 1806413824 --------------------------------- I made this patch to improve the readability of container information in the error log. The change has been tested with hotspot:tier1 on Linux/x86-64. Could you please review it? Thanks a lot. Best regards, Jie From aph at redhat.com Thu Mar 14 10:03:21 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 14 Mar 2019 10:03:21 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> Message-ID: On 3/11/19 8:08 AM, Robbin Ehn wrote: > For simplicity I'll talk about x86, which have instruction cache > coherency. I don't think this is the right approach. You've put fences into shared code, but the semantics to make this work correctly are different on different platforms. For example, some platforms may not use patching, but instead use the constant pool. Some platforms may need cache flushes as well as fences, so the solution presented here is either inadequate, inefficient, or both. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From richard.reingruber at sap.com Thu Mar 14 10:12:59 2019 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 14 Mar 2019 10:12:59 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: Hi Robbin, > the reason I heard for not doing invalidate_range/word in nmethod::oops_do > is that it was deemed that a safepoint is 'long enough'. This doesn't seem to be a very good reason. I did a little modification to your initial figure: JavaThread: | VMThread StoreLoad | | update | ? Global ICache Invalidate ? | disarm load thread_poll | Question would be: does an ICache flush happen after the immediate oop update? I would consider it problematic if not. Even if the safepoint was very long. Cheers Richard. -----Original Message----- From: Robbin Ehn Sent: Donnerstag, 14. M?rz 2019 10:34 To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net Cc: Reingruber, Richard Subject: Re: RFR(m): 8220351: Cross-modifying code Hi Martin, On 3/14/19 10:25 AM, Doerr, Martin wrote: > Hi Robbin, > >> But that is whole new subject and let's keep that separate from this :) > Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? > > > Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: > > orderAccess_aix/linux_ppc.hpp: > inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } > > orderAccess_linux_s390.hpp: > inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } > Thanks! >> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >> I couldn't find a call to AbstractICache::invalidate_range. I think I miss-understood you here, the reason I heard for not doing invalidate_range/word in nmethod::oops_do is that it was deemed that a safepoint is 'long enough'. /Robbin > > Look for ICache::invalidate_range, I see 11 calls on x64. > And one call to AbstractICache/ICache::invalidate_word. > Btw I have spent some time looking at x86/icache_x86.cpp... > But that is whole new subject and let's keep that separate from this :) > > I'll create issues for some of these topics, which if nothing else should be > discussed. > > /Robbin > >> >> Does anybody know where it is or if it's missing? >> >> Best regards, >> Martin >> From robbin.ehn at oracle.com Thu Mar 14 10:20:51 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 14 Mar 2019 11:20:51 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> Message-ID: <1d1c7c0b-0707-3cd7-3049-43a7bd9c37e3@oracle.com> On 3/14/19 11:03 AM, Andrew Haley wrote: > On 3/11/19 8:08 AM, Robbin Ehn wrote: > >> For simplicity I'll talk about x86, which have instruction cache >> coherency. > > I don't think this is the right approach. I was talking about the example in which I used the x86 native wrapper. Why this is a problem even on platforms which have coherency. > > You've put fences into shared code, but the semantics to make this > work correctly are different on different platforms. For example, some > platforms may not use patching, but instead use the constant > pool. Some platforms may need cache flushes as well as fences, so the > solution presented here is either inadequate, inefficient, or both. > The barrier placement, if any barrier is needed, would be the same for all platforms. So this patch places those barrier where we need them and add implementation for x86 (and PPC/s390). So for platforms that also need cache flushes this is inadequate and is not intended to fix that. /Robbin From lois.foltan at oracle.com Thu Mar 14 11:39:49 2019 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 14 Mar 2019 07:39:49 -0400 Subject: RFR(M) JDK-8218994: Consolidate indy and condy JVM information within a BootstrapInfo data structure Message-ID: <6cdcd602-1543-e19e-718e-3a6c0784c2c1@oracle.com> Please review this change to consolidate indy and condy JVM information to invoke a bootstrap method into a new BootstrapInfo data structure as well as merge the condy and indy methods to invoke the bootstrap within SystemDictionary. open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8218994.0/webrev/ bug link: https://bugs.openjdk.java.net/browse/JDK-8218994 co-contributors: Lois Foltan & John Rose Testing: hs-tier1-3 & jdk-tier1-3 (all platforms), hs-tier4-8 (linux only), JCK vm & lang Thanks, Lois From robbin.ehn at oracle.com Thu Mar 14 12:24:18 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 14 Mar 2019 13:24:18 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: Hi Richard, On 3/14/19 11:12 AM, Reingruber, Richard wrote: > Hi Robbin, > > > the reason I heard for not doing invalidate_range/word in nmethod::oops_do > > is that it was deemed that a safepoint is 'long enough'. > > This doesn't seem to be a very good reason. I did a little modification to your > initial figure: That's why we are adding the instruction barrier! > > JavaThread: | VMThread > StoreLoad | > | update > | ? Global ICache Invalidate ? > | disarm > load thread_poll | > > Question would be: does an ICache flush happen after the immediate oop update? > I would consider it problematic if not. Even if the safepoint was very long. The icache flush assumes it can stop speculation and flush instruction cache in patchee thread. Correct me if anything is wrong: - sparc needs flush in all patchee threads. - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. - ppc needs isync in all patchee threads? - s390 needs zarch_sync all in patchee threads? - x86 needs cpuid in all in patchee threads. So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) have no effect in patchee threads!? Adding a few words on x86 ICache::Invalidate/word. We have TSO, so if the patchee thread can see poll is disarmed all other stores are visible. I have seen no case ICache::Invalidate/word would make a difference from a StoreStore/compiler barrier (still talking about platform code for x86) ? The problem is if we prefect an instruction over the load of the poll. If poll is disarmed this perfected instruction can be fully executed. If that happens to contain an immediate oop, we don't know if it's the patched oop or not. /Robbin > > Cheers Richard. > > -----Original Message----- > From: Robbin Ehn > Sent: Donnerstag, 14. M?rz 2019 10:34 > To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net > Cc: Reingruber, Richard > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Martin, > > On 3/14/19 10:25 AM, Doerr, Martin wrote: >> Hi Robbin, >> >>> But that is whole new subject and let's keep that separate from this :) >> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >> >> >> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >> >> orderAccess_aix/linux_ppc.hpp: >> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >> >> orderAccess_linux_s390.hpp: >> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >> > > Thanks! > >>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>> I couldn't find a call to AbstractICache::invalidate_range. > > I think I miss-understood you here, the reason I heard for not doing > invalidate_range/word in nmethod::oops_do is that it was deemed that a > safepoint is 'long enough'. > > /Robbin > >> >> Look for ICache::invalidate_range, I see 11 calls on x64. >> And one call to AbstractICache/ICache::invalidate_word. >> Btw I have spent some time looking at x86/icache_x86.cpp... >> But that is whole new subject and let's keep that separate from this :) >> >> I'll create issues for some of these topics, which if nothing else should be >> discussed. >> >> /Robbin >> >>> >>> Does anybody know where it is or if it's missing? >>> >>> Best regards, >>> Martin >>> From jamsheed.c.m at oracle.com Thu Mar 14 15:10:53 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Thu, 14 Mar 2019 20:40:53 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <00ccb069-f213-63ff-4501-9992fc2a4229@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> <18b4cfab-9304-e076-a94e-459d2ff92530@oracle.com> <00ccb069-f213-63ff-4501-9992fc2a4229@oracle.com> Message-ID: <3cdfd9aa-314e-6773-3359-1f603453ff4a@oracle.com> Hi Tobias Hi Tobias, On 14/03/19 1:56 PM, Tobias Hartmann wrote: > Hi Jamsheed, > > On 13.03.19 06:48, Jamsheed wrote: >> Thank you for the review. >> >> from the comments it says "StubRoutines are used by Compiled code and runtime system". and >> StubRoutines::contains(pc) is expected be checked under code running in Java condition. >> >> missed adding is_Java check only in windows. later i had corrected it in fast failure handling >> code[1], but missed putting it in the code under review. >> >> i presumed the stub routine[2] code? sigbus memory access error could happen only in case of Unsafe. >> but will double check. > If there's a bug in one of the other stubs, we could hit a SIGBUS there as well right? With your fix > such a failure would be converted to an exception and therefore might go through unnoticed. I think > the same applies to methods with has_unsafe_access() already but in this case we don't even check > for unsafe access (because we would need to check the caller method of the stub). very intermittent bug in compiler generating some invalid address can be issue, but this could happen for all has_unsafe_access() set nmethods too. if it is frequently failing bug, there is high chance of getting attention, so probably adding a diagnostic option to generate core file in that case would work. Best regards, Jamsheed > > Best regards, > Tobias From jamsheed.c.m at oracle.com Thu Mar 14 15:16:06 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Thu, 14 Mar 2019 20:46:06 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> Message-ID: <144af13e-33e2-2089-f8d2-902e94485eb1@oracle.com> Hi Tobias, On 14/03/19 1:58 PM, Tobias Hartmann wrote: > Hi Jamsheed, > > On 14.03.19 02:29, Jamsheed wrote: >> i will try to limit my checks to array copy stubs. will make changes and send it for review. > Okay, if there's a simple way to do that. Otherwise, I'm fine with the current version of the fix. either i should have separate code generated for unsafe copy. or setting/resetting doing_unsafe_access before after unsafe_copy should do. Best regards, Jamsheed > > Best regards, > Tobias From zgu at redhat.com Thu Mar 14 15:37:15 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 14 Mar 2019 11:37:15 -0400 Subject: RFR: 8220602: Shenandoah-SA: Enable best-effort implementation of heap walk Message-ID: Please review this patch that provides best-effort implementation of live regions iteration for Shenandoah GC. There are minor changes in shared code, to adjust oop offset from base of allocation cell. Bug: https://bugs.openjdk.java.net/browse/JDK-8220602 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8220602/webrev.00/index.html Test: hotspot_gc_shenandoah hotspot_serviceability vmTestbase/nsk/jdb manual test: jhsdb hsdb on Linux x64 Passed submit test. Thanks, -Zhengyu From rkennke at redhat.com Thu Mar 14 16:21:05 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 14 Mar 2019 17:21:05 +0100 Subject: RFR: 8220602: Shenandoah-SA: Enable best-effort implementation of heap walk In-Reply-To: References: Message-ID: Hi Zhengyu, > Please review this patch that provides best-effort implementation of > live regions iteration for Shenandoah GC. > > There are minor changes in shared code, to adjust oop offset from base > of allocation cell. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220602 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8220602/webrev.00/index.html > > > Test: > ? hotspot_gc_shenandoah > ? hotspot_serviceability > ? vmTestbase/nsk/jdb > ? manual test: jhsdb hsdb > ? on Linux x64 > > ? Passed submit test. It looks mostly ok. But why do we need the change in CollectedHeap? Why not keep this an impl detail of Shenandoah heap? Roman From sgehwolf at redhat.com Thu Mar 14 16:51:33 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 14 Mar 2019 17:51:33 +0100 Subject: RFR: 8220579: [Containers] SubSystem.java out of sync with osContainer_linux.cpp Message-ID: <8e1a66f748861591316e7411d4829d4096327a6d.camel@redhat.com> Hi, I'm not sure what the right list for Metrics.java[1] is. Assuming it's serviceability-dev: Please review this one-liner for for SubSystem.java which currently behaves differently from the native implementation in osContainer_linux.cpp. Please see the details in the bug. Bug: https://bugs.openjdk.java.net/browse/JDK-8220579 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220579/01/webrev/ Testing: Manual testing of JDK-8217338 with Metrics.java support with/without this fix on Linux x86_64. Metrics tests and Docker tests continue to pass for fastdebug jvms (NOT for release jvms. see JDK-8220674, which was fun). Thoughts? Thanks, Severin [1] http://hg.openjdk.java.net/jdk/jdk/file/641768acb12e/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/Metrics.java From zgu at redhat.com Thu Mar 14 16:52:36 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 14 Mar 2019 12:52:36 -0400 Subject: RFR: 8220602: Shenandoah-SA: Enable best-effort implementation of heap walk In-Reply-To: References: Message-ID: <4c6c152f-66d2-c09c-2364-9c89089d633f@redhat.com> Hi Roman, Thanks for reviewing. >> ?? Passed submit test. > > It looks mostly ok. But why do we need the change in CollectedHeap? Why > not keep this an impl detail of Shenandoah heap? Because the region iteration code is in shared (ObjectHeap.java) Thanks, -Zhengyu > > Roman > From hboehm at google.com Wed Mar 13 22:19:42 2019 From: hboehm at google.com (Hans Boehm) Date: Wed, 13 Mar 2019 15:19:42 -0700 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: References: Message-ID: Looks great to me! Just to be clear, this is intended to guarantee that it's not possible to promote a weak global reference that indirectly points to A after a PhantomReference or a WeakGlobalReference to A has been cleared? Thus this reference clearing must happen more or less atomically. I think this property has always been needed for the intended use of PhantomReferences, since otherwise the referent of a cleared PhantomReference could be resurrected. It's great to get it in the spec. On Wed, Mar 13, 2019 at 1:08 PM Kim Barrett wrote: > Please review this change to the JNI specification. The specified > behavior of Weak Global References, and in particular their > relationship to Java PhantomReference, is being updated to account > for a change to PhantomReference by JDK-8071507. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8188066 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8220617 > > Changes: > http://cr.openjdk.java.net/~kbarrett/8188066/jni_specdiff_rev1.html > specdiff for the changes. In addition to updating the section > describing Weak Global References, the description of NewGlobalRef > is updated, as well as the copyright footer. > > http://cr.openjdk.java.net/~kbarrett/8188066/weak_global_refs_rev1.md > Copies of the before and after text for the JNI Weak Global > References section; easier to read than diffs or specdiff. > > > From coleen.phillimore at oracle.com Thu Mar 14 17:40:49 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 14 Mar 2019 13:40:49 -0400 Subject: RFR (S) 8220512: Deoptimize redefinition functions that have dirty ICs Message-ID: <00317af6-2a5f-5b32-408b-3bd888682c19@oracle.com> Summary: Walk ICs to determine whether nmethods are dependent on redefined classes. See bug for more details.? Tested with redefinition tests: #redefinition tests. make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jvmti >&jvmti.out make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jdi >&jdi.out make test TEST=open/test/hotspot/jtreg/runtime/RedefineTests >&redefine.out make test TEST=open/test/jdk/java/lang/instrument >&instrument.out make test TEST=open/test/jdk/com/sun/jdi >&jtreg.jdi.out hs-tier1-6 as well as java/lang/instrument tests with -Xcomp. open webrev at http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8220512 Thanks, Coleen From rkennke at redhat.com Thu Mar 14 17:46:05 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 14 Mar 2019 18:46:05 +0100 Subject: RFR: 8220602: Shenandoah-SA: Enable best-effort implementation of heap walk In-Reply-To: <4c6c152f-66d2-c09c-2364-9c89089d633f@redhat.com> References: <4c6c152f-66d2-c09c-2364-9c89089d633f@redhat.com> Message-ID: <382e4b54-6313-2029-0add-b80636d7f4da@redhat.com> Hi Zhengyu, >>> ?? Passed submit test. >> >> It looks mostly ok. But why do we need the change in CollectedHeap? >> Why not keep this an impl detail of Shenandoah heap? > > Because the region iteration code is in shared (ObjectHeap.java) Ah ok. I have a patch up my sleeve that eliminates the extra word ;-) But I suppose you should not wait for it (probably some weeks). Reviewed! Roman From zgu at redhat.com Thu Mar 14 17:51:15 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 14 Mar 2019 13:51:15 -0400 Subject: RFR: 8220602: Shenandoah-SA: Enable best-effort implementation of heap walk In-Reply-To: <382e4b54-6313-2029-0add-b80636d7f4da@redhat.com> References: <4c6c152f-66d2-c09c-2364-9c89089d633f@redhat.com> <382e4b54-6313-2029-0add-b80636d7f4da@redhat.com> Message-ID: <5bc4a0f7-11e2-fe58-842b-06f9b47262f6@redhat.com> On 3/14/19 1:46 PM, Roman Kennke wrote: > Hi Zhengyu, > >>>> ?? Passed submit test. >>> >>> It looks mostly ok. But why do we need the change in CollectedHeap? >>> Why not keep this an impl detail of Shenandoah heap? >> >> Because the region iteration code is in shared (ObjectHeap.java) > > > Ah ok. > > I have a patch up my sleeve that eliminates the extra word ;-) But I > suppose you should not wait for it (probably some weeks). Yep, we can revert this part once it arrives. Thanks, -Zhengyu > > Reviewed! > > Roman > From bob.vandette at oracle.com Thu Mar 14 17:58:54 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 14 Mar 2019 13:58:54 -0400 Subject: RFR: 8220579: [Containers] SubSystem.java out of sync with osContainer_linux.cpp In-Reply-To: <8e1a66f748861591316e7411d4829d4096327a6d.camel@redhat.com> References: <8e1a66f748861591316e7411d4829d4096327a6d.camel@redhat.com> Message-ID: The change looks good. Thanks for fixing this. I?d send this to core-libs (cc?d). Bob. > On Mar 14, 2019, at 12:51 PM, Severin Gehwolf wrote: > > Hi, > > I'm not sure what the right list for Metrics.java[1] is. Assuming it's > serviceability-dev: > > Please review this one-liner for for SubSystem.java which currently > behaves differently from the native implementation in > osContainer_linux.cpp. Please see the details in the bug. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220579 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220579/01/webrev/ > > Testing: > Manual testing of JDK-8217338 with Metrics.java support with/without > this fix on Linux x86_64. Metrics tests and Docker tests continue to > pass for fastdebug jvms (NOT for release jvms. see JDK-8220674, which > was fun). > > Thoughts? > > Thanks, > Severin > > [1] http://hg.openjdk.java.net/jdk/jdk/file/641768acb12e/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/Metrics.java > From aph at redhat.com Thu Mar 14 18:18:54 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 14 Mar 2019 18:18:54 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <1d1c7c0b-0707-3cd7-3049-43a7bd9c37e3@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> <1d1c7c0b-0707-3cd7-3049-43a7bd9c37e3@oracle.com> Message-ID: <26959ebc-82ab-a755-5ef7-82afa30f2c50@redhat.com> On 3/14/19 10:20 AM, Robbin Ehn wrote: > On 3/14/19 11:03 AM, Andrew Haley wrote: >> On 3/11/19 8:08 AM, Robbin Ehn wrote: >> >> You've put fences into shared code, but the semantics to make this >> work correctly are different on different platforms. For example, some >> platforms may not use patching, but instead use the constant >> pool. Some platforms may need cache flushes as well as fences, so the >> solution presented here is either inadequate, inefficient, or both. >> > The barrier placement, if any barrier is needed, would be the same for all > platforms. I'm not so sure of that. Not all platforms necessarily patch instructions; it should up to the port to do something appropriate when necessary. AArch64, for example, patches code in only limited circumstances. > So this patch places those barrier where we need them and add > implementation for x86 (and PPC/s390). So for platforms that also > need cache flushes this is inadequate and is not intended to fix > that. I have an idea that might work. Let's say that icache::flush(), executed whenever we patch code, increments a global patching counter. When a Java thread returns from a handshake, it compares a thread-local copy of the old value of the patching counter with the current value. If they are different, do whatever local fencing is necessary and record the current value of the patching counter. That should work, right? So we only execute necessary barriers, and if some port keeps most constants in the constant pool (rather than patching) it'll benefit. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Thu Mar 14 18:23:10 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 14 Mar 2019 18:23:10 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <26959ebc-82ab-a755-5ef7-82afa30f2c50@redhat.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> <1d1c7c0b-0707-3cd7-3049-43a7bd9c37e3@oracle.com> <26959ebc-82ab-a755-5ef7-82afa30f2c50@redhat.com> Message-ID: <96bd20d9-1544-0bc9-8070-fd3f854c0de0@redhat.com> On 3/14/19 6:18 PM, Andrew Haley wrote: > I have an idea that might work. Let's say that icache::flush(), > executed whenever we patch code, increments a global patching > counter. When a Java thread returns from a handshake, it compares a > thread-local copy of the old value of the patching counter with the > current value. If they are different, do whatever local fencing is > necessary and record the current value of the patching counter. And, just to complicate this even more, on some processors you'd not only need to to do local fencing but you might need to know which memory addresses had changed. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From sgehwolf at redhat.com Thu Mar 14 18:24:58 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 14 Mar 2019 19:24:58 +0100 Subject: RFR: 8220579: [Containers] SubSystem.java out of sync with osContainer_linux.cpp In-Reply-To: References: <8e1a66f748861591316e7411d4829d4096327a6d.camel@redhat.com> Message-ID: On Thu, 2019-03-14 at 13:58 -0400, Bob Vandette wrote: > The change looks good. Thanks for fixing this. Thanks for the review, Bob! Cheers, Severin > I?d send this to core-libs (cc?d). > > Bob. > > > > On Mar 14, 2019, at 12:51 PM, Severin Gehwolf > > wrote: > > > > Hi, > > > > I'm not sure what the right list for Metrics.java[1] is. Assuming > > it's > > serviceability-dev: > > > > Please review this one-liner for for SubSystem.java which currently > > behaves differently from the native implementation in > > osContainer_linux.cpp. Please see the details in the bug. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220579 > > webrev: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220579/01/webrev/ > > > > Testing: > > Manual testing of JDK-8217338 with Metrics.java support > > with/without > > this fix on Linux x86_64. Metrics tests and Docker tests continue > > to > > pass for fastdebug jvms (NOT for release jvms. see JDK-8220674, > > which > > was fun). > > > > Thoughts? > > > > Thanks, > > Severin > > > > [1] > > http://hg.openjdk.java.net/jdk/jdk/file/641768acb12e/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/Metrics.java > > From aph at redhat.com Thu Mar 14 18:34:43 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 14 Mar 2019 18:34:43 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: On 3/14/19 12:24 PM, Robbin Ehn wrote: > - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. No, it's not as bad as that. The patchee only needs an ISB. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Thu Mar 14 18:36:34 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 14 Mar 2019 18:36:34 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <26959ebc-82ab-a755-5ef7-82afa30f2c50@redhat.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> <1d1c7c0b-0707-3cd7-3049-43a7bd9c37e3@oracle.com> <26959ebc-82ab-a755-5ef7-82afa30f2c50@redhat.com> Message-ID: On 3/14/19 6:18 PM, Andrew Haley wrote: > I have an idea that might work. Let's say that icache::flush(), > executed whenever we patch code, increments a global patching > counter. When a Java thread returns from a handshake, it compares a > thread-local copy of the old value of the patching counter with the > current value. If they are different, do whatever local fencing is > necessary and record the current value of the patching counter. > > That should work, right? So we only execute necessary barriers, and > if some port keeps most constants in the constant pool (rather than > patching) it'll benefit. But, I should stress, all of this can be done in platform-specific code, so the main code base doesn't have to be bothered by any of it. > And, just to complicate this even more, on some processors you'd not > only need to to do local fencing but you might need to know which > memory addresses had changed. For avoidance of doubt, I don't think that we support any such processors. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Thu Mar 14 19:09:23 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 14 Mar 2019 15:09:23 -0400 Subject: RFR (S) 8220512: Deoptimize redefinition functions that have dirty ICs In-Reply-To: <15470b26-c400-1bc5-947f-985ee53cb002@oracle.com> References: <00317af6-2a5f-5b32-408b-3bd888682c19@oracle.com> <15470b26-c400-1bc5-947f-985ee53cb002@oracle.com> Message-ID: <38cdb82d-c51e-b906-c1e4-2211715fa7b6@oracle.com> On 3/14/19 2:24 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > It looks good to me. > > Just one minor suggestion: > > http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev/src/hotspot/share/classfile/metadataOnStackMark.cpp.udiff.html > - Threads::metadata_do(Metadata::mark_on_stack); > - CodeCache::metadata_do(Metadata::mark_on_stack); > + MetadataOnStackClosure mon_stack; > + Threads::metadata_do(&mon_stack); > + CodeCache::metadata_do(&mon_stack); > ?The 'mon_stack' can be associated with monitors. > ?How about to rename it to something like 'md_on_stack'? Okay, I'll change the name to md_on_stack. Thanks for the code review! Coleen > > Thanks, > Serguei > > > On 3/14/19 10:40, coleen.phillimore at oracle.com wrote: >> Summary: Walk ICs to determine whether nmethods are dependent on >> redefined classes. >> >> See bug for more details.? Tested with redefinition tests: >> >> #redefinition tests. >> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jvmti >&jvmti.out >> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jdi >&jdi.out >> make test TEST=open/test/hotspot/jtreg/runtime/RedefineTests >> >&redefine.out >> make test TEST=open/test/jdk/java/lang/instrument >&instrument.out >> make test TEST=open/test/jdk/com/sun/jdi >&jtreg.jdi.out >> >> hs-tier1-6 as well as java/lang/instrument tests with -Xcomp. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8220512 >> >> Thanks, >> Coleen >> >> > From igor.ignatyev at oracle.com Thu Mar 14 21:33:20 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 14 Mar 2019 14:33:20 -0700 Subject: RFR(S) : 8219139 : move hotspot tests from test/jdk/vm In-Reply-To: <0c282875-2f0b-ed17-4a05-621bd5e0d862@oracle.com> References: <036DE223-2FAB-43A4-BAC7-63A9BBFCA65D@oracle.com> <0c282875-2f0b-ed17-4a05-621bd5e0d862@oracle.com> Message-ID: Hi Alan, I double checked, the test is linked w/ -ljli, and calls JNI_CreateJavaVM from libjli.so. hence I'll leave JniInvocationTest in jdk/tools/launcher. -- Igor > On Feb 21, 2019, at 12:17 PM, Alan Bateman wrote: > > On 21/02/2019 19:55, Igor Ignatyev wrote: >> : >> Alan, you are right, this test is a JNI test and should be moved to hotspot/runtime/jni. more details in my answer to the same comment in David's email. in two words, I accidentally looked at another test. >> > Can you double check that it is actually using the JNI invocation interface directly? I don't think we were able to find anyone in Eclipse to explain what their launcher on macOS is doing. I suspect it may be directly (or indirectly) reading the CFBundleExecutable property from Info.plist and calling the JNI_CreateJavaVM function in libjli (not libjvm). We probably need more tests in this area and also a bit more archaeology to see whether this was a supported interface in Apple's JDK. > > -Alan From igor.ignatyev at oracle.com Thu Mar 14 21:38:08 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 14 Mar 2019 14:38:08 -0700 Subject: RFR(S) : 8219139 : move hotspot tests from test/jdk/vm In-Reply-To: <511d81a3-847c-49d0-76ff-fdc836211f37@oracle.com> References: <036DE223-2FAB-43A4-BAC7-63A9BBFCA65D@oracle.com> <132812c3-56be-edfd-f812-78b30c75973b@oracle.com> <511d81a3-847c-49d0-76ff-fdc836211f37@oracle.com> Message-ID: <726FFE91-7F8C-46F6-9DE9-A112F5C5C17B@oracle.com> Hi Misha, thanks for your suggestions, I have moved all runtime tests into subdirectories. here is the updated webrev: http://cr.openjdk.java.net/~iignatyev//8219139/webrev.01/index.html Thanks, -- Igor > On Mar 4, 2019, at 1:57 PM, mikhailo.seledtsov at oracle.com wrote: > > Hi Igor, > > Sorry it took a while to get back to you on this one. See my comment below > > > On 2/22/19 10:35 AM, mikhailo.seledtsov at oracle.com wrote: >> Overall the change looks good; thank you Igor for doing this. I have couple of comments: >> >> - I am in favor of the approach where we move tests first under corresponding sub-component directories in >> test/hotspot sub-tree, and then figure out whether to keep them. Even though in general I am in favor >> of removing tests that are obsolete or of questionable value, this requires time, consideration and discussions. >> Hence, I recommend filing an RFE for that, which can be addressed after the migration. >> >> - Runtime normally avoids tests directly in test/hotspot/jtreg/runtime >> The tests should go into underlying sub-directories which best match functional area or topic of that test. >> In most cases you did it in your change, but there are several tests that your change places directly under >> test/hotspot/jtreg/runtime/: >> >> ExplicitArithmeticCheck.java >> MonitorCacheMaybeExpand_DeadLock.java >> ReflectStackOverflow.java >> ShiftTest.java - David commented this can go under compiler (a jit test) >> WideStrictInline.java > I have looked at the tests in more detail, and here are my recommendation of placements: > ExplicitArithmeticCheck > This test checks that ArithmeticException is thrown when appropriate > I would recommend placing it under runtime/ErrorHandling > MonitorCacheMaybeExpand_DeadLock > Existing folder: runtime/Thread (it does have a locking test) > Or, alternatively, create a new folder: 'locking' or 'monitors' > ReflectStackOverflow > Uses recursive reflection attempting to provoke stack overflow > Can go under: runtime/reflect > WideStrictInline: > checks for correct FP inlining by the interpreter > I could not find existing sections; perhaps create 'interpreter' > folder under 'runtime' > > Thank you, > Misha >> >> Since we plan (as discussed) to follow up this work with an RFE to review and consider removal of old and >> not-that-useful tests, you could place them under 'misc' for now. Alternatively, find the best match >> or create new sub-directories under runtime/ if necessary. >> >> >> Thank you, >> Misha >> >> >> On 2/21/19 11:53 AM, Igor Ignatyev wrote: >>> >>>> On Feb 21, 2019, at 12:11 AM, David Holmes wrote: >>>> >>>> Hi Igor, >>>> >>>> On 21/02/2019 3:19 pm, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>>> 40 lines changed: 17 ins; 2 del; 21 mod; >>>>> Hi all, >>>>> could you please review this small patch which moves tests from test/jdk/vm? >>>> I find some of these tests - the runtime ones at least - of extremely dubious value. They either cover basic functionality that is already well covered, or are regression tests for bugs in code that hasn't existed for many many years! >>> as I wrote in another related email: "one of the reason I'm proposing this move is exactly questionable value of these tests, I want to believe that having these tests in hotspot/ test directories will bring more attention to them from corresponding component teams and hence they will be removed/reworked/re-whatever faster and better. and I also believe that one of the reason we got duplications exactly because these tests were located in jdk test suite." >>> >>>> BTW: >>>> >>>> test/hotspot/jtreg/runtime/ShiftTest.java >>>> >>>> is actually a jit test according to the test comment. >>> sure, I will move it to hotspot/compiler. >>>>> there are 16 tests in test/jdk/vm directory. all but JniInvocationTest are hotspot tests, so they are moved to test/hotspot test suite; JniInvocationTest is a launcher test >>>> No its a JNI invocation API test - nothing to do with our launcher. It belongs in runtime/jni. But we already have tests in runtime that use the JNI invocation API so this test adds no new coverage. >>> this is actually was my first reaction, and I even have the webrev which moves it to runtime/jni, but then I looked at the associated bug and it is filed against tools/launcher. and I even got a false (as I know by now) memory that I saw JLI_ method being called from the test. there is actually another test (dk/tools/launcher/exeJliLaunchTest.c) associated w/ this bug which calls JLI_Launch. anyhow, I'll move this test to hotspot/runtime/jni. >>> >>>> I really think the value of these tests needs to be examined before they are brought over. >>> I'd prefer to have follow-up RFEs/tasks, b/c the longer we keep jdk/vm directory the more tests can end up there and the more rotten these tests become. >>> >>> Thanks, >>> -- Igor >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> and hence it's moved to test/jdk/tools/launcher. as hotspot/compiler and hotspot/gc tests use packages, the tests moved there have been updated to have a package statement. >>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219139 >>>>> testing: tier[1-2] (in progress), all the touched tests locally >>>>> Thanks, >>>>> -- Igor From mikhailo.seledtsov at oracle.com Thu Mar 14 22:53:07 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Thu, 14 Mar 2019 15:53:07 -0700 Subject: RFR(S) : 8219139 : move hotspot tests from test/jdk/vm In-Reply-To: <726FFE91-7F8C-46F6-9DE9-A112F5C5C17B@oracle.com> References: <036DE223-2FAB-43A4-BAC7-63A9BBFCA65D@oracle.com> <132812c3-56be-edfd-f812-78b30c75973b@oracle.com> <511d81a3-847c-49d0-76ff-fdc836211f37@oracle.com> <726FFE91-7F8C-46F6-9DE9-A112F5C5C17B@oracle.com> Message-ID: <3bd885b4-1e50-dbb4-10f2-31e4b274354b@oracle.com> Looks good to me, Thank you, Misha On 3/14/19 2:38 PM, Igor Ignatyev wrote: > Hi Misha, > > thanks for your suggestions, I have moved all runtime tests into > subdirectories. here is the updated webrev: > http://cr.openjdk.java.net/~iignatyev//8219139/webrev.01/index.html > > > Thanks, > -- Igor > >> On Mar 4, 2019, at 1:57 PM, mikhailo.seledtsov at oracle.com >> wrote: >> >> Hi Igor, >> >> ? Sorry it took a while to get back to you on this one. See my >> comment below >> >> >> On 2/22/19 10:35 AM,mikhailo.seledtsov at oracle.com >> wrote: >>> Overall the change looks good; thank you Igor for doing this. I have >>> couple of comments: >>> >>> - I am in favor of the approach where we move tests first under >>> corresponding sub-component directories in >>> test/hotspot sub-tree, and then figure out whether to keep them. >>> Even though in general I am in favor >>> of removing tests that are obsolete or of questionable value, this >>> requires time, consideration and discussions. >>> Hence, I recommend filing an RFE for that, which can be addressed >>> after the migration. >>> >>> - Runtime normally avoids tests directly in test/hotspot/jtreg/runtime >>> The tests should go into underlying sub-directories which best match >>> functional area or topic of that test. >>> In most cases you did it in your change, but there are several tests >>> that your change places directly under >>> test/hotspot/jtreg/runtime/: >>> >>> ExplicitArithmeticCheck.java >>> MonitorCacheMaybeExpand_DeadLock.java >>> ReflectStackOverflow.java >>> ShiftTest.java - David commented this can go under compiler (a jit test) >>> WideStrictInline.java >> I have looked at the tests in more detail, and here are my >> recommendation of placements: >> ??? ExplicitArithmeticCheck >> ??????? This test checks that ArithmeticException is thrown when >> appropriate >> ??????? I would recommend placing it under runtime/ErrorHandling >> MonitorCacheMaybeExpand_DeadLock >> ??????? Existing folder: runtime/Thread (it does have a locking test) >> ??????? Or, alternatively, create a new folder: 'locking' or 'monitors' >> ??? ReflectStackOverflow >> ??????? Uses recursive reflection attempting to provoke stack overflow >> ??????? Can go under: runtime/reflect >> ??? WideStrictInline: >> ??????? checks for correct FP inlining by the interpreter >> ??????? I could not find existing sections; perhaps create 'interpreter' >> ??????? folder under 'runtime' >> >> Thank you, >> Misha >>> >>> Since we plan (as discussed) to follow up this work with an RFE to >>> review and consider removal of old and >>> not-that-useful tests, you could place them under 'misc' for now. >>> Alternatively, find the best match >>> or create new sub-directories under runtime/ if necessary. >>> >>> >>> Thank you, >>> Misha >>> >>> >>> On 2/21/19 11:53 AM, Igor Ignatyev wrote: >>>> >>>>> On Feb 21, 2019, at 12:11 AM, David Holmes >>>>> > wrote: >>>>> >>>>> Hi Igor, >>>>> >>>>> On 21/02/2019 3:19 pm, Igor Ignatyev wrote: >>>>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>>> >>>>>>> 40 lines changed: 17 ins; 2 del; 21 mod; >>>>>> Hi all, >>>>>> could you please review this small patch which moves tests from >>>>>> test/jdk/vm? >>>>> I find some of these tests - the runtime ones at least - of >>>>> extremely dubious value. They either cover basic functionality >>>>> that is already well covered, or are regression tests for bugs in >>>>> code that hasn't existed for many many years! >>>> as I wrote in another related email: "one of the reason I'm >>>> proposing this move is exactly questionable value of these tests, I >>>> want to believe that having these tests in hotspot/ test >>>> directories will bring more attention to them from corresponding >>>> component teams and hence they will be removed/reworked/re-whatever >>>> faster and better. and I also believe that one of the reason we got >>>> duplications exactly because these tests were located in jdk test >>>> suite." >>>> >>>>> BTW: >>>>> >>>>> test/hotspot/jtreg/runtime/ShiftTest.java >>>>> >>>>> is actually a jit test according to the test comment. >>>> sure, I will move it to hotspot/compiler. >>>>>> there are 16 tests in test/jdk/vm directory. all but >>>>>> JniInvocationTest are hotspot tests, so they are moved to >>>>>> test/hotspot test suite; JniInvocationTest is a launcher test >>>>> No its a JNI invocation API test - nothing to do with our >>>>> launcher. It belongs in runtime/jni. But we already have tests in >>>>> runtime that use the JNI invocation API so this test adds no new >>>>> coverage. >>>> this is actually was my first reaction, and I even have the webrev >>>> which moves it to runtime/jni, but then I looked at the associated >>>> bug and it is filed against tools/launcher. and I even got a false >>>> (as I know by now) memory that I saw JLI_ method being called from >>>> the test. there is actually another test >>>> (dk/tools/launcher/exeJliLaunchTest.c) associated w/ this bug which >>>> calls JLI_Launch. anyhow, I'll move this test to hotspot/runtime/jni. >>>> >>>>> I really think the value of these tests needs to be examined >>>>> before they are brought over. >>>> I'd prefer to have follow-up RFEs/tasks, b/c the longer we keep >>>> jdk/vm directory the more tests can end up there and the more >>>> rotten these tests become. >>>> >>>> Thanks, >>>> -- Igor >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> and hence it's moved to test/jdk/tools/launcher. as >>>>>> hotspot/compiler and hotspot/gc tests use packages, the tests >>>>>> moved there have been updated to have a package statement. >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219139 >>>>>> testing: tier[1-2] (in progress), all the touched tests locally >>>>>> Thanks, >>>>>> -- Igor > From yumin.qi at gmail.com Thu Mar 14 22:57:09 2019 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 14 Mar 2019 15:57:09 -0700 Subject: RFR: 8203832: JWarmup precompile java hot methods at application startup Message-ID: Please review the patch for implementing JEP 8203832: Bug: https://bugs.openjdk.java.net/browse/JDK-8203832 Webrev: http://cr.openjdk.java.net/~minqi/8203832/webrev-01/ Summary: This patch implements the first version of JIT warmup to solve the java application peak time CPU contest between JIT compiler threads and regula r application java threads. The CPU contest is avoided by pre-run to collect comilation data and precompile the hot java methods during application startu p based on the pre-run collected data. The main flags (-XX:) used to control are: CompilationWarmUpRecording: used for collecting compilation data. Result stored in a file set by flag CompilationWarmUpLogfile. CompilationWarmUp: use collected data to precompile the methods recorded in above file. CompilationWarmUpExclude: configure a file which contains the names to be excluded for precompile. CompilationWarmUpRecordTime: control the warmup recording time. CompilationWarmUpDeoptTime: time (from start) to deoptimize precomiled methods. In this version, the profiled method data is not used at precomilation, it will be addressed in followed bug fix. After the first version integrated, will file bug f or it. Also, it does not work with CDS since the data added to ConstantPool data structure. It also will be addressed in another separate bug followed. Test: Submitted to submit repo and passed all tests. Newly added test suites: test/hotspot/jtreg/jwarmup include test cases for most of the use cases and issues. Thanks Yumin From igor.ignatyev at oracle.com Thu Mar 14 23:49:47 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 14 Mar 2019 16:49:47 -0700 Subject: RFR(S) : 8219139 : move hotspot tests from test/jdk/vm In-Reply-To: <3bd885b4-1e50-dbb4-10f2-31e4b274354b@oracle.com> References: <036DE223-2FAB-43A4-BAC7-63A9BBFCA65D@oracle.com> <132812c3-56be-edfd-f812-78b30c75973b@oracle.com> <511d81a3-847c-49d0-76ff-fdc836211f37@oracle.com> <726FFE91-7F8C-46F6-9DE9-A112F5C5C17B@oracle.com> <3bd885b4-1e50-dbb4-10f2-31e4b274354b@oracle.com> Message-ID: Thanks for the review Misha. can I get a LGTM from a Reviewer? -- Igor > On Mar 14, 2019, at 3:53 PM, mikhailo.seledtsov at oracle.com wrote: > > Looks good to me, > > Thank you, > > Misha > > > On 3/14/19 2:38 PM, Igor Ignatyev wrote: >> Hi Misha, >> >> thanks for your suggestions, I have moved all runtime tests into subdirectories. here is the updated webrev: http://cr.openjdk.java.net/~iignatyev//8219139/webrev.01/index.html >> >> Thanks, >> -- Igor >> >>> On Mar 4, 2019, at 1:57 PM, mikhailo.seledtsov at oracle.com wrote: >>> >>> Hi Igor, >>> >>> Sorry it took a while to get back to you on this one. See my comment below >>> >>> >>> On 2/22/19 10:35 AM, mikhailo.seledtsov at oracle.com wrote: >>>> Overall the change looks good; thank you Igor for doing this. I have couple of comments: >>>> >>>> - I am in favor of the approach where we move tests first under corresponding sub-component directories in >>>> test/hotspot sub-tree, and then figure out whether to keep them. Even though in general I am in favor >>>> of removing tests that are obsolete or of questionable value, this requires time, consideration and discussions. >>>> Hence, I recommend filing an RFE for that, which can be addressed after the migration. >>>> >>>> - Runtime normally avoids tests directly in test/hotspot/jtreg/runtime >>>> The tests should go into underlying sub-directories which best match functional area or topic of that test. >>>> In most cases you did it in your change, but there are several tests that your change places directly under >>>> test/hotspot/jtreg/runtime/: >>>> >>>> ExplicitArithmeticCheck.java >>>> MonitorCacheMaybeExpand_DeadLock.java >>>> ReflectStackOverflow.java >>>> ShiftTest.java - David commented this can go under compiler (a jit test) >>>> WideStrictInline.java >>> I have looked at the tests in more detail, and here are my recommendation of placements: >>> ExplicitArithmeticCheck >>> This test checks that ArithmeticException is thrown when appropriate >>> I would recommend placing it under runtime/ErrorHandling >>> MonitorCacheMaybeExpand_DeadLock >>> Existing folder: runtime/Thread (it does have a locking test) >>> Or, alternatively, create a new folder: 'locking' or 'monitors' >>> ReflectStackOverflow >>> Uses recursive reflection attempting to provoke stack overflow >>> Can go under: runtime/reflect >>> WideStrictInline: >>> checks for correct FP inlining by the interpreter >>> I could not find existing sections; perhaps create 'interpreter' >>> folder under 'runtime' >>> >>> Thank you, >>> Misha >>>> >>>> Since we plan (as discussed) to follow up this work with an RFE to review and consider removal of old and >>>> not-that-useful tests, you could place them under 'misc' for now. Alternatively, find the best match >>>> or create new sub-directories under runtime/ if necessary. >>>> >>>> >>>> Thank you, >>>> Misha >>>> >>>> >>>> On 2/21/19 11:53 AM, Igor Ignatyev wrote: >>>>> >>>>>> On Feb 21, 2019, at 12:11 AM, David Holmes > wrote: >>>>>> >>>>>> Hi Igor, >>>>>> >>>>>> On 21/02/2019 3:19 pm, Igor Ignatyev wrote: >>>>>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>>>>> 40 lines changed: 17 ins; 2 del; 21 mod; >>>>>>> Hi all, >>>>>>> could you please review this small patch which moves tests from test/jdk/vm? >>>>>> I find some of these tests - the runtime ones at least - of extremely dubious value. They either cover basic functionality that is already well covered, or are regression tests for bugs in code that hasn't existed for many many years! >>>>> as I wrote in another related email: "one of the reason I'm proposing this move is exactly questionable value of these tests, I want to believe that having these tests in hotspot/ test directories will bring more attention to them from corresponding component teams and hence they will be removed/reworked/re-whatever faster and better. and I also believe that one of the reason we got duplications exactly because these tests were located in jdk test suite." >>>>> >>>>>> BTW: >>>>>> >>>>>> test/hotspot/jtreg/runtime/ShiftTest.java >>>>>> >>>>>> is actually a jit test according to the test comment. >>>>> sure, I will move it to hotspot/compiler. >>>>>>> there are 16 tests in test/jdk/vm directory. all but JniInvocationTest are hotspot tests, so they are moved to test/hotspot test suite; JniInvocationTest is a launcher test >>>>>> No its a JNI invocation API test - nothing to do with our launcher. It belongs in runtime/jni. But we already have tests in runtime that use the JNI invocation API so this test adds no new coverage. >>>>> this is actually was my first reaction, and I even have the webrev which moves it to runtime/jni, but then I looked at the associated bug and it is filed against tools/launcher. and I even got a false (as I know by now) memory that I saw JLI_ method being called from the test. there is actually another test (dk/tools/launcher/exeJliLaunchTest.c) associated w/ this bug which calls JLI_Launch. anyhow, I'll move this test to hotspot/runtime/jni. >>>>> >>>>>> I really think the value of these tests needs to be examined before they are brought over. >>>>> I'd prefer to have follow-up RFEs/tasks, b/c the longer we keep jdk/vm directory the more tests can end up there and the more rotten these tests become. >>>>> >>>>> Thanks, >>>>> -- Igor >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> and hence it's moved to test/jdk/tools/launcher. as hotspot/compiler and hotspot/gc tests use packages, the tests moved there have been updated to have a package statement. >>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219139 >>>>>>> testing: tier[1-2] (in progress), all the touched tests locally >>>>>>> Thanks, >>>>>>> -- Igor >> > From jamsheed.c.m at oracle.com Fri Mar 15 00:07:42 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Fri, 15 Mar 2019 05:37:42 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <144af13e-33e2-2089-f8d2-902e94485eb1@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> <144af13e-33e2-2089-f8d2-902e94485eb1@oracle.com> Message-ID: <62ad6130-0498-5222-43c7-befa3cea9741@oracle.com> Hi Tobias, as a fix, for handling? the bugs in hand written assembly stubs, i will put checks like.[1]? instead of StubRoutines::contains(pc). Best regards, Jamsheed [1] bool StubRoutines::is_unsafe_copy(address pc) { ??? if ((pc > StubRoutines::_jbyte_disjoint_arraycopy && pc < StubRoutines::_arrayof_jbyte_disjoint_arraycopy) || ??????? (pc > StubRoutines::_jshort_disjoint_arraycopy && pc < StubRoutines::_arrayof_jshort_disjoint_arraycopy) || ??????? (pc > StubRoutines::_jint_disjoint_arraycopy && pc < StubRoutines::_arrayof_oop_disjoint_arraycopy)) { ????? return true; ??? } ??? return false; ? } PS: these code ranges are shared by others too. On 14/03/19 8:46 PM, Jamsheed wrote: > Hi Tobias, > > On 14/03/19 1:58 PM, Tobias Hartmann wrote: >> Hi Jamsheed, >> >> On 14.03.19 02:29, Jamsheed wrote: >>> i will try to limit my checks to array copy stubs. will make changes >>> and send it for review. >> Okay, if there's a simple way to do that. Otherwise, I'm fine with >> the current version of the fix. > either i should have separate code generated for unsafe copy. or > setting/resetting doing_unsafe_access before after unsafe_copy should do. > > Best regards, > > Jamsheed >> >> Best regards, >> Tobias From david.holmes at oracle.com Fri Mar 15 00:49:48 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 15 Mar 2019 10:49:48 +1000 Subject: RFR: 8203832: JWarmup precompile java hot methods at application startup In-Reply-To: References: Message-ID: <7b8e98b9-bf0c-0d48-2a4e-5196ccbf9eaa@oracle.com> Hi Yumin, On 15/03/2019 8:57 am, yumin qi wrote: > Please review the patch for implementing JEP 8203832: It seems a bit premature for a RFR at this point given the JEP is still a draft. Perhaps a RFC is more appropriate? You need to create an Enhancement issue in JBS, linked to the JEP, to be used to push any changes once the JEP has been targeted, then issue your RFC/RFR under that bug id. Thanks, David ----- > Bug: https://bugs.openjdk.java.net/browse/JDK-8203832 > Webrev: http://cr.openjdk.java.net/~minqi/8203832/webrev-01/ > > Summary: This patch implements the first version of JIT warmup to > solve the java application peak time CPU contest between JIT compiler > threads and regula > r application java threads. The CPU contest is avoided by pre-run to > collect comilation data and precompile the hot java methods during > application startu > p based on the pre-run collected data. > > The main flags (-XX:) used to control are: > CompilationWarmUpRecording: used for collecting compilation data. > Result stored in a file set by flag CompilationWarmUpLogfile. > CompilationWarmUp: use collected data to precompile the methods > recorded in above file. > CompilationWarmUpExclude: configure a file which contains the names to > be excluded for precompile. > CompilationWarmUpRecordTime: control the warmup recording time. > CompilationWarmUpDeoptTime: time (from start) to deoptimize precomiled methods. > > In this version, the profiled method data is not used at > precomilation, it will be addressed in followed bug fix. After the > first version integrated, will file bug f > or it. Also, it does not work with CDS since the data added to > ConstantPool data structure. It also will be addressed in another > separate bug followed. > > Test: Submitted to submit repo and passed all tests. > Newly added test suites: test/hotspot/jtreg/jwarmup include test > cases for most of the use cases and issues. > > Thanks > Yumin > From david.holmes at oracle.com Fri Mar 15 01:18:12 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 15 Mar 2019 11:18:12 +1000 Subject: RFR(S) : 8219139 : move hotspot tests from test/jdk/vm In-Reply-To: <726FFE91-7F8C-46F6-9DE9-A112F5C5C17B@oracle.com> References: <036DE223-2FAB-43A4-BAC7-63A9BBFCA65D@oracle.com> <132812c3-56be-edfd-f812-78b30c75973b@oracle.com> <511d81a3-847c-49d0-76ff-fdc836211f37@oracle.com> <726FFE91-7F8C-46F6-9DE9-A112F5C5C17B@oracle.com> Message-ID: <96d6b416-f9cb-9c44-922a-dda93fd50cce@oracle.com> Hi Igor, This all seems fine to me. Thanks, David On 15/03/2019 7:38 am, Igor Ignatyev wrote: > Hi Misha, > > thanks for your suggestions, I have moved all runtime tests into > subdirectories. here is the updated webrev: > http://cr.openjdk.java.net/~iignatyev//8219139/webrev.01/index.html > > Thanks, > -- Igor > >> On Mar 4, 2019, at 1:57 PM, mikhailo.seledtsov at oracle.com >> wrote: >> >> Hi Igor, >> >> ? Sorry it took a while to get back to you on this one. See my comment >> below >> >> >> On 2/22/19 10:35 AM,mikhailo.seledtsov at oracle.com >> wrote: >>> Overall the change looks good; thank you Igor for doing this. I have >>> couple of comments: >>> >>> - I am in favor of the approach where we move tests first under >>> corresponding sub-component directories in >>> test/hotspot sub-tree, and then figure out whether to keep them. Even >>> though in general I am in favor >>> of removing tests that are obsolete or of questionable value, this >>> requires time, consideration and discussions. >>> Hence, I recommend filing an RFE for that, which can be addressed >>> after the migration. >>> >>> - Runtime normally avoids tests directly in test/hotspot/jtreg/runtime >>> The tests should go into underlying sub-directories which best match >>> functional area or topic of that test. >>> In most cases you did it in your change, but there are several tests >>> that your change places directly under >>> test/hotspot/jtreg/runtime/: >>> >>> ExplicitArithmeticCheck.java >>> MonitorCacheMaybeExpand_DeadLock.java >>> ReflectStackOverflow.java >>> ShiftTest.java - David commented this can go under compiler (a jit test) >>> WideStrictInline.java >> I have looked at the tests in more detail, and here are my >> recommendation of placements: >> ??? ExplicitArithmeticCheck >> ??????? This test checks that ArithmeticException is thrown when >> appropriate >> ??????? I would recommend placing it under runtime/ErrorHandling >> ??? MonitorCacheMaybeExpand_DeadLock >> ??????? Existing folder: runtime/Thread (it does have a locking test) >> ??????? Or, alternatively, create a new folder: 'locking' or 'monitors' >> ??? ReflectStackOverflow >> ??????? Uses recursive reflection attempting to provoke stack overflow >> ??????? Can go under: runtime/reflect >> ??? WideStrictInline: >> ??????? checks for correct FP inlining by the interpreter >> ??????? I could not find existing sections; perhaps create 'interpreter' >> ??????? folder under 'runtime' >> >> Thank you, >> Misha >>> >>> Since we plan (as discussed) to follow up this work with an RFE to >>> review and consider removal of old and >>> not-that-useful tests, you could place them under 'misc' for now. >>> Alternatively, find the best match >>> or create new sub-directories under runtime/ if necessary. >>> >>> >>> Thank you, >>> Misha >>> >>> >>> On 2/21/19 11:53 AM, Igor Ignatyev wrote: >>>> >>>>> On Feb 21, 2019, at 12:11 AM, David Holmes >>>> > wrote: >>>>> >>>>> Hi Igor, >>>>> >>>>> On 21/02/2019 3:19 pm, Igor Ignatyev wrote: >>>>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>>>> 40 lines changed: 17 ins; 2 del; 21 mod; >>>>>> Hi all, >>>>>> could you please review this small patch which moves tests from >>>>>> test/jdk/vm? >>>>> I find some of these tests - the runtime ones at least - of >>>>> extremely dubious value. They either cover basic functionality that >>>>> is already well covered, or are regression tests for bugs in code >>>>> that hasn't existed for many many years! >>>> as I wrote in another related email: "one of the reason I'm >>>> proposing this move is exactly questionable value of these tests, I >>>> want to believe that having these tests in hotspot/ test directories >>>> will bring more attention to them from corresponding component teams >>>> and hence they will be removed/reworked/re-whatever faster and >>>> better. and I also believe that one of the reason we got >>>> duplications exactly because these tests were located in jdk test >>>> suite." >>>> >>>>> BTW: >>>>> >>>>> test/hotspot/jtreg/runtime/ShiftTest.java >>>>> >>>>> is actually a jit test according to the test comment. >>>> sure, I will move it to hotspot/compiler. >>>>>> there are 16 tests in test/jdk/vm directory. all but >>>>>> JniInvocationTest are hotspot tests, so they are moved to >>>>>> test/hotspot test suite; JniInvocationTest is a launcher test >>>>> No its a JNI invocation API test - nothing to do with our launcher. >>>>> It belongs in runtime/jni. But we already have tests in runtime >>>>> that use the JNI invocation API so this test adds no new coverage. >>>> this is actually was my first reaction, and I even have the webrev >>>> which moves it to runtime/jni, but then I looked at the associated >>>> bug and it is filed against tools/launcher. and I even got a false >>>> (as I know by now) memory that I saw JLI_ method being called from >>>> the test. there is actually another test >>>> (dk/tools/launcher/exeJliLaunchTest.c) associated w/ this bug which >>>> calls JLI_Launch. anyhow, I'll move this test to hotspot/runtime/jni. >>>> >>>>> I really think the value of these tests needs to be examined before >>>>> they are brought over. >>>> I'd prefer to have follow-up RFEs/tasks, b/c the longer we keep >>>> jdk/vm directory the more tests can end up there and the more rotten >>>> these tests become. >>>> >>>> Thanks, >>>> -- Igor >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> and hence it's moved to test/jdk/tools/launcher. as >>>>>> hotspot/compiler and hotspot/gc tests use packages, the tests >>>>>> moved there have been updated to have a package statement. >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~iignatyev//8219139/webrev.00/index.html >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219139 >>>>>> testing: tier[1-2] (in progress), all the touched tests locally >>>>>> Thanks, >>>>>> -- Igor > From david.holmes at oracle.com Fri Mar 15 01:16:19 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 15 Mar 2019 11:16:19 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> <1d1c7c0b-0707-3cd7-3049-43a7bd9c37e3@oracle.com> <26959ebc-82ab-a755-5ef7-82afa30f2c50@redhat.com> Message-ID: <528e0a7b-38b5-1670-606d-7ab1c07481a8@oracle.com> On 15/03/2019 4:36 am, Andrew Haley wrote: > On 3/14/19 6:18 PM, Andrew Haley wrote: >> I have an idea that might work. Let's say that icache::flush(), >> executed whenever we patch code, increments a global patching >> counter. When a Java thread returns from a handshake, it compares a >> thread-local copy of the old value of the patching counter with the >> current value. If they are different, do whatever local fencing is >> necessary and record the current value of the patching counter. >> >> That should work, right? So we only execute necessary barriers, and >> if some port keeps most constants in the constant pool (rather than >> patching) it'll benefit. > > But, I should stress, all of this can be done in platform-specific code, > so the main code base doesn't have to be bothered by any of it. I'm glad to see all the discussion on this - it is multi-faceted and a typical "multi-reader/single-writer" style synchronization problem: depending on the platform we may have to do something in the writer (patching thread), something in the reader (patchee thread), or both. The patching thread changes are most likely in platform specific code. But the patchee thread, AFAICS, will be executing shared handshake/safepoint code, so there has to be some kind of barrier function added there, even if it is empty on some platforms. Optimising the barrier is then a second level problem. Tracking when the barrier is actually needed is desirable as long as the bookkeeping cost doesn't outweigh the actual barrier cost. I suppose that if the optimising results in the barrier locations being pushed "deep enough" they may end up in platform-specific code after all. But it's far from clear to me that we will always enter platform specific code. Cheers, David ----- >> And, just to complicate this even more, on some processors you'd not >> only need to to do local fencing but you might need to know which >> memory addresses had changed. > > For avoidance of doubt, I don't think that we support any such processors. > From yumin.qi at gmail.com Fri Mar 15 01:56:53 2019 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 14 Mar 2019 18:56:53 -0700 Subject: RFR: 8203832: JWarmup precompile java hot methods at application startup In-Reply-To: <7b8e98b9-bf0c-0d48-2a4e-5196ccbf9eaa@oracle.com> References: <7b8e98b9-bf0c-0d48-2a4e-5196ccbf9eaa@oracle.com> Message-ID: Hi, David Thanks for the information. I will create an Enhancement issue in JBS. Not aware of this procedure. Thanks Yumin On Thu, Mar 14, 2019 at 5:49 PM David Holmes wrote: > Hi Yumin, > > On 15/03/2019 8:57 am, yumin qi wrote: > > Please review the patch for implementing JEP 8203832: > > It seems a bit premature for a RFR at this point given the JEP is still > a draft. Perhaps a RFC is more appropriate? > > You need to create an Enhancement issue in JBS, linked to the JEP, to be > used to push any changes once the JEP has been targeted, then issue your > RFC/RFR under that bug id. > > Thanks, > David > ----- > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203832 > > Webrev: http://cr.openjdk.java.net/~minqi/8203832/webrev-01/ > > > > Summary: This patch implements the first version of JIT warmup to > > solve the java application peak time CPU contest between JIT compiler > > threads and regula > > r application java threads. The CPU contest is avoided by pre-run to > > collect comilation data and precompile the hot java methods during > > application startu > > p based on the pre-run collected data. > > > > The main flags (-XX:) used to control are: > > CompilationWarmUpRecording: used for collecting compilation data. > > Result stored in a file set by flag CompilationWarmUpLogfile. > > CompilationWarmUp: use collected data to precompile the methods > > recorded in above file. > > CompilationWarmUpExclude: configure a file which contains the names to > > be excluded for precompile. > > CompilationWarmUpRecordTime: control the warmup recording time. > > CompilationWarmUpDeoptTime: time (from start) to deoptimize precomiled > methods. > > > > In this version, the profiled method data is not used at > > precomilation, it will be addressed in followed bug fix. After the > > first version integrated, will file bug f > > or it. Also, it does not work with CDS since the data added to > > ConstantPool data structure. It also will be addressed in another > > separate bug followed. > > > > Test: Submitted to submit repo and passed all tests. > > Newly added test suites: test/hotspot/jtreg/jwarmup include test > > cases for most of the use cases and issues. > > > > Thanks > > Yumin > > > From yumin.qi at gmail.com Fri Mar 15 05:59:23 2019 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 14 Mar 2019 22:59:23 -0700 Subject: RFC: JWarmup precompile java hot methods at application startup Message-ID: Please review the patch for implementing JEP 8203832: (*https://bugs.openjdk.java.net/browse/JDK-8203832 )* Bug: *https://bugs.openjdk.java.net/browse/JDK-8220692 * Webrev: http://cr.openjdk.java.net/~minqi/8220692/webrev-01/ Summary: This patch implements the first version of JIT warmup to solve the java application peak time CPU contest between JIT compiler threads and regular application java threads. The CPU contest is avoided by pre-run to collect comilation data and precompile the hot java methods during application startup based on the pre-run collected data. The main flags (-XX:) used to control are: CompilationWarmUpRecording: used for collecting compilation data. Result stored in a file set by flag CompilationWarmUpLogfile. CompilationWarmUp: use collected data to precompile the methods recorded in above file. CompilationWarmUpExclude: configure a file which contains the names to be excluded for precompile. CompilationWarmUpRecordTime: control the warmup recording time. CompilationWarmUpDeoptTime: time (from start) to deoptimize precomiled methods. In this version, the profiled method data is not used at precomilation, it will be addressed in followed bug fix. After the first version integrated, will file bug for it. Also, it does not work with CDS since the data added to ConstantPool data structure. It also will be addressed in another separate bug followed. Test: Submitted to submit repo and passed all tests. Newly added test suites: test/hotspot/jtreg/jwarmup include test cases for most of the use cases and issues. Thanks Yumin From Alan.Bateman at oracle.com Fri Mar 15 07:30:08 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Mar 2019 07:30:08 +0000 Subject: RFC: JWarmup precompile java hot methods at application startup In-Reply-To: References: Message-ID: On 15/03/2019 05:59, yumin qi wrote: > Please review the patch for implementing JEP 8203832: > (*https://bugs.openjdk.java.net/browse/JDK-8203832 > )* > > Bug: *https://bugs.openjdk.java.net/browse/JDK-8220692 > * > Webrev: http://cr.openjdk.java.net/~minqi/8220692/webrev-01/ > > > Summary: This patch implements the first version of JIT warmup to > solve the java application peak time CPU contest between JIT compiler > threads and regular application java threads. The CPU contest isa > avoided by pre-run to collect comilation data and precompile the hot > java methods during application startup based on the pre-run collected > data. > com.alibaba.jwarmup will need to find a new home, it cannot be exported by java.base. I think you'll need to look at create a new module for this API if it's really needed, it will need a better package name too. -Alan. From tobias.hartmann at oracle.com Fri Mar 15 07:44:46 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 15 Mar 2019 08:44:46 +0100 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: <62ad6130-0498-5222-43c7-befa3cea9741@oracle.com> References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> <144af13e-33e2-2089-f8d2-902e94485eb1@oracle.com> <62ad6130-0498-5222-43c7-befa3cea9741@oracle.com> Message-ID: Hi Jamsheed, okay, that looks reasonable to me! Best regards, Tobias On 15.03.19 01:07, Jamsheed wrote: > Hi Tobias, > > as a fix, for handling? the bugs in hand written assembly stubs, i will put checks like.[1]? instead > of StubRoutines::contains(pc). > > Best regards, > > Jamsheed > > [1] > > bool StubRoutines::is_unsafe_copy(address pc) { > ??? if ((pc > StubRoutines::_jbyte_disjoint_arraycopy && pc < > StubRoutines::_arrayof_jbyte_disjoint_arraycopy) || > ??????? (pc > StubRoutines::_jshort_disjoint_arraycopy && pc < > StubRoutines::_arrayof_jshort_disjoint_arraycopy) || > ??????? (pc > StubRoutines::_jint_disjoint_arraycopy && pc < > StubRoutines::_arrayof_oop_disjoint_arraycopy)) { > ????? return true; > ??? } > ??? return false; > ? } > > PS: these code ranges are shared by others too. > > On 14/03/19 8:46 PM, Jamsheed wrote: >> Hi Tobias, >> >> On 14/03/19 1:58 PM, Tobias Hartmann wrote: >>> Hi Jamsheed, >>> >>> On 14.03.19 02:29, Jamsheed wrote: >>>> i will try to limit my checks to array copy stubs. will make changes and send it for review. >>> Okay, if there's a simple way to do that. Otherwise, I'm fine with the current version of the fix. >> either i should have separate code generated for unsafe copy. or setting/resetting >> doing_unsafe_access before after unsafe_copy should do. >> >> Best regards, >> >> Jamsheed >>> >>> Best regards, >>> Tobias From aph at redhat.com Fri Mar 15 09:42:50 2019 From: aph at redhat.com (Andrew Haley) Date: Fri, 15 Mar 2019 09:42:50 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <528e0a7b-38b5-1670-606d-7ab1c07481a8@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <16f70099-4a57-5801-e667-2a6d305d6051@oracle.com> <1d1c7c0b-0707-3cd7-3049-43a7bd9c37e3@oracle.com> <26959ebc-82ab-a755-5ef7-82afa30f2c50@redhat.com> <528e0a7b-38b5-1670-606d-7ab1c07481a8@oracle.com> Message-ID: <4f297814-277b-e6e8-c966-b96a1be4a62d@redhat.com> On 3/15/19 1:16 AM, David Holmes wrote: > I suppose that if the optimising results in the barrier locations being > pushed "deep enough" they may end up in platform-specific code after > all. But it's far from clear to me that we will always enter platform > specific code. On AArch64 we do an ISB when returning from native to Java. I can't see any code path which can execute patched Java code but does not do that. That doesn't mean there isn't one, I guess. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From tobias.hartmann at oracle.com Fri Mar 15 10:55:46 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 15 Mar 2019 11:55:46 +0100 Subject: RFC: JWarmup precompile java hot methods at application startup In-Reply-To: References: Message-ID: Hi Yumin, I haven't looked at the code yet but just executed some sanity testing. Many of the new jwarmup tests crash the VM on Sparc: # SIGBUS (0xa) at pc=0xffffffff7d865780, pid=27039, tid=8 # # JRE version: Java(TM) SE Runtime Environment (13.0) (fastdebug build 13-internal+0-2019-03-15-0750126.tobias.hartmann.jdkjdk) # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 13-internal+0-2019-03-15-0750126.tobias.hartmann.jdkjdk, compiled mode, compressed oops, g1 gc, solaris-sparc) # Problematic frame: # V [libjvm.so+0x1865780] void ProfileRecorder::write_header()+0xf0 Stack: [0xffffffff54600000,0xffffffff54700000], sp=0xffffffff546ff960, free space=1022k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x1865780] void ProfileRecorder::write_header()+0xf0 V [libjvm.so+0x1866800] void ProfileRecorder::flush()+0xa0 V [libjvm.so+0x186426c] JitWarmUp::JitWarmUpState JitWarmUp::flush_logfile()+0x1c V [libjvm.so+0x187059c] void JitWarmUpFlushThread::run()+0x9c V [libjvm.so+0x227c8f8] void Thread::call_run()+0x178 V [libjvm.so+0x1f4d32c] thread_native_entry+0x3ac And some tests fail when executed with a different GC due to: Error occurred during initialization of VM Multiple garbage collectors selected They should either not specify the GC or have a @requires vm.gc == "null" Also, these tests should be moved to hotspot/jtreg/compiler/ or ../runtime/ Best regards, Tobias On 15.03.19 06:59, yumin qi wrote: > Please review the patch for implementing JEP 8203832: > (*https://bugs.openjdk.java.net/browse/JDK-8203832 > )* > > Bug: *https://bugs.openjdk.java.net/browse/JDK-8220692 > * > Webrev: http://cr.openjdk.java.net/~minqi/8220692/webrev-01/ > > > Summary: This patch implements the first version of JIT warmup to > solve the java application peak time CPU contest between JIT compiler > threads and regular application java threads. The CPU contest is > avoided by pre-run to collect comilation data and precompile the hot > java methods during application startup based on the pre-run collected > data. > > The main flags (-XX:) used to control are: > CompilationWarmUpRecording: used for collecting compilation data. > Result stored in a file set by flag CompilationWarmUpLogfile. > CompilationWarmUp: use collected data to precompile the methods > recorded in above file. > CompilationWarmUpExclude: configure a file which contains the names to > be excluded for precompile. > CompilationWarmUpRecordTime: control the warmup recording time. > CompilationWarmUpDeoptTime: time (from start) to deoptimize precomiled methods. > > In this version, the profiled method data is not used at > precomilation, it will be addressed in followed bug fix. After the > first version integrated, will file bug for it. Also, it does not work > with CDS since the data added to ConstantPool data structure. It also > will be addressed in another separate bug followed. > > Test: Submitted to submit repo and passed all tests. > Newly added test suites: test/hotspot/jtreg/jwarmup include test > cases for most of the use cases and issues. > > Thanks > Yumin > From richard.reingruber at sap.com Fri Mar 15 11:08:12 2019 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 15 Mar 2019 11:08:12 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: Hi Robbin, I'm referring to x86 here. My comment/question about the requirement of an ICache invalidation in the patcher thread was a little "undereducated". After reading recent manuals [1]: 1. I think your right that a serializing instruction after the safepoint poll is necessary. 2. I doubt that cache flushing is required at all, according to [1]. At least SPECjvm2008 runs with an AbstractICache::_flush_icache_stub that immediately returns. In the sense of [1] JIT compilers and JIT runtimes are cross-modifying code, too, when they compile bytecode and resolve compiled calls. But afaik the java threads don't execute a serializing instruction before executing that code. So, if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee threads jump to the modified code is legal as well. So at least theoretically you should be able to replace the serializing instruction with a call to an address you would load from the thread. The address should lead to the runtime, where the thread would block if the safepoint is still active or just return if it is not. The last point was rather exploring the space of legal solutions than a suggestion to solve the issue. I'd favor your approach if you manage to do the serialization conditionally (BTW: a mechanism similar to JavaThread::*deopt_suspend* would be yet another way to do that). Cheers, Richard. [1] 8.1.3 Handling Self- and Cross-Modifying Code in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 (3A, 3B, 3C & 3D): System Programming Guide https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf on https://software.intel.com/en-us/articles/intel-sdm -----Original Message----- From: Robbin Ehn Sent: Donnerstag, 14. M?rz 2019 13:24 To: Reingruber, Richard ; Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code Hi Richard, On 3/14/19 11:12 AM, Reingruber, Richard wrote: > Hi Robbin, > > > the reason I heard for not doing invalidate_range/word in nmethod::oops_do > > is that it was deemed that a safepoint is 'long enough'. > > This doesn't seem to be a very good reason. I did a little modification to your > initial figure: That's why we are adding the instruction barrier! > > JavaThread: | VMThread > StoreLoad | > | update > | ? Global ICache Invalidate ? > | disarm > load thread_poll | > > Question would be: does an ICache flush happen after the immediate oop update? > I would consider it problematic if not. Even if the safepoint was very long. The icache flush assumes it can stop speculation and flush instruction cache in patchee thread. Correct me if anything is wrong: - sparc needs flush in all patchee threads. - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. - ppc needs isync in all patchee threads? - s390 needs zarch_sync all in patchee threads? - x86 needs cpuid in all in patchee threads. So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) have no effect in patchee threads!? Adding a few words on x86 ICache::Invalidate/word. We have TSO, so if the patchee thread can see poll is disarmed all other stores are visible. I have seen no case ICache::Invalidate/word would make a difference from a StoreStore/compiler barrier (still talking about platform code for x86) ? The problem is if we prefect an instruction over the load of the poll. If poll is disarmed this perfected instruction can be fully executed. If that happens to contain an immediate oop, we don't know if it's the patched oop or not. /Robbin > > Cheers Richard. > > -----Original Message----- > From: Robbin Ehn > Sent: Donnerstag, 14. M?rz 2019 10:34 > To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net > Cc: Reingruber, Richard > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Martin, > > On 3/14/19 10:25 AM, Doerr, Martin wrote: >> Hi Robbin, >> >>> But that is whole new subject and let's keep that separate from this :) >> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >> >> >> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >> >> orderAccess_aix/linux_ppc.hpp: >> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >> >> orderAccess_linux_s390.hpp: >> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >> > > Thanks! > >>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>> I couldn't find a call to AbstractICache::invalidate_range. > > I think I miss-understood you here, the reason I heard for not doing > invalidate_range/word in nmethod::oops_do is that it was deemed that a > safepoint is 'long enough'. > > /Robbin > >> >> Look for ICache::invalidate_range, I see 11 calls on x64. >> And one call to AbstractICache/ICache::invalidate_word. >> Btw I have spent some time looking at x86/icache_x86.cpp... >> But that is whole new subject and let's keep that separate from this :) >> >> I'll create issues for some of these topics, which if nothing else should be >> discussed. >> >> /Robbin >> >>> >>> Does anybody know where it is or if it's missing? >>> >>> Best regards, >>> Martin >>> From david.holmes at oracle.com Fri Mar 15 11:55:10 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 15 Mar 2019 21:55:10 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: On 15/03/2019 9:08 pm, Reingruber, Richard wrote: > Hi Robbin, > > I'm referring to x86 here. > > My comment/question about the requirement of an ICache invalidation in the > patcher thread was a little "undereducated". After reading recent manuals [1]: > > 1. I think your right that a serializing instruction after the safepoint poll > is necessary. > > 2. I doubt that cache flushing is required at all, according to [1]. At least > SPECjvm2008 runs with an AbstractICache::_flush_icache_stub that > immediately returns. > > In the sense of [1] JIT compilers and JIT runtimes are cross-modifying code, > too, when they compile bytecode and resolve compiled calls. But afaik the java > threads don't execute a serializing instruction before executing that code. So, > if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee threads jump to the > modified code is legal as well. OPTION 1 is for self-modifying code, not cross-modifying code. David ----- > So at least theoretically you should be able to replace the serializing > instruction with a call to an address you would load from the thread. The > address should lead to the runtime, where the thread would block if the > safepoint is still active or just return if it is not. > > The last point was rather exploring the space of legal solutions than a > suggestion to solve the issue. I'd favor your approach if you manage to do the > serialization conditionally (BTW: a mechanism similar to JavaThread::*deopt_suspend* > would be yet another way to do that). > > Cheers, Richard. > > [1] 8.1.3 Handling Self- and Cross-Modifying Code > in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 > (3A, 3B, 3C & 3D): System Programming Guide > https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf > on https://software.intel.com/en-us/articles/intel-sdm > > -----Original Message----- > From: Robbin Ehn > Sent: Donnerstag, 14. M?rz 2019 13:24 > To: Reingruber, Richard ; Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Richard, > > On 3/14/19 11:12 AM, Reingruber, Richard wrote: >> Hi Robbin, >> >> > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >> > is that it was deemed that a safepoint is 'long enough'. >> >> This doesn't seem to be a very good reason. I did a little modification to your >> initial figure: > > That's why we are adding the instruction barrier! > >> >> JavaThread: | VMThread >> StoreLoad | >> | update >> | ? Global ICache Invalidate ? >> | disarm >> load thread_poll | >> >> Question would be: does an ICache flush happen after the immediate oop update? >> I would consider it problematic if not. Even if the safepoint was very long. > > The icache flush assumes it can stop speculation and flush instruction cache in > patchee thread. > Correct me if anything is wrong: > - sparc needs flush in all patchee threads. > - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. > - ppc needs isync in all patchee threads? > - s390 needs zarch_sync all in patchee threads? > - x86 needs cpuid in all in patchee threads. > > So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) > have no effect in patchee threads!? > > Adding a few words on x86 ICache::Invalidate/word. > We have TSO, so if the patchee thread can see poll is disarmed all other stores > are visible. I have seen no case ICache::Invalidate/word would make a difference > from a StoreStore/compiler barrier (still talking about platform code for x86) ? > The problem is if we prefect an instruction over the load of the poll. > If poll is disarmed this perfected instruction can be fully executed. > If that happens to contain an immediate oop, we don't know if it's the patched > oop or not. > > /Robbin > >> >> Cheers Richard. >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Donnerstag, 14. M?rz 2019 10:34 >> To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >> Cc: Reingruber, Richard >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Martin, >> >> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>> Hi Robbin, >>> >>>> But that is whole new subject and let's keep that separate from this :) >>> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >>> >>> >>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>> >>> orderAccess_aix/linux_ppc.hpp: >>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>> >>> orderAccess_linux_s390.hpp: >>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>> >> >> Thanks! >> >>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>>> I couldn't find a call to AbstractICache::invalidate_range. >> >> I think I miss-understood you here, the reason I heard for not doing >> invalidate_range/word in nmethod::oops_do is that it was deemed that a >> safepoint is 'long enough'. >> >> /Robbin >> >>> >>> Look for ICache::invalidate_range, I see 11 calls on x64. >>> And one call to AbstractICache/ICache::invalidate_word. >>> Btw I have spent some time looking at x86/icache_x86.cpp... >>> But that is whole new subject and let's keep that separate from this :) >>> >>> I'll create issues for some of these topics, which if nothing else should be >>> discussed. >>> >>> /Robbin >>> >>>> >>>> Does anybody know where it is or if it's missing? >>>> >>>> Best regards, >>>> Martin >>>> From Alan.Bateman at oracle.com Fri Mar 15 12:19:08 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 15 Mar 2019 12:19:08 +0000 Subject: RFR(S) : 8219139 : move hotspot tests from test/jdk/vm In-Reply-To: References: <036DE223-2FAB-43A4-BAC7-63A9BBFCA65D@oracle.com> <0c282875-2f0b-ed17-4a05-621bd5e0d862@oracle.com> Message-ID: <5fdb4be4-ecd6-ca5f-1cfe-019d18eba7c9@oracle.com> On 14/03/2019 21:33, Igor Ignatyev wrote: > Hi Alan, > > I double checked, the test is linked w/ -ljli, and calls JNI_CreateJavaVM from libjli.so. hence I'll leave JniInvocationTest in jdk/tools/launcher. > Thanks for checking, in which case I think we need to find a better name for this test as JniInvocationTest is a misleading name when it's a JLI rather than JNI test. -Alan. From robbin.ehn at oracle.com Fri Mar 15 12:32:16 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 15 Mar 2019 13:32:16 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: <440ce735-add2-db60-b672-d929731304cd@oracle.com> >> >> The last point was rather exploring the space of legal solutions than a >> suggestion to solve the issue. I'd favor your approach if you manage to do the >> serialization conditionally (BTW: a mechanism similar to >> JavaThread::*deopt_suspend* >> would be yet another way to do that). FYI: In the long run I'm removing the suspend-flags and turning suspend/deopt/.. into handshakes instead. So we only have the poll to check. I'm testing a change where I removed the changes to the native wrappers, leaving native threads armed (as Martin suggested) and in the poll path notice that we are still armed (when returning from a safepoint/handshake we should be disarmed). Disarm yourself, re-check safepoint/handshake, execute instruction barrier. So we don't need any changes in any native wrapper, we handle the barrier in the polling code instead. I have just one issue to sort-out with that and benchmarking. The v2 patch (not yet published) didn't show to bad of a result when we always emit cpuid, but there is no super JNI heavy test in that suite. /Robbin >> >> Cheers, Richard. >> >> [1] 8.1.3 Handling Self- and Cross-Modifying Code >> ???? in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >> ???? (3A, 3B, 3C & 3D): System Programming Guide >> >> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >> >> ???? on https://software.intel.com/en-us/articles/intel-sdm >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Donnerstag, 14. M?rz 2019 13:24 >> To: Reingruber, Richard ; Doerr, Martin >> ; David Holmes ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Richard, >> >> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>> Hi Robbin, >>> >>> ??? > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>> ??? > is that it was deemed that a safepoint is 'long enough'. >>> >>> This doesn't seem to be a very good reason. I did a little modification to your >>> initial figure: >> >> That's why we are adding the instruction barrier! >> >>> >>> JavaThread:???????????? | VMThread >>> StoreLoad?????????????? | >>> ????????????????????????? | update >>> ????????????????????????? | ? Global ICache Invalidate ? >>> ????????????????????????? | disarm >>> load thread_poll??????? | >>> >>> Question would be: does an ICache flush happen after the immediate oop update? >>> I would consider it problematic if not. Even if the safepoint was very long. >> >> The icache flush assumes it can stop speculation and flush instruction cache in >> patchee thread. >> Correct me if anything is wrong: >> - sparc needs flush in all patchee threads. >> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >> - ppc needs isync in all patchee threads? >> - s390 needs zarch_sync all in patchee threads? >> - x86 needs cpuid in all in patchee threads. >> >> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >> have no effect in patchee threads!? >> >> Adding a few words on x86 ICache::Invalidate/word. >> We have TSO, so if the patchee thread can see poll is disarmed all other stores >> are visible. I have seen no case ICache::Invalidate/word would make a difference >> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >> The problem is if we prefect an instruction over the load of the poll. >> If poll is disarmed this perfected instruction can be fully executed. >> If that happens to contain an immediate oop, we don't know if it's the patched >> oop or not. >> >> /Robbin >> >>> >>> Cheers Richard. >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>> To: Doerr, Martin ; David Holmes >>> ; hotspot-dev at openjdk.java.net >>> Cc: Reingruber, Richard >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Martin, >>> >>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>> Hi Robbin, >>>> >>>>> But that is whole new subject and let's keep that separate from this :) >>>> Sure, we can keep the modification side discussion separate. This change is >>>> already difficult enough ?? >>>> >>>> >>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>> >>>> orderAccess_aix/linux_ppc.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>> >>>> orderAccess_linux_s390.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>> >>> >>> Thanks! >>> >>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops >>>>> encoded into the instruction stream. >>>>> I couldn't find a call to AbstractICache::invalidate_range. >>> >>> I think I miss-understood you here, the reason I heard for not doing >>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>> safepoint is 'long enough'. >>> >>> /Robbin >>> >>>> >>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>> And one call to AbstractICache/ICache::invalidate_word. >>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>> But that is whole new subject and let's keep that separate from this :) >>>> >>>> I'll create issues for some of these topics, which if nothing else should be >>>> discussed. >>>> >>>> /Robbin >>>> >>>>> >>>>> Does anybody know where it is or if it's missing? >>>>> >>>>> Best regards, >>>>> Martin >>>>> From richard.reingruber at sap.com Fri Mar 15 13:04:37 2019 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 15 Mar 2019 13:04:37 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: > OPTION 1 is for self-modifying code, not cross-modifying code. I know. I wanted to point out that it seems to work for cross-modifying code, too, in the sense that patchee java threads don't execute a serializing instruction before jumping to compiled code or before executing a compiled call another java thread just resolved. Cheers, Richard. -----Original Message----- From: David Holmes Sent: Freitag, 15. M?rz 2019 12:55 To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code On 15/03/2019 9:08 pm, Reingruber, Richard wrote: > Hi Robbin, > > I'm referring to x86 here. > > My comment/question about the requirement of an ICache invalidation in the > patcher thread was a little "undereducated". After reading recent manuals [1]: > > 1. I think your right that a serializing instruction after the safepoint poll > is necessary. > > 2. I doubt that cache flushing is required at all, according to [1]. At least > SPECjvm2008 runs with an AbstractICache::_flush_icache_stub that > immediately returns. > > In the sense of [1] JIT compilers and JIT runtimes are cross-modifying code, > too, when they compile bytecode and resolve compiled calls. But afaik the java > threads don't execute a serializing instruction before executing that code. So, > if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee threads jump to the > modified code is legal as well. OPTION 1 is for self-modifying code, not cross-modifying code. David ----- > So at least theoretically you should be able to replace the serializing > instruction with a call to an address you would load from the thread. The > address should lead to the runtime, where the thread would block if the > safepoint is still active or just return if it is not. > > The last point was rather exploring the space of legal solutions than a > suggestion to solve the issue. I'd favor your approach if you manage to do the > serialization conditionally (BTW: a mechanism similar to JavaThread::*deopt_suspend* > would be yet another way to do that). > > Cheers, Richard. > > [1] 8.1.3 Handling Self- and Cross-Modifying Code > in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 > (3A, 3B, 3C & 3D): System Programming Guide > https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf > on https://software.intel.com/en-us/articles/intel-sdm > > -----Original Message----- > From: Robbin Ehn > Sent: Donnerstag, 14. M?rz 2019 13:24 > To: Reingruber, Richard ; Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi Richard, > > On 3/14/19 11:12 AM, Reingruber, Richard wrote: >> Hi Robbin, >> >> > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >> > is that it was deemed that a safepoint is 'long enough'. >> >> This doesn't seem to be a very good reason. I did a little modification to your >> initial figure: > > That's why we are adding the instruction barrier! > >> >> JavaThread: | VMThread >> StoreLoad | >> | update >> | ? Global ICache Invalidate ? >> | disarm >> load thread_poll | >> >> Question would be: does an ICache flush happen after the immediate oop update? >> I would consider it problematic if not. Even if the safepoint was very long. > > The icache flush assumes it can stop speculation and flush instruction cache in > patchee thread. > Correct me if anything is wrong: > - sparc needs flush in all patchee threads. > - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. > - ppc needs isync in all patchee threads? > - s390 needs zarch_sync all in patchee threads? > - x86 needs cpuid in all in patchee threads. > > So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) > have no effect in patchee threads!? > > Adding a few words on x86 ICache::Invalidate/word. > We have TSO, so if the patchee thread can see poll is disarmed all other stores > are visible. I have seen no case ICache::Invalidate/word would make a difference > from a StoreStore/compiler barrier (still talking about platform code for x86) ? > The problem is if we prefect an instruction over the load of the poll. > If poll is disarmed this perfected instruction can be fully executed. > If that happens to contain an immediate oop, we don't know if it's the patched > oop or not. > > /Robbin > >> >> Cheers Richard. >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Donnerstag, 14. M?rz 2019 10:34 >> To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >> Cc: Reingruber, Richard >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Martin, >> >> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>> Hi Robbin, >>> >>>> But that is whole new subject and let's keep that separate from this :) >>> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >>> >>> >>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>> >>> orderAccess_aix/linux_ppc.hpp: >>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>> >>> orderAccess_linux_s390.hpp: >>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>> >> >> Thanks! >> >>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>>> I couldn't find a call to AbstractICache::invalidate_range. >> >> I think I miss-understood you here, the reason I heard for not doing >> invalidate_range/word in nmethod::oops_do is that it was deemed that a >> safepoint is 'long enough'. >> >> /Robbin >> >>> >>> Look for ICache::invalidate_range, I see 11 calls on x64. >>> And one call to AbstractICache/ICache::invalidate_word. >>> Btw I have spent some time looking at x86/icache_x86.cpp... >>> But that is whole new subject and let's keep that separate from this :) >>> >>> I'll create issues for some of these topics, which if nothing else should be >>> discussed. >>> >>> /Robbin >>> >>>> >>>> Does anybody know where it is or if it's missing? >>>> >>>> Best regards, >>>> Martin >>>> From coleen.phillimore at oracle.com Fri Mar 15 14:58:46 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 15 Mar 2019 10:58:46 -0400 Subject: RFR (S) 8220512: Deoptimize redefinition functions that have dirty ICs In-Reply-To: <38cdb82d-c51e-b906-c1e4-2211715fa7b6@oracle.com> References: <00317af6-2a5f-5b32-408b-3bd888682c19@oracle.com> <15470b26-c400-1bc5-947f-985ee53cb002@oracle.com> <38cdb82d-c51e-b906-c1e4-2211715fa7b6@oracle.com> Message-ID: <5682cb5a-4b23-4ea5-6b3b-1471a0bc6499@oracle.com> From some offline feedback, I changed the name of has_evol_ics => has_evol_metadata and a couple of other small things.? I reran this though builds and tier1 tests. Incremental: http://cr.openjdk.java.net/~coleenp/2019/8220512.02.incr/webrev/index.html Full: http://cr.openjdk.java.net/~coleenp/2019/8220512.02/webrev/index.html Thanks, Coleen On 3/14/19 3:09 PM, coleen.phillimore at oracle.com wrote: > > > On 3/14/19 2:24 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> It looks good to me. >> >> Just one minor suggestion: >> >> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev/src/hotspot/share/classfile/metadataOnStackMark.cpp.udiff.html >> >> - Threads::metadata_do(Metadata::mark_on_stack); >> - CodeCache::metadata_do(Metadata::mark_on_stack); >> + MetadataOnStackClosure mon_stack; >> + Threads::metadata_do(&mon_stack); >> + CodeCache::metadata_do(&mon_stack); >> ?The 'mon_stack' can be associated with monitors. >> ?How about to rename it to something like 'md_on_stack'? > > Okay, I'll change the name to md_on_stack. > > Thanks for the code review! > Coleen > >> >> Thanks, >> Serguei >> >> >> On 3/14/19 10:40, coleen.phillimore at oracle.com wrote: >>> Summary: Walk ICs to determine whether nmethods are dependent on >>> redefined classes. >>> >>> See bug for more details.? Tested with redefinition tests: >>> >>> #redefinition tests. >>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jvmti >&jvmti.out >>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jdi >&jdi.out >>> make test TEST=open/test/hotspot/jtreg/runtime/RedefineTests >>> >&redefine.out >>> make test TEST=open/test/jdk/java/lang/instrument >&instrument.out >>> make test TEST=open/test/jdk/com/sun/jdi >&jtreg.jdi.out >>> >>> hs-tier1-6 as well as java/lang/instrument tests with -Xcomp. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8220512 >>> >>> Thanks, >>> Coleen >>> >>> >> > From erik.osterlund at oracle.com Fri Mar 15 15:02:50 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 15 Mar 2019 16:02:50 +0100 Subject: RFR (S) 8220512: Deoptimize redefinition functions that have dirty ICs In-Reply-To: <5682cb5a-4b23-4ea5-6b3b-1471a0bc6499@oracle.com> References: <00317af6-2a5f-5b32-408b-3bd888682c19@oracle.com> <15470b26-c400-1bc5-947f-985ee53cb002@oracle.com> <38cdb82d-c51e-b906-c1e4-2211715fa7b6@oracle.com> <5682cb5a-4b23-4ea5-6b3b-1471a0bc6499@oracle.com> Message-ID: <7f7aa7e2-a6fa-49b7-c6f4-83bafbad3d6f@oracle.com> Hi Coleen, Looks good. Thanks for doing this! /Erik On 2019-03-15 15:58, coleen.phillimore at oracle.com wrote: > > From some offline feedback, I changed the name of has_evol_ics => > has_evol_metadata and a couple of other small things.? I reran this > though builds and tier1 tests. > > Incremental: > http://cr.openjdk.java.net/~coleenp/2019/8220512.02.incr/webrev/index.html > Full: > http://cr.openjdk.java.net/~coleenp/2019/8220512.02/webrev/index.html > > Thanks, > Coleen > > On 3/14/19 3:09 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 3/14/19 2:24 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> It looks good to me. >>> >>> Just one minor suggestion: >>> >>> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev/src/hotspot/share/classfile/metadataOnStackMark.cpp.udiff.html >>> >>> - Threads::metadata_do(Metadata::mark_on_stack); >>> - CodeCache::metadata_do(Metadata::mark_on_stack); >>> + MetadataOnStackClosure mon_stack; >>> + Threads::metadata_do(&mon_stack); >>> + CodeCache::metadata_do(&mon_stack); >>> ?The 'mon_stack' can be associated with monitors. >>> ?How about to rename it to something like 'md_on_stack'? >> >> Okay, I'll change the name to md_on_stack. >> >> Thanks for the code review! >> Coleen >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 3/14/19 10:40, coleen.phillimore at oracle.com wrote: >>>> Summary: Walk ICs to determine whether nmethods are dependent on >>>> redefined classes. >>>> >>>> See bug for more details.? Tested with redefinition tests: >>>> >>>> #redefinition tests. >>>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jvmti >>>> >&jvmti.out >>>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jdi >&jdi.out >>>> make test TEST=open/test/hotspot/jtreg/runtime/RedefineTests >>>> >&redefine.out >>>> make test TEST=open/test/jdk/java/lang/instrument >&instrument.out >>>> make test TEST=open/test/jdk/com/sun/jdi >&jtreg.jdi.out >>>> >>>> hs-tier1-6 as well as java/lang/instrument tests with -Xcomp. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8220512 >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Mar 15 16:04:20 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 15 Mar 2019 12:04:20 -0400 Subject: RFR (S) 8220512: Deoptimize redefinition functions that have dirty ICs In-Reply-To: <7f7aa7e2-a6fa-49b7-c6f4-83bafbad3d6f@oracle.com> References: <00317af6-2a5f-5b32-408b-3bd888682c19@oracle.com> <15470b26-c400-1bc5-947f-985ee53cb002@oracle.com> <38cdb82d-c51e-b906-c1e4-2211715fa7b6@oracle.com> <5682cb5a-4b23-4ea5-6b3b-1471a0bc6499@oracle.com> <7f7aa7e2-a6fa-49b7-c6f4-83bafbad3d6f@oracle.com> Message-ID: <6813b329-5d4a-8ce2-230c-3ceaad9fe935@oracle.com> Thank you Erik! Coleen On 3/15/19 11:02 AM, Erik ?sterlund wrote: > Hi Coleen, > > Looks good. Thanks for doing this! > > /Erik > > On 2019-03-15 15:58, coleen.phillimore at oracle.com wrote: >> >> From some offline feedback, I changed the name of has_evol_ics => >> has_evol_metadata and a couple of other small things.? I reran this >> though builds and tier1 tests. >> >> Incremental: >> http://cr.openjdk.java.net/~coleenp/2019/8220512.02.incr/webrev/index.html >> Full: >> http://cr.openjdk.java.net/~coleenp/2019/8220512.02/webrev/index.html >> >> Thanks, >> Coleen >> >> On 3/14/19 3:09 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 3/14/19 2:24 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Coleen, >>>> >>>> It looks good to me. >>>> >>>> Just one minor suggestion: >>>> >>>> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev/src/hotspot/share/classfile/metadataOnStackMark.cpp.udiff.html >>>> >>>> - Threads::metadata_do(Metadata::mark_on_stack); >>>> - CodeCache::metadata_do(Metadata::mark_on_stack); >>>> + MetadataOnStackClosure mon_stack; >>>> + Threads::metadata_do(&mon_stack); >>>> + CodeCache::metadata_do(&mon_stack); >>>> ?The 'mon_stack' can be associated with monitors. >>>> ?How about to rename it to something like 'md_on_stack'? >>> >>> Okay, I'll change the name to md_on_stack. >>> >>> Thanks for the code review! >>> Coleen >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 3/14/19 10:40, coleen.phillimore at oracle.com wrote: >>>>> Summary: Walk ICs to determine whether nmethods are dependent on >>>>> redefined classes. >>>>> >>>>> See bug for more details.? Tested with redefinition tests: >>>>> >>>>> #redefinition tests. >>>>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jvmti >>>>> >&jvmti.out >>>>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jdi >&jdi.out >>>>> make test TEST=open/test/hotspot/jtreg/runtime/RedefineTests >>>>> >&redefine.out >>>>> make test TEST=open/test/jdk/java/lang/instrument >&instrument.out >>>>> make test TEST=open/test/jdk/com/sun/jdi >&jtreg.jdi.out >>>>> >>>>> hs-tier1-6 as well as java/lang/instrument tests with -Xcomp. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8220512 >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>> >>> >> > From igor.ignatyev at oracle.com Fri Mar 15 16:23:32 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 15 Mar 2019 09:23:32 -0700 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list Message-ID: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 > 3 lines changed: 3 ins; 0 del; 0 mod; Hi all, could you please review the trivial patch which put ArraySize an InfiniteList tests into zgc problem list? webrev: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 JBS: https://bugs.openjdk.java.net/browse/JDK-8220727 Thanks, -- Igor From stefan.karlsson at oracle.com Fri Mar 15 16:29:45 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 15 Mar 2019 17:29:45 +0100 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: References: Message-ID: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> Hi Igor, I think this could easily be fixed by changing the heap size to -Xmx128m. There's no need to problem list these tests, IMHO. However, if you still want to problem list the tests, then this looks good. Thanks, StefanK On 2019-03-15 17:23, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >> 3 lines changed: 3 ins; 0 del; 0 mod; > > Hi all, > > could you please review the trivial patch which put ArraySize an InfiniteList tests into zgc problem list? > > webrev: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8220727 > > Thanks, > -- Igor > From igor.ignatyev at oracle.com Fri Mar 15 16:44:24 2019 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Fri, 15 Mar 2019 09:44:24 -0700 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> References: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> Message-ID: <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> Hi Stefan, Frankly I haven?t looked much into it, and decided to go w/ problem listing to quickly remove noise from test results. If you have a proper fix ready, please go ahead w/ it, I?ll withdraw this RFR. I won?t be able to come back to this till 1pm PT. ? Igor > On Mar 15, 2019, at 9:29 AM, Stefan Karlsson wrote: > > Hi Igor, > > I think this could easily be fixed by changing the heap size to -Xmx128m. There's no need to problem list these tests, IMHO. > > However, if you still want to problem list the tests, then this looks good. > > Thanks, > StefanK > >> On 2019-03-15 17:23, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>> 3 lines changed: 3 ins; 0 del; 0 mod; >> Hi all, >> could you please review the trivial patch which put ArraySize an InfiniteList tests into zgc problem list? >> webrev: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8220727 >> Thanks, >> -- Igor From stefan.karlsson at oracle.com Fri Mar 15 17:16:52 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 15 Mar 2019 18:16:52 +0100 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> References: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> Message-ID: <453da301-5325-aa57-27c5-4944a852f823@oracle.com> On 2019-03-15 17:44, Igor Ignatev wrote: > Hi Stefan, > > Frankly I haven?t looked much into it, and decided to go w/ problem listing to quickly remove noise from test results. If you have a proper fix ready, please go ahead w/ it, I?ll withdraw this RFR. I'm on my way home, and the proper fix is what I suggested below. I'll leave it up to you to decide how to deal with this change. StefanK > > I won?t be able to come back to this till 1pm PT. > > ? Igor > >> On Mar 15, 2019, at 9:29 AM, Stefan Karlsson wrote: >> >> Hi Igor, >> >> I think this could easily be fixed by changing the heap size to -Xmx128m. There's no need to problem list these tests, IMHO. >> >> However, if you still want to problem list the tests, then this looks good. >> >> Thanks, >> StefanK >> >>> On 2019-03-15 17:23, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>>> 3 lines changed: 3 ins; 0 del; 0 mod; >>> Hi all, >>> could you please review the trivial patch which put ArraySize an InfiniteList tests into zgc problem list? >>> webrev: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220727 >>> Thanks, >>> -- Igor > From jesper.wilhelmsson at oracle.com Fri Mar 15 17:38:48 2019 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Fri, 15 Mar 2019 18:38:48 +0100 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: <453da301-5325-aa57-27c5-4944a852f823@oracle.com> References: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> <453da301-5325-aa57-27c5-4944a852f823@oracle.com> Message-ID: <86CC8BD2-FC44-4D9A-A90D-BA1E0020A181@oracle.com> I f this is the correct fix I can push it immediately. Please review asap. diff --git a/test/hotspot/jtreg/gc/ArraySize.java b/test/hotspot/jtreg/gc/ArraySize.java --- a/test/hotspot/jtreg/gc/ArraySize.java +++ b/test/hotspot/jtreg/gc/ArraySize.java @@ -25,7 +25,7 @@ * @test * @bug 4063078 * @summary Allocating a ridiculously large array should not crash the VM - * @run main/othervm -Xmx32m -Xms32m gc.ArraySize + * @run main/othervm -Xmx128m -Xms32m gc.ArraySize */ package gc; diff --git a/test/hotspot/jtreg/gc/InfiniteList.java b/test/hotspot/jtreg/gc/InfiniteList.java --- a/test/hotspot/jtreg/gc/InfiniteList.java +++ b/test/hotspot/jtreg/gc/InfiniteList.java @@ -26,7 +26,7 @@ * @bug 4098578 * @summary Check if the VM properly throws OutOfMemoryError * @author Sheng Liang - * @run main/othervm -Xmx25M gc.InfiniteList + * @run main/othervm -Xmx128m gc.InfiniteList */ package gc; Thanks, /Jesper > On 15 Mar 2019, at 18:16, Stefan Karlsson wrote: > > On 2019-03-15 17:44, Igor Ignatev wrote: >> Hi Stefan, >> Frankly I haven?t looked much into it, and decided to go w/ problem listing to quickly remove noise from test results. If you have a proper fix ready, please go ahead w/ it, I?ll withdraw this RFR. > > I'm on my way home, and the proper fix is what I suggested below. I'll leave it up to you to decide how to deal with this change. > > StefanK > >> I won?t be able to come back to this till 1pm PT. >> ? Igor >>> On Mar 15, 2019, at 9:29 AM, Stefan Karlsson wrote: >>> >>> Hi Igor, >>> >>> I think this could easily be fixed by changing the heap size to -Xmx128m. There's no need to problem list these tests, IMHO. >>> >>> However, if you still want to problem list the tests, then this looks good. >>> >>> Thanks, >>> StefanK >>> >>>> On 2019-03-15 17:23, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>>>> 3 lines changed: 3 ins; 0 del; 0 mod; >>>> Hi all, >>>> could you please review the trivial patch which put ArraySize an InfiniteList tests into zgc problem list? >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220727 >>>> Thanks, >>>> -- Igor From igor.ignatyev at oracle.com Fri Mar 15 17:45:50 2019 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Fri, 15 Mar 2019 10:45:50 -0700 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: <86CC8BD2-FC44-4D9A-A90D-BA1E0020A181@oracle.com> References: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> <453da301-5325-aa57-27c5-4944a852f823@oracle.com> <86CC8BD2-FC44-4D9A-A90D-BA1E0020A181@oracle.com> Message-ID: If the tests work w/ zGC now, then the fix looks good to me. ? Igor > On Mar 15, 2019, at 10:38 AM, jesper.wilhelmsson at oracle.com wrote: > > I f this is the correct fix I can push it immediately. Please review asap. > > > diff --git a/test/hotspot/jtreg/gc/ArraySize.java b/test/hotspot/jtreg/gc/ArraySize.java > --- a/test/hotspot/jtreg/gc/ArraySize.java > +++ b/test/hotspot/jtreg/gc/ArraySize.java > @@ -25,7 +25,7 @@ > * @test > * @bug 4063078 > * @summary Allocating a ridiculously large array should not crash the VM > - * @run main/othervm -Xmx32m -Xms32m gc.ArraySize > + * @run main/othervm -Xmx128m -Xms32m gc.ArraySize > */ > > package gc; > diff --git a/test/hotspot/jtreg/gc/InfiniteList.java b/test/hotspot/jtreg/gc/InfiniteList.java > --- a/test/hotspot/jtreg/gc/InfiniteList.java > +++ b/test/hotspot/jtreg/gc/InfiniteList.java > @@ -26,7 +26,7 @@ > * @bug 4098578 > * @summary Check if the VM properly throws OutOfMemoryError > * @author Sheng Liang > - * @run main/othervm -Xmx25M gc.InfiniteList > + * @run main/othervm -Xmx128m gc.InfiniteList > */ > > package gc; > > > Thanks, > /Jesper > > >> On 15 Mar 2019, at 18:16, Stefan Karlsson wrote: >> >> On 2019-03-15 17:44, Igor Ignatev wrote: >>> Hi Stefan, >>> Frankly I haven?t looked much into it, and decided to go w/ problem listing to quickly remove noise from test results. If you have a proper fix ready, please go ahead w/ it, I?ll withdraw this RFR. >> >> I'm on my way home, and the proper fix is what I suggested below. I'll leave it up to you to decide how to deal with this change. >> >> StefanK >> >>> I won?t be able to come back to this till 1pm PT. >>> ? Igor >>>> On Mar 15, 2019, at 9:29 AM, Stefan Karlsson wrote: >>>> >>>> Hi Igor, >>>> >>>> I think this could easily be fixed by changing the heap size to -Xmx128m. There's no need to problem list these tests, IMHO. >>>> >>>> However, if you still want to problem list the tests, then this looks good. >>>> >>>> Thanks, >>>> StefanK >>>> >>>>>> On 2019-03-15 17:23, Igor Ignatyev wrote: >>>>>> http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>>>>> 3 lines changed: 3 ins; 0 del; 0 mod; >>>>> Hi all, >>>>> could you please review the trivial patch which put ArraySize an InfiniteList tests into zgc problem list? >>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220727 >>>>> Thanks, >>>>> -- Igor > From thomas.schatzl at oracle.com Fri Mar 15 17:47:31 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 15 Mar 2019 18:47:31 +0100 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: <86CC8BD2-FC44-4D9A-A90D-BA1E0020A181@oracle.com> References: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> <453da301-5325-aa57-27c5-4944a852f823@oracle.com> <86CC8BD2-FC44-4D9A-A90D-BA1E0020A181@oracle.com> Message-ID: <3f0109a8a27690ec73331cdb6df97ff01a8de06e.camel@oracle.com> Hi, On Fri, 2019-03-15 at 18:38 +0100, jesper.wilhelmsson at oracle.com wrote: > I f this is the correct fix I can push it immediately. Please review > asap. > > > diff --git a/test/hotspot/jtreg/gc/ArraySize.java > b/test/hotspot/jtreg/gc/ArraySize.java > --- a/test/hotspot/jtreg/gc/ArraySize.java > +++ b/test/hotspot/jtreg/gc/ArraySize.java > @@ -25,7 +25,7 @@ > * @test > * @bug 4063078 > * @summary Allocating a ridiculously large array should not crash > the VM > - * @run main/othervm -Xmx32m -Xms32m gc.ArraySize > + * @run main/othervm -Xmx128m -Xms32m gc.ArraySize > */ > > package gc; > diff --git a/test/hotspot/jtreg/gc/InfiniteList.java > b/test/hotspot/jtreg/gc/InfiniteList.java > --- a/test/hotspot/jtreg/gc/InfiniteList.java > +++ b/test/hotspot/jtreg/gc/InfiniteList.java > @@ -26,7 +26,7 @@ > * @bug 4098578 > * @summary Check if the VM properly throws OutOfMemoryError > * @author Sheng Liang > - * @run main/othervm -Xmx25M gc.InfiniteList > + * @run main/othervm -Xmx128m gc.InfiniteList > */ > the change is good, do not know if it suffices. Thanks, Thomas From jesper.wilhelmsson at oracle.com Fri Mar 15 17:56:48 2019 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Fri, 15 Mar 2019 18:56:48 +0100 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: References: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> <453da301-5325-aa57-27c5-4944a852f823@oracle.com> <86CC8BD2-FC44-4D9A-A90D-BA1E0020A181@oracle.com> Message-ID: <89B02936-DB21-41B7-8D26-16619E00FCD8@oracle.com> Both tests passes with this change. Thanks for the review! /Jesper > On 15 Mar 2019, at 18:45, Igor Ignatev wrote: > > If the tests work w/ zGC now, then the fix looks good to me. > > ? Igor > >> On Mar 15, 2019, at 10:38 AM, jesper.wilhelmsson at oracle.com wrote: >> >> I f this is the correct fix I can push it immediately. Please review asap. >> >> >> diff --git a/test/hotspot/jtreg/gc/ArraySize.java b/test/hotspot/jtreg/gc/ArraySize.java >> --- a/test/hotspot/jtreg/gc/ArraySize.java >> +++ b/test/hotspot/jtreg/gc/ArraySize.java >> @@ -25,7 +25,7 @@ >> * @test >> * @bug 4063078 >> * @summary Allocating a ridiculously large array should not crash the VM >> - * @run main/othervm -Xmx32m -Xms32m gc.ArraySize >> + * @run main/othervm -Xmx128m -Xms32m gc.ArraySize >> */ >> >> package gc; >> diff --git a/test/hotspot/jtreg/gc/InfiniteList.java b/test/hotspot/jtreg/gc/InfiniteList.java >> --- a/test/hotspot/jtreg/gc/InfiniteList.java >> +++ b/test/hotspot/jtreg/gc/InfiniteList.java >> @@ -26,7 +26,7 @@ >> * @bug 4098578 >> * @summary Check if the VM properly throws OutOfMemoryError >> * @author Sheng Liang >> - * @run main/othervm -Xmx25M gc.InfiniteList >> + * @run main/othervm -Xmx128m gc.InfiniteList >> */ >> >> package gc; >> >> >> Thanks, >> /Jesper >> >> >>> On 15 Mar 2019, at 18:16, Stefan Karlsson wrote: >>> >>> On 2019-03-15 17:44, Igor Ignatev wrote: >>>> Hi Stefan, >>>> Frankly I haven?t looked much into it, and decided to go w/ problem listing to quickly remove noise from test results. If you have a proper fix ready, please go ahead w/ it, I?ll withdraw this RFR. >>> >>> I'm on my way home, and the proper fix is what I suggested below. I'll leave it up to you to decide how to deal with this change. >>> >>> StefanK >>> >>>> I won?t be able to come back to this till 1pm PT. >>>> ? Igor >>>>> On Mar 15, 2019, at 9:29 AM, Stefan Karlsson wrote: >>>>> >>>>> Hi Igor, >>>>> >>>>> I think this could easily be fixed by changing the heap size to -Xmx128m. There's no need to problem list these tests, IMHO. >>>>> >>>>> However, if you still want to problem list the tests, then this looks good. >>>>> >>>>> Thanks, >>>>> StefanK >>>>> >>>>>>> On 2019-03-15 17:23, Igor Ignatyev wrote: >>>>>>> http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>>>>>> 3 lines changed: 3 ins; 0 del; 0 mod; >>>>>> Hi all, >>>>>> could you please review the trivial patch which put ArraySize an InfiniteList tests into zgc problem list? >>>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8220727/webrev.00 >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220727 >>>>>> Thanks, >>>>>> -- Igor >> > From jesper.wilhelmsson at oracle.com Fri Mar 15 17:57:48 2019 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Fri, 15 Mar 2019 18:57:48 +0100 Subject: RFR(T) : 8220727 : put ArraySize an InfiniteList into zgc problem list In-Reply-To: <3f0109a8a27690ec73331cdb6df97ff01a8de06e.camel@oracle.com> References: <092bc14c-5785-d375-63a2-f62f42e12e46@oracle.com> <9F445FCC-8165-43E8-90D1-9DA2BE09FBD7@oracle.com> <453da301-5325-aa57-27c5-4944a852f823@oracle.com> <86CC8BD2-FC44-4D9A-A90D-BA1E0020A181@oracle.com> <3f0109a8a27690ec73331cdb6df97ff01a8de06e.camel@oracle.com> Message-ID: <855FE948-987B-4C3A-8056-FC64A03FFD65@oracle.com> Thank you Thomas! We'll see soon if it's enough ;-) It passed when I tried it at least. /Jesper > On 15 Mar 2019, at 18:47, Thomas Schatzl wrote: > > Hi, > > On Fri, 2019-03-15 at 18:38 +0100, jesper.wilhelmsson at oracle.com wrote: >> I f this is the correct fix I can push it immediately. Please review >> asap. >> >> >> diff --git a/test/hotspot/jtreg/gc/ArraySize.java >> b/test/hotspot/jtreg/gc/ArraySize.java >> --- a/test/hotspot/jtreg/gc/ArraySize.java >> +++ b/test/hotspot/jtreg/gc/ArraySize.java >> @@ -25,7 +25,7 @@ >> * @test >> * @bug 4063078 >> * @summary Allocating a ridiculously large array should not crash >> the VM >> - * @run main/othervm -Xmx32m -Xms32m gc.ArraySize >> + * @run main/othervm -Xmx128m -Xms32m gc.ArraySize >> */ >> >> package gc; >> diff --git a/test/hotspot/jtreg/gc/InfiniteList.java >> b/test/hotspot/jtreg/gc/InfiniteList.java >> --- a/test/hotspot/jtreg/gc/InfiniteList.java >> +++ b/test/hotspot/jtreg/gc/InfiniteList.java >> @@ -26,7 +26,7 @@ >> * @bug 4098578 >> * @summary Check if the VM properly throws OutOfMemoryError >> * @author Sheng Liang >> - * @run main/othervm -Xmx25M gc.InfiniteList >> + * @run main/othervm -Xmx128m gc.InfiniteList >> */ >> > > the change is good, do not know if it suffices. > > Thanks, > Thomas From kim.barrett at oracle.com Fri Mar 15 22:18:11 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 15 Mar 2019 18:18:11 -0400 Subject: RFR(XS): 8220690: ATTRIBUTE_ALIGNED requires GNU extensions enabled Message-ID: <2879CC12-CC8C-4ECA-9D7C-0D60F8AC8263@oracle.com> Please review this change to the GCC-version of the ATTRIBUTE_ALIGNED macro, to not require enabling GNU extensions. Instead of using a typeof form we use __typeof__ instead. See the CR for more details. CR: https://bugs.openjdk.java.net/browse/JDK-8220690 Webrev: http://cr.openjdk.java.net/~kbarrett/8220690/open.00/ Testing: mach5 tier1 From kim.barrett at oracle.com Fri Mar 15 23:03:33 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 15 Mar 2019 19:03:33 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support Message-ID: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> Please review this fix for intermittent Mac-only crashes involving the OWSTTaskTerminator. This is joint work with Patricio Chilano Mateo. This turned out to be a bug in the macOSX kernel's pthread support (which may be fixed in OSX 10.14 (Mojave)). When using a condvar, an associated broadcast entry may be added to a kernel table and left there after the last use of the condvar. If we then destroy and delete the condvar and later happen to allocate and init a mutex at the same address (this really does happen), that stale condvar entry in the kernel table confuses operations on the mutex, causing crashes and hangs. We have been able to reproduce those failure modes with standalone programs (no JDK code involved) (see the CR for reproducers), and have reported the problem to Apple. This reproducer didn't fail when tested on Mojave, but did fail on all previous OS versions we tried. The OWSTTaskTerminator is subject to this problem because a new HotSpot Monitor object is allocated for each terminator. Terminators are allocated for various parallel tasks during garbage collection, so there are quite a few Monitor objects (and their contained PlatformMonitors) being created and deleted, so many opportunities to get the kernel into the bad state and then later reuse a previous condvar address for a mutex. That was enough for the stress test (gc/stress/TestStressIHOPMultiThread.java) to hit the problem occasionally. This problem would not have shown up before JDK-8210832, which was made shortly before the first sighting of the stress test failure. Before that change we re-used park events in the implementation of HotSpot Mutex/Monitor. Because of that re-use, there wasn't an opportunity to allocate a pthread_mutex_t at the same address as a former pthread_cond_t. We work around this problem by allocating from a freelist the mutex/condvar pairs needed by the macOSX PlatformMonitor. An alternative workaround that was explored is to (for macOSX only) add a short timedwait on the condvar when deleting a PlatformMonitor. The idea is that (1) if there is a lingering kernel table entry for the condvar, the timedwait will eat it, and (2) if there isn't such an entry the timedwait will (almost) immediately expire. This would use pthread_cond_timedwait_relative_np to avoid unneeded clock accesses. This approach isn't being taken because it might be sensitive to implementation details that could vary between OS versions. CR: https://bugs.openjdk.java.net/browse/JDK-8218975 Webrev: http://cr.openjdk.java.net/~kbarrett/8218975/open.01/ Testing: mach5 tier1-5 mach5 the specific test, 2000 times with no failures. Without this change, typical failure rate seems to be on the order of 0.5-1.0%. Performance testing on Mac found no regressions. From patricio.chilano.mateo at oracle.com Sat Mar 16 02:43:12 2019 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 15 Mar 2019 22:43:12 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> Message-ID: <39689327-717e-b063-5b9a-28f0f632dbeb@oracle.com> Hi Kim, Change looks good to me! Thanks, Patricio On 3/15/19 7:03 PM, Kim Barrett wrote: > Please review this fix for intermittent Mac-only crashes involving the > OWSTTaskTerminator. > > This is joint work with Patricio Chilano Mateo. > > This turned out to be a bug in the macOSX kernel's pthread support > (which may be fixed in OSX 10.14 (Mojave)). When using a condvar, an > associated broadcast entry may be added to a kernel table and left > there after the last use of the condvar. If we then destroy and delete > the condvar and later happen to allocate and init a mutex at the same > address (this really does happen), that stale condvar entry in the > kernel table confuses operations on the mutex, causing crashes and > hangs. > > We have been able to reproduce those failure modes with standalone > programs (no JDK code involved) (see the CR for reproducers), and have > reported the problem to Apple. This reproducer didn't fail when > tested on Mojave, but did fail on all previous OS versions we tried. > > The OWSTTaskTerminator is subject to this problem because a new > HotSpot Monitor object is allocated for each terminator. Terminators > are allocated for various parallel tasks during garbage collection, so > there are quite a few Monitor objects (and their contained > PlatformMonitors) being created and deleted, so many opportunities to > get the kernel into the bad state and then later reuse a previous > condvar address for a mutex. That was enough for the stress test > (gc/stress/TestStressIHOPMultiThread.java) to hit the problem > occasionally. > > This problem would not have shown up before JDK-8210832, which was > made shortly before the first sighting of the stress test failure. > Before that change we re-used park events in the implementation of > HotSpot Mutex/Monitor. Because of that re-use, there wasn't an > opportunity to allocate a pthread_mutex_t at the same address as a > former pthread_cond_t. > > We work around this problem by allocating from a freelist the > mutex/condvar pairs needed by the macOSX PlatformMonitor. > > An alternative workaround that was explored is to (for macOSX only) > add a short timedwait on the condvar when deleting a PlatformMonitor. > The idea is that (1) if there is a lingering kernel table entry for the > condvar, the timedwait will eat it, and (2) if there isn't such an > entry the timedwait will (almost) immediately expire. This would use > pthread_cond_timedwait_relative_np to avoid unneeded clock accesses. > This approach isn't being taken because it might be sensitive to > implementation details that could vary between OS versions. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8218975 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8218975/open.01/ > > Testing: > mach5 tier1-5 > mach5 the specific test, 2000 times with no failures. Without this > change, typical failure rate seems to be on the order of 0.5-1.0%. > Performance testing on Mac found no regressions. > From thomas.schatzl at oracle.com Sat Mar 16 12:35:37 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Sat, 16 Mar 2019 13:35:37 +0100 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> Message-ID: Hi, On Fri, 2019-03-15 at 19:03 -0400, Kim Barrett wrote: > Please review this fix for intermittent Mac-only crashes involving > the OWSTTaskTerminator. > > This is joint work with Patricio Chilano Mateo. > [...] Thanks for the hard work on this! > CR: > https://bugs.openjdk.java.net/browse/JDK-8218975 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8218975/open.01/ > > Testing: > mach5 tier1-5 > mach5 the specific test, 2000 times with no failures. Without this > change, typical failure rate seems to be on the order of 0.5-1.0%. > Performance testing on Mac found no regressions. > looks good, with one note: Afaik the getters for particular member variable are (at least in gc code) typically not postfixed at all, or at most with "_addr", and definitely not with "_ptr". The "_ptr" suffix just feels odd as it is really never used elsewhere in similar cases (in gc code only?). I let you decide whether it is appropriate to do any renaming. There is no need for a re-review from me for either decision. If this were GC code I would probably be a bit more insistent on not having a "_ptr" suffix. Thanks, Thomas From david.holmes at oracle.com Mon Mar 18 01:43:06 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Mar 2019 11:43:06 +1000 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> Message-ID: <5ff8d1fc-8a3f-c270-6766-b38c40c1645a@oracle.com> Hi Kim and Patricio On 16/03/2019 9:03 am, Kim Barrett wrote: > Please review this fix for intermittent Mac-only crashes involving the > OWSTTaskTerminator. > > This is joint work with Patricio Chilano Mateo. Fantastic job in pinning this one down! > This turned out to be a bug in the macOSX kernel's pthread support > (which may be fixed in OSX 10.14 (Mojave)). When using a condvar, an > associated broadcast entry may be added to a kernel table and left > there after the last use of the condvar. If we then destroy and delete > the condvar and later happen to allocate and init a mutex at the same > address (this really does happen), that stale condvar entry in the > kernel table confuses operations on the mutex, causing crashes and > hangs. > > We have been able to reproduce those failure modes with standalone > programs (no JDK code involved) (see the CR for reproducers), and have > reported the problem to Apple. This reproducer didn't fail when > tested on Mojave, but did fail on all previous OS versions we tried. > > The OWSTTaskTerminator is subject to this problem because a new > HotSpot Monitor object is allocated for each terminator. Terminators > are allocated for various parallel tasks during garbage collection, so > there are quite a few Monitor objects (and their contained > PlatformMonitors) being created and deleted, so many opportunities to > get the kernel into the bad state and then later reuse a previous > condvar address for a mutex. That was enough for the stress test > (gc/stress/TestStressIHOPMultiThread.java) to hit the problem > occasionally. > > This problem would not have shown up before JDK-8210832, which was > made shortly before the first sighting of the stress test failure. > Before that change we re-used park events in the implementation of > HotSpot Mutex/Monitor. Because of that re-use, there wasn't an > opportunity to allocate a pthread_mutex_t at the same address as a > former pthread_cond_t. > > We work around this problem by allocating from a freelist the > mutex/condvar pairs needed by the macOSX PlatformMonitor. The fix generally looks good though I agree with Thomas that X_ptr() should just be X(). There is one minor issue: + pthread_mutex_t os::PlatformMonitor::_freelist_lock = PTHREAD_MUTEX_INITIALIZER; on FreeBSD the default mutex initialization doesn't use PTHREAD_MUTEX_NORMAL and so we get a slow debug version IIRC. That's why we explicitly use non-static pthread_mutex initialization using a pthread_mutex_attr. You should be able to move the _freelist_lock initialization to pthread_init_common() with the PLATFORM_MONITOR_IMPL_INDIRECT guard. Thanks, David ----- > An alternative workaround that was explored is to (for macOSX only) > add a short timedwait on the condvar when deleting a PlatformMonitor. > The idea is that (1) if there is a lingering kernel table entry for the > condvar, the timedwait will eat it, and (2) if there isn't such an > entry the timedwait will (almost) immediately expire. This would use > pthread_cond_timedwait_relative_np to avoid unneeded clock accesses. > This approach isn't being taken because it might be sensitive to > implementation details that could vary between OS versions. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8218975 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8218975/open.01/ > > Testing: > mach5 tier1-5 > mach5 the specific test, 2000 times with no failures. Without this > change, typical failure rate seems to be on the order of 0.5-1.0%. > Performance testing on Mac found no regressions. > From david.holmes at oracle.com Mon Mar 18 02:06:32 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Mar 2019 12:06:32 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> Message-ID: <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> On 15/03/2019 11:04 pm, Reingruber, Richard wrote: > > OPTION 1 is for self-modifying code, not cross-modifying code. > > I know. I wanted to point out that it seems to work for cross-modifying code, > too, in the sense that patchee java threads don't execute a serializing instruction > before jumping to compiled code or before executing a compiled call another java > thread just resolved. That's the "accidental success" that is being avoided by this fix. Today it all seems to work just fine because the "distance" between the actions of the patching thread and the patchee thread that wants to use the new code is "vast". Cheers, David > Cheers, Richard. > > -----Original Message----- > From: David Holmes > Sent: Freitag, 15. M?rz 2019 12:55 > To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > On 15/03/2019 9:08 pm, Reingruber, Richard wrote: >> Hi Robbin, >> >> I'm referring to x86 here. >> >> My comment/question about the requirement of an ICache invalidation in the >> patcher thread was a little "undereducated". After reading recent manuals [1]: >> >> 1. I think your right that a serializing instruction after the safepoint poll >> is necessary. >> >> 2. I doubt that cache flushing is required at all, according to [1]. At least >> SPECjvm2008 runs with an AbstractICache::_flush_icache_stub that >> immediately returns. >> >> In the sense of [1] JIT compilers and JIT runtimes are cross-modifying code, >> too, when they compile bytecode and resolve compiled calls. But afaik the java >> threads don't execute a serializing instruction before executing that code. So, >> if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee threads jump to the >> modified code is legal as well. > > OPTION 1 is for self-modifying code, not cross-modifying code. > > David > ----- > >> So at least theoretically you should be able to replace the serializing >> instruction with a call to an address you would load from the thread. The >> address should lead to the runtime, where the thread would block if the >> safepoint is still active or just return if it is not. >> >> The last point was rather exploring the space of legal solutions than a >> suggestion to solve the issue. I'd favor your approach if you manage to do the >> serialization conditionally (BTW: a mechanism similar to JavaThread::*deopt_suspend* >> would be yet another way to do that). >> >> Cheers, Richard. >> >> [1] 8.1.3 Handling Self- and Cross-Modifying Code >> in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >> (3A, 3B, 3C & 3D): System Programming Guide >> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >> on https://software.intel.com/en-us/articles/intel-sdm >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Donnerstag, 14. M?rz 2019 13:24 >> To: Reingruber, Richard ; Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Richard, >> >> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>> Hi Robbin, >>> >>> > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>> > is that it was deemed that a safepoint is 'long enough'. >>> >>> This doesn't seem to be a very good reason. I did a little modification to your >>> initial figure: >> >> That's why we are adding the instruction barrier! >> >>> >>> JavaThread: | VMThread >>> StoreLoad | >>> | update >>> | ? Global ICache Invalidate ? >>> | disarm >>> load thread_poll | >>> >>> Question would be: does an ICache flush happen after the immediate oop update? >>> I would consider it problematic if not. Even if the safepoint was very long. >> >> The icache flush assumes it can stop speculation and flush instruction cache in >> patchee thread. >> Correct me if anything is wrong: >> - sparc needs flush in all patchee threads. >> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >> - ppc needs isync in all patchee threads? >> - s390 needs zarch_sync all in patchee threads? >> - x86 needs cpuid in all in patchee threads. >> >> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >> have no effect in patchee threads!? >> >> Adding a few words on x86 ICache::Invalidate/word. >> We have TSO, so if the patchee thread can see poll is disarmed all other stores >> are visible. I have seen no case ICache::Invalidate/word would make a difference >> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >> The problem is if we prefect an instruction over the load of the poll. >> If poll is disarmed this perfected instruction can be fully executed. >> If that happens to contain an immediate oop, we don't know if it's the patched >> oop or not. >> >> /Robbin >> >>> >>> Cheers Richard. >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>> To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >>> Cc: Reingruber, Richard >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Martin, >>> >>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>> Hi Robbin, >>>> >>>>> But that is whole new subject and let's keep that separate from this :) >>>> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >>>> >>>> >>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>> >>>> orderAccess_aix/linux_ppc.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>> >>>> orderAccess_linux_s390.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>> >>> >>> Thanks! >>> >>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>>>> I couldn't find a call to AbstractICache::invalidate_range. >>> >>> I think I miss-understood you here, the reason I heard for not doing >>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>> safepoint is 'long enough'. >>> >>> /Robbin >>> >>>> >>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>> And one call to AbstractICache/ICache::invalidate_word. >>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>> But that is whole new subject and let's keep that separate from this :) >>>> >>>> I'll create issues for some of these topics, which if nothing else should be >>>> discussed. >>>> >>>> /Robbin >>>> >>>>> >>>>> Does anybody know where it is or if it's missing? >>>>> >>>>> Best regards, >>>>> Martin >>>>> From sgehwolf at redhat.com Mon Mar 18 10:06:02 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 18 Mar 2019 11:06:02 +0100 Subject: [PING] RFR: 8220579: [Containers] SubSystem.java out of sync with osContainer_linux.cpp In-Reply-To: References: <8e1a66f748861591316e7411d4829d4096327a6d.camel@redhat.com> Message-ID: Hi, Could I get a review from an OpenJDK Reviewer for this, please? Bob is already OK with it. Thanks, Severin On Thu, 2019-03-14 at 13:58 -0400, Bob Vandette wrote: > The change looks good. Thanks for fixing this. > > I?d send this to core-libs (cc?d). > > Bob. > > > > On Mar 14, 2019, at 12:51 PM, Severin Gehwolf wrote: > > > > Hi, > > > > I'm not sure what the right list for Metrics.java[1] is. Assuming it's > > serviceability-dev: > > > > Please review this one-liner for for SubSystem.java which currently > > behaves differently from the native implementation in > > osContainer_linux.cpp. Please see the details in the bug. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8220579 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8220579/01/webrev/ > > > > Testing: > > Manual testing of JDK-8217338 with Metrics.java support with/without > > this fix on Linux x86_64. Metrics tests and Docker tests continue to > > pass for fastdebug jvms (NOT for release jvms. see JDK-8220674, which > > was fun). > > > > Thoughts? > > > > Thanks, > > Severin > > > > [1] http://hg.openjdk.java.net/jdk/jdk/file/641768acb12e/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/Metrics.java > > From adinn at redhat.com Mon Mar 18 10:14:56 2019 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 18 Mar 2019 10:14:56 +0000 Subject: [PING] RFR: 8220579: [Containers] SubSystem.java out of sync with osContainer_linux.cpp In-Reply-To: References: <8e1a66f748861591316e7411d4829d4096327a6d.camel@redhat.com> Message-ID: On 18/03/2019 10:06, Severin Gehwolf wrote: > Could I get a review from an OpenJDK Reviewer for this, please? Bob is > already OK with it. Yes, this is fine. Reviewed. regards, Andrew Dinn ----------- From richard.reingruber at sap.com Mon Mar 18 10:37:04 2019 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 18 Mar 2019 10:37:04 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> Message-ID: <29eea53113104c9eaf4325c56d70dc11@sap.com> > That's the "accidental success" that is being avoided by this fix I don't think so, but maybe I'm missing something. So if you look at this example: static void methodA() { // ... // <- J1 executes this methodB(); // <- J2 executes this // .. } Where * methodA is jit compiled with an unresolved call to methodB * Java thread J1 executes that compiled version of methodA up to just before the unresolved call to methodB * JIT compiles methodB and registers the code. * Java thread J2 executes the unresolved call which takes it to SharedRuntime::resolve_static_call_C() * J2 finds the target methodB and the compiled code for it with entry point E. It patches the call in the compiled code for methodA setting E as its target. This is done without safepoint. (see SharedRuntime::resolve_sub_helper_internal()) * Now to the interesting part: J1 executes the patched call. Until here I see 2 crossmodifications: (1) jit compilation of methodB: JIT is the patcher thread, J1 and J2 are the patchees (2) patching the call: J2 is the patcher (this is actually also a self-modification) and J1 is the patchee. Let's assume J2's modification makes it into J1's instruction stream. How is it guaranteed that the new compiled code for methodB from the JIT makes it into J1's instruction stream, too, if no serializing instruction is executed by J1? My explanation was that the call instruction has the serializing effect, just like in OPTION 1. Note as well the difference between OPTION 1 and 2: if the new code is not reached by a jump then a serializing instruction is required as well. > That's the "accidental success" that is being avoided by this fix Which part of the fix would that be? Thanks, Richard. -----Original Message----- From: David Holmes Sent: Montag, 18. M?rz 2019 03:07 To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code On 15/03/2019 11:04 pm, Reingruber, Richard wrote: > > OPTION 1 is for self-modifying code, not cross-modifying code. > > I know. I wanted to point out that it seems to work for cross-modifying code, > too, in the sense that patchee java threads don't execute a serializing instruction > before jumping to compiled code or before executing a compiled call another java > thread just resolved. That's the "accidental success" that is being avoided by this fix. Today it all seems to work just fine because the "distance" between the actions of the patching thread and the patchee thread that wants to use the new code is "vast". Cheers, David > Cheers, Richard. > > -----Original Message----- > From: David Holmes > Sent: Freitag, 15. M?rz 2019 12:55 > To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > On 15/03/2019 9:08 pm, Reingruber, Richard wrote: >> Hi Robbin, >> >> I'm referring to x86 here. >> >> My comment/question about the requirement of an ICache invalidation in the >> patcher thread was a little "undereducated". After reading recent manuals [1]: >> >> 1. I think your right that a serializing instruction after the safepoint poll >> is necessary. >> >> 2. I doubt that cache flushing is required at all, according to [1]. At least >> SPECjvm2008 runs with an AbstractICache::_flush_icache_stub that >> immediately returns. >> >> In the sense of [1] JIT compilers and JIT runtimes are cross-modifying code, >> too, when they compile bytecode and resolve compiled calls. But afaik the java >> threads don't execute a serializing instruction before executing that code. So, >> if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee threads jump to the >> modified code is legal as well. > > OPTION 1 is for self-modifying code, not cross-modifying code. > > David > ----- > >> So at least theoretically you should be able to replace the serializing >> instruction with a call to an address you would load from the thread. The >> address should lead to the runtime, where the thread would block if the >> safepoint is still active or just return if it is not. >> >> The last point was rather exploring the space of legal solutions than a >> suggestion to solve the issue. I'd favor your approach if you manage to do the >> serialization conditionally (BTW: a mechanism similar to JavaThread::*deopt_suspend* >> would be yet another way to do that). >> >> Cheers, Richard. >> >> [1] 8.1.3 Handling Self- and Cross-Modifying Code >> in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >> (3A, 3B, 3C & 3D): System Programming Guide >> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >> on https://software.intel.com/en-us/articles/intel-sdm >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Donnerstag, 14. M?rz 2019 13:24 >> To: Reingruber, Richard ; Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Richard, >> >> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>> Hi Robbin, >>> >>> > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>> > is that it was deemed that a safepoint is 'long enough'. >>> >>> This doesn't seem to be a very good reason. I did a little modification to your >>> initial figure: >> >> That's why we are adding the instruction barrier! >> >>> >>> JavaThread: | VMThread >>> StoreLoad | >>> | update >>> | ? Global ICache Invalidate ? >>> | disarm >>> load thread_poll | >>> >>> Question would be: does an ICache flush happen after the immediate oop update? >>> I would consider it problematic if not. Even if the safepoint was very long. >> >> The icache flush assumes it can stop speculation and flush instruction cache in >> patchee thread. >> Correct me if anything is wrong: >> - sparc needs flush in all patchee threads. >> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >> - ppc needs isync in all patchee threads? >> - s390 needs zarch_sync all in patchee threads? >> - x86 needs cpuid in all in patchee threads. >> >> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >> have no effect in patchee threads!? >> >> Adding a few words on x86 ICache::Invalidate/word. >> We have TSO, so if the patchee thread can see poll is disarmed all other stores >> are visible. I have seen no case ICache::Invalidate/word would make a difference >> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >> The problem is if we prefect an instruction over the load of the poll. >> If poll is disarmed this perfected instruction can be fully executed. >> If that happens to contain an immediate oop, we don't know if it's the patched >> oop or not. >> >> /Robbin >> >>> >>> Cheers Richard. >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>> To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >>> Cc: Reingruber, Richard >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Martin, >>> >>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>> Hi Robbin, >>>> >>>>> But that is whole new subject and let's keep that separate from this :) >>>> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >>>> >>>> >>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>> >>>> orderAccess_aix/linux_ppc.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>> >>>> orderAccess_linux_s390.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>> >>> >>> Thanks! >>> >>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>>>> I couldn't find a call to AbstractICache::invalidate_range. >>> >>> I think I miss-understood you here, the reason I heard for not doing >>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>> safepoint is 'long enough'. >>> >>> /Robbin >>> >>>> >>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>> And one call to AbstractICache/ICache::invalidate_word. >>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>> But that is whole new subject and let's keep that separate from this :) >>>> >>>> I'll create issues for some of these topics, which if nothing else should be >>>> discussed. >>>> >>>> /Robbin >>>> >>>>> >>>>> Does anybody know where it is or if it's missing? >>>>> >>>>> Best regards, >>>>> Martin >>>>> From robbin.ehn at oracle.com Mon Mar 18 11:00:03 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 18 Mar 2019 12:00:03 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <29eea53113104c9eaf4325c56d70dc11@sap.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> <29eea53113104c9eaf4325c56d70dc11@sap.com> Message-ID: > Let's assume J2's modification makes it into J1's instruction stream. How is it > guaranteed that the new compiled code for methodB from the JIT makes it into > J1's instruction stream, too, if no serializing instruction is executed by J1? In short, no, there are no such guarantee AFAIK. If methodB is generate in memory with a valid instruction stream, we may execute the old one. And J1 may see the unresolved call and also try to resolve it, seem much more likely. Erik? have looked more into these cases. (I don't plan to address them in this patch) /Robbin > My explanation was that the call instruction has the serializing effect, just > like in OPTION 1. Note as well the difference between OPTION 1 and 2: if the new > code is not reached by a jump then a serializing instruction is required as > well. > > > That's the "accidental success" that is being avoided by this fix > > Which part of the fix would that be? > > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Montag, 18. M?rz 2019 03:07 > To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > On 15/03/2019 11:04 pm, Reingruber, Richard wrote: >> > OPTION 1 is for self-modifying code, not cross-modifying code. >> >> I know. I wanted to point out that it seems to work for cross-modifying code, >> too, in the sense that patchee java threads don't execute a serializing instruction >> before jumping to compiled code or before executing a compiled call another java >> thread just resolved. > > That's the "accidental success" that is being avoided by this fix. Today > it all seems to work just fine because the "distance" between the > actions of the patching thread and the patchee thread that wants to use > the new code is "vast". > > Cheers, > David > >> Cheers, Richard. >> >> -----Original Message----- >> From: David Holmes >> Sent: Freitag, 15. M?rz 2019 12:55 >> To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 15/03/2019 9:08 pm, Reingruber, Richard wrote: >>> Hi Robbin, >>> >>> I'm referring to x86 here. >>> >>> My comment/question about the requirement of an ICache invalidation in the >>> patcher thread was a little "undereducated". After reading recent manuals [1]: >>> >>> 1. I think your right that a serializing instruction after the safepoint poll >>> is necessary. >>> >>> 2. I doubt that cache flushing is required at all, according to [1]. At least >>> SPECjvm2008 runs with an AbstractICache::_flush_icache_stub that >>> immediately returns. >>> >>> In the sense of [1] JIT compilers and JIT runtimes are cross-modifying code, >>> too, when they compile bytecode and resolve compiled calls. But afaik the java >>> threads don't execute a serializing instruction before executing that code. So, >>> if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee threads jump to the >>> modified code is legal as well. >> >> OPTION 1 is for self-modifying code, not cross-modifying code. >> >> David >> ----- >> >>> So at least theoretically you should be able to replace the serializing >>> instruction with a call to an address you would load from the thread. The >>> address should lead to the runtime, where the thread would block if the >>> safepoint is still active or just return if it is not. >>> >>> The last point was rather exploring the space of legal solutions than a >>> suggestion to solve the issue. I'd favor your approach if you manage to do the >>> serialization conditionally (BTW: a mechanism similar to JavaThread::*deopt_suspend* >>> would be yet another way to do that). >>> >>> Cheers, Richard. >>> >>> [1] 8.1.3 Handling Self- and Cross-Modifying Code >>> in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >>> (3A, 3B, 3C & 3D): System Programming Guide >>> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >>> on https://software.intel.com/en-us/articles/intel-sdm >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 13:24 >>> To: Reingruber, Richard ; Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Richard, >>> >>> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>>> Hi Robbin, >>>> >>>> > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>>> > is that it was deemed that a safepoint is 'long enough'. >>>> >>>> This doesn't seem to be a very good reason. I did a little modification to your >>>> initial figure: >>> >>> That's why we are adding the instruction barrier! >>> >>>> >>>> JavaThread: | VMThread >>>> StoreLoad | >>>> | update >>>> | ? Global ICache Invalidate ? >>>> | disarm >>>> load thread_poll | >>>> >>>> Question would be: does an ICache flush happen after the immediate oop update? >>>> I would consider it problematic if not. Even if the safepoint was very long. >>> >>> The icache flush assumes it can stop speculation and flush instruction cache in >>> patchee thread. >>> Correct me if anything is wrong: >>> - sparc needs flush in all patchee threads. >>> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >>> - ppc needs isync in all patchee threads? >>> - s390 needs zarch_sync all in patchee threads? >>> - x86 needs cpuid in all in patchee threads. >>> >>> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >>> have no effect in patchee threads!? >>> >>> Adding a few words on x86 ICache::Invalidate/word. >>> We have TSO, so if the patchee thread can see poll is disarmed all other stores >>> are visible. I have seen no case ICache::Invalidate/word would make a difference >>> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >>> The problem is if we prefect an instruction over the load of the poll. >>> If poll is disarmed this perfected instruction can be fully executed. >>> If that happens to contain an immediate oop, we don't know if it's the patched >>> oop or not. >>> >>> /Robbin >>> >>>> >>>> Cheers Richard. >>>> >>>> -----Original Message----- >>>> From: Robbin Ehn >>>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>>> To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >>>> Cc: Reingruber, Richard >>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>> >>>> Hi Martin, >>>> >>>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>>> Hi Robbin, >>>>> >>>>>> But that is whole new subject and let's keep that separate from this :) >>>>> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >>>>> >>>>> >>>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>>> >>>>> orderAccess_aix/linux_ppc.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>>> >>>>> orderAccess_linux_s390.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>>> >>>> >>>> Thanks! >>>> >>>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>>>>> I couldn't find a call to AbstractICache::invalidate_range. >>>> >>>> I think I miss-understood you here, the reason I heard for not doing >>>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>>> safepoint is 'long enough'. >>>> >>>> /Robbin >>>> >>>>> >>>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>>> And one call to AbstractICache/ICache::invalidate_word. >>>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>>> But that is whole new subject and let's keep that separate from this :) >>>>> >>>>> I'll create issues for some of these topics, which if nothing else should be >>>>> discussed. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Does anybody know where it is or if it's missing? >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> From erik.osterlund at oracle.com Mon Mar 18 11:30:59 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 18 Mar 2019 12:30:59 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> <29eea53113104c9eaf4325c56d70dc11@sap.com> Message-ID: Hi, Safely publishing JIT-compiled code is outside the scope of this patch, and a completely different can of worms. This patch only intends to make the code patching we do in HotSpot safer. To safely publish JIT-compiled code, some more effort will be required, and I think that discussion can wait. Just pretend it makes sense for now. Thanks, /Erik On 2019-03-18 12:00, Robbin Ehn wrote: >> Let's assume J2's modification makes it into J1's instruction stream. >> How is it >> guaranteed that the new compiled code for methodB from the JIT makes >> it into >> J1's instruction stream, too, if no serializing instruction is >> executed by J1? > > In short, no, there are no such guarantee AFAIK. > If methodB is generate in memory with a valid instruction stream, we > may execute the old one. > And J1 may see the unresolved call and also try to resolve it, seem > much more likely. > > Erik? have looked more into these cases. (I don't plan to address them > in this patch) > > /Robbin > >> My explanation was that the call instruction has the serializing >> effect, just >> like in OPTION 1. Note as well the difference between OPTION 1 and 2: >> if the new >> code is not reached by a jump then a serializing instruction is >> required as >> well. >> >> ?? > That's the "accidental success" that is being avoided by this fix >> >> Which part of the fix would that be? >> >> Thanks, Richard. >> >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 18. M?rz 2019 03:07 >> To: Reingruber, Richard ; Robbin Ehn >> ; Doerr, Martin ; Andrew >> Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 15/03/2019 11:04 pm, Reingruber, Richard wrote: >>> ??? > OPTION 1 is for self-modifying code, not cross-modifying code. >>> >>> I know. I wanted to point out that it seems to work for >>> cross-modifying code, >>> too, in the sense that patchee java threads don't execute a >>> serializing instruction >>> before jumping to compiled code or before executing a compiled call >>> another java >>> thread just resolved. >> >> That's the "accidental success" that is being avoided by this fix. Today >> it all seems to work just fine because the "distance" between the >> actions of the patching thread and the patchee thread that wants to use >> the new code is "vast". >> >> Cheers, >> David >> >>> Cheers, Richard. >>> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Freitag, 15. M?rz 2019 12:55 >>> To: Reingruber, Richard ; Robbin Ehn >>> ; Doerr, Martin ; >>> Andrew Haley ; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> On 15/03/2019 9:08 pm, Reingruber, Richard wrote: >>>> Hi Robbin, >>>> >>>> I'm referring to x86 here. >>>> >>>> My comment/question about the requirement of an ICache invalidation >>>> in the >>>> patcher thread was a little "undereducated". After reading recent >>>> manuals [1]: >>>> >>>> ???? 1. I think your right that a serializing instruction after the >>>> safepoint poll >>>> ??????? is necessary. >>>> >>>> ???? 2. I doubt that cache flushing is required at all, according >>>> to [1]. At least >>>> ??????? SPECjvm2008 runs with an AbstractICache::_flush_icache_stub >>>> that >>>> ??????? immediately returns. >>>> >>>> In the sense of [1] JIT compilers and JIT runtimes are >>>> cross-modifying code, >>>> too, when they compile bytecode and resolve compiled calls. But >>>> afaik the java >>>> threads don't execute a serializing instruction before executing >>>> that code. So, >>>> if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee >>>> threads jump to the >>>> modified code is legal as well. >>> >>> OPTION 1 is for self-modifying code, not cross-modifying code. >>> >>> David >>> ----- >>> >>>> So at least theoretically you should be able to replace the >>>> serializing >>>> instruction with a call to an address you would load from the >>>> thread. The >>>> address should lead to the runtime, where the thread would block if >>>> the >>>> safepoint is still active or just return if it is not. >>>> >>>> The last point was rather exploring the space of legal solutions >>>> than a >>>> suggestion to solve the issue. I'd favor your approach if you >>>> manage to do the >>>> serialization conditionally (BTW: a mechanism similar to >>>> JavaThread::*deopt_suspend* >>>> would be yet another way to do that). >>>> >>>> Cheers, Richard. >>>> >>>> [1] 8.1.3 Handling Self- and Cross-Modifying Code >>>> ?????? in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's >>>> Manual. Volume 3 >>>> ?????? (3A, 3B, 3C & 3D): System Programming Guide >>>> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >>>> ?????? on https://software.intel.com/en-us/articles/intel-sdm >>>> >>>> -----Original Message----- >>>> From: Robbin Ehn >>>> Sent: Donnerstag, 14. M?rz 2019 13:24 >>>> To: Reingruber, Richard ; Doerr, Martin >>>> ; David Holmes ; >>>> hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>> >>>> Hi Richard, >>>> >>>> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>>>> Hi Robbin, >>>>> >>>>> ????? > the reason I heard for not doing invalidate_range/word in >>>>> nmethod::oops_do >>>>> ????? > is that it was deemed that a safepoint is 'long enough'. >>>>> >>>>> This doesn't seem to be a very good reason. I did a little >>>>> modification to your >>>>> initial figure: >>>> >>>> That's why we are adding the instruction barrier! >>>> >>>>> >>>>> JavaThread:???????????? | VMThread >>>>> StoreLoad?????????????? | >>>>> ??????????????????????????? | update >>>>> ??????????????????????????? | ? Global ICache Invalidate ? >>>>> ??????????????????????????? | disarm >>>>> load thread_poll??????? | >>>>> >>>>> Question would be: does an ICache flush happen after the immediate >>>>> oop update? >>>>> I would consider it problematic if not. Even if the safepoint was >>>>> very long. >>>> >>>> The icache flush assumes it can stop speculation and flush >>>> instruction cache in >>>> patchee thread. >>>> Correct me if anything is wrong: >>>> - sparc needs flush in all patchee threads. >>>> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee >>>> threads. >>>> - ppc needs isync in all patchee threads? >>>> - s390 needs zarch_sync all in patchee threads? >>>> - x86 needs cpuid in all in patchee threads. >>>> >>>> So a "Global ICache Invalidate" done by patcher thread (VMThread in >>>> this case) >>>> have no effect in patchee threads!? >>>> >>>> Adding a few words on x86 ICache::Invalidate/word. >>>> We have TSO, so if the patchee thread can see poll is disarmed all >>>> other stores >>>> are visible. I have seen no case ICache::Invalidate/word would make >>>> a difference >>>> from a StoreStore/compiler barrier (still talking about platform >>>> code for x86) ? >>>> The problem is if we prefect an instruction over the load of the poll. >>>> If poll is disarmed this perfected instruction can be fully executed. >>>> If that happens to contain an immediate oop, we don't know if it's >>>> the patched >>>> oop or not. >>>> >>>> /Robbin >>>> >>>>> >>>>> Cheers Richard. >>>>> >>>>> -----Original Message----- >>>>> From: Robbin Ehn >>>>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>>>> To: Doerr, Martin ; David Holmes >>>>> ; hotspot-dev at openjdk.java.net >>>>> Cc: Reingruber, Richard >>>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>>> >>>>> Hi Martin, >>>>> >>>>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>>>> Hi Robbin, >>>>>> >>>>>>> But that is whole new subject and let's keep that separate from >>>>>>> this :) >>>>>> Sure, we can keep the modification side discussion separate. This >>>>>> change is already difficult enough ?? >>>>>> >>>>>> >>>>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here >>>>>> they are: >>>>>> >>>>>> orderAccess_aix/linux_ppc.hpp: >>>>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>>>> >>>>>> orderAccess_linux_s390.hpp: >>>>>> inline void OrderAccess::instruction_pipeline() { >>>>>> inlasm_zarch_sync(); } >>>>>> >>>>> >>>>> Thanks! >>>>> >>>>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for >>>>>>> oops encoded into the instruction stream. >>>>>>> I couldn't find a call to AbstractICache::invalidate_range. >>>>> >>>>> I think I miss-understood you here, the reason I heard for not doing >>>>> invalidate_range/word in nmethod::oops_do is that it was deemed >>>>> that a >>>>> safepoint is 'long enough'. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>>>> And one call to AbstractICache/ICache::invalidate_word. >>>>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>>>> But that is whole new subject and let's keep that separate from >>>>>> this :) >>>>>> >>>>>> I'll create issues for some of these topics, which if nothing >>>>>> else should be >>>>>> discussed. >>>>>> >>>>>> /Robbin >>>>>> >>>>>>> >>>>>>> Does anybody know where it is or if it's missing? >>>>>>> >>>>>>> Best regards, >>>>>>> Martin >>>>>>> From richard.reingruber at sap.com Mon Mar 18 11:36:00 2019 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 18 Mar 2019 11:36:00 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> <29eea53113104c9eaf4325c56d70dc11@sap.com> Message-ID: Totally accepted. I was only trying to understand the rules for cross-modifying code on x86, which is a prerequisite for the issue addressed by Robbins patch. Thanks, Richard. -----Original Message----- From: Erik ?sterlund Sent: Montag, 18. M?rz 2019 12:31 To: Robbin Ehn ; Reingruber, Richard ; David Holmes ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code Hi, Safely publishing JIT-compiled code is outside the scope of this patch, and a completely different can of worms. This patch only intends to make the code patching we do in HotSpot safer. To safely publish JIT-compiled code, some more effort will be required, and I think that discussion can wait. Just pretend it makes sense for now. Thanks, /Erik On 2019-03-18 12:00, Robbin Ehn wrote: >> Let's assume J2's modification makes it into J1's instruction stream. >> How is it >> guaranteed that the new compiled code for methodB from the JIT makes >> it into >> J1's instruction stream, too, if no serializing instruction is >> executed by J1? > > In short, no, there are no such guarantee AFAIK. > If methodB is generate in memory with a valid instruction stream, we > may execute the old one. > And J1 may see the unresolved call and also try to resolve it, seem > much more likely. > > Erik? have looked more into these cases. (I don't plan to address them > in this patch) > > /Robbin > >> My explanation was that the call instruction has the serializing >> effect, just >> like in OPTION 1. Note as well the difference between OPTION 1 and 2: >> if the new >> code is not reached by a jump then a serializing instruction is >> required as >> well. >> >> ?? > That's the "accidental success" that is being avoided by this fix >> >> Which part of the fix would that be? >> >> Thanks, Richard. >> >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 18. M?rz 2019 03:07 >> To: Reingruber, Richard ; Robbin Ehn >> ; Doerr, Martin ; Andrew >> Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 15/03/2019 11:04 pm, Reingruber, Richard wrote: >>> ??? > OPTION 1 is for self-modifying code, not cross-modifying code. >>> >>> I know. I wanted to point out that it seems to work for >>> cross-modifying code, >>> too, in the sense that patchee java threads don't execute a >>> serializing instruction >>> before jumping to compiled code or before executing a compiled call >>> another java >>> thread just resolved. >> >> That's the "accidental success" that is being avoided by this fix. Today >> it all seems to work just fine because the "distance" between the >> actions of the patching thread and the patchee thread that wants to use >> the new code is "vast". >> >> Cheers, >> David >> >>> Cheers, Richard. >>> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Freitag, 15. M?rz 2019 12:55 >>> To: Reingruber, Richard ; Robbin Ehn >>> ; Doerr, Martin ; >>> Andrew Haley ; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> On 15/03/2019 9:08 pm, Reingruber, Richard wrote: >>>> Hi Robbin, >>>> >>>> I'm referring to x86 here. >>>> >>>> My comment/question about the requirement of an ICache invalidation >>>> in the >>>> patcher thread was a little "undereducated". After reading recent >>>> manuals [1]: >>>> >>>> ???? 1. I think your right that a serializing instruction after the >>>> safepoint poll >>>> ??????? is necessary. >>>> >>>> ???? 2. I doubt that cache flushing is required at all, according >>>> to [1]. At least >>>> ??????? SPECjvm2008 runs with an AbstractICache::_flush_icache_stub >>>> that >>>> ??????? immediately returns. >>>> >>>> In the sense of [1] JIT compilers and JIT runtimes are >>>> cross-modifying code, >>>> too, when they compile bytecode and resolve compiled calls. But >>>> afaik the java >>>> threads don't execute a serializing instruction before executing >>>> that code. So, >>>> if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee >>>> threads jump to the >>>> modified code is legal as well. >>> >>> OPTION 1 is for self-modifying code, not cross-modifying code. >>> >>> David >>> ----- >>> >>>> So at least theoretically you should be able to replace the >>>> serializing >>>> instruction with a call to an address you would load from the >>>> thread. The >>>> address should lead to the runtime, where the thread would block if >>>> the >>>> safepoint is still active or just return if it is not. >>>> >>>> The last point was rather exploring the space of legal solutions >>>> than a >>>> suggestion to solve the issue. I'd favor your approach if you >>>> manage to do the >>>> serialization conditionally (BTW: a mechanism similar to >>>> JavaThread::*deopt_suspend* >>>> would be yet another way to do that). >>>> >>>> Cheers, Richard. >>>> >>>> [1] 8.1.3 Handling Self- and Cross-Modifying Code >>>> ?????? in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's >>>> Manual. Volume 3 >>>> ?????? (3A, 3B, 3C & 3D): System Programming Guide >>>> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >>>> ?????? on https://software.intel.com/en-us/articles/intel-sdm >>>> >>>> -----Original Message----- >>>> From: Robbin Ehn >>>> Sent: Donnerstag, 14. M?rz 2019 13:24 >>>> To: Reingruber, Richard ; Doerr, Martin >>>> ; David Holmes ; >>>> hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>> >>>> Hi Richard, >>>> >>>> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>>>> Hi Robbin, >>>>> >>>>> ????? > the reason I heard for not doing invalidate_range/word in >>>>> nmethod::oops_do >>>>> ????? > is that it was deemed that a safepoint is 'long enough'. >>>>> >>>>> This doesn't seem to be a very good reason. I did a little >>>>> modification to your >>>>> initial figure: >>>> >>>> That's why we are adding the instruction barrier! >>>> >>>>> >>>>> JavaThread:???????????? | VMThread >>>>> StoreLoad?????????????? | >>>>> ??????????????????????????? | update >>>>> ??????????????????????????? | ? Global ICache Invalidate ? >>>>> ??????????????????????????? | disarm >>>>> load thread_poll??????? | >>>>> >>>>> Question would be: does an ICache flush happen after the immediate >>>>> oop update? >>>>> I would consider it problematic if not. Even if the safepoint was >>>>> very long. >>>> >>>> The icache flush assumes it can stop speculation and flush >>>> instruction cache in >>>> patchee thread. >>>> Correct me if anything is wrong: >>>> - sparc needs flush in all patchee threads. >>>> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee >>>> threads. >>>> - ppc needs isync in all patchee threads? >>>> - s390 needs zarch_sync all in patchee threads? >>>> - x86 needs cpuid in all in patchee threads. >>>> >>>> So a "Global ICache Invalidate" done by patcher thread (VMThread in >>>> this case) >>>> have no effect in patchee threads!? >>>> >>>> Adding a few words on x86 ICache::Invalidate/word. >>>> We have TSO, so if the patchee thread can see poll is disarmed all >>>> other stores >>>> are visible. I have seen no case ICache::Invalidate/word would make >>>> a difference >>>> from a StoreStore/compiler barrier (still talking about platform >>>> code for x86) ? >>>> The problem is if we prefect an instruction over the load of the poll. >>>> If poll is disarmed this perfected instruction can be fully executed. >>>> If that happens to contain an immediate oop, we don't know if it's >>>> the patched >>>> oop or not. >>>> >>>> /Robbin >>>> >>>>> >>>>> Cheers Richard. >>>>> >>>>> -----Original Message----- >>>>> From: Robbin Ehn >>>>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>>>> To: Doerr, Martin ; David Holmes >>>>> ; hotspot-dev at openjdk.java.net >>>>> Cc: Reingruber, Richard >>>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>>> >>>>> Hi Martin, >>>>> >>>>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>>>> Hi Robbin, >>>>>> >>>>>>> But that is whole new subject and let's keep that separate from >>>>>>> this :) >>>>>> Sure, we can keep the modification side discussion separate. This >>>>>> change is already difficult enough ?? >>>>>> >>>>>> >>>>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here >>>>>> they are: >>>>>> >>>>>> orderAccess_aix/linux_ppc.hpp: >>>>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>>>> >>>>>> orderAccess_linux_s390.hpp: >>>>>> inline void OrderAccess::instruction_pipeline() { >>>>>> inlasm_zarch_sync(); } >>>>>> >>>>> >>>>> Thanks! >>>>> >>>>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for >>>>>>> oops encoded into the instruction stream. >>>>>>> I couldn't find a call to AbstractICache::invalidate_range. >>>>> >>>>> I think I miss-understood you here, the reason I heard for not doing >>>>> invalidate_range/word in nmethod::oops_do is that it was deemed >>>>> that a >>>>> safepoint is 'long enough'. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>>>> And one call to AbstractICache/ICache::invalidate_word. >>>>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>>>> But that is whole new subject and let's keep that separate from >>>>>> this :) >>>>>> >>>>>> I'll create issues for some of these topics, which if nothing >>>>>> else should be >>>>>> discussed. >>>>>> >>>>>> /Robbin >>>>>> >>>>>>> >>>>>>> Does anybody know where it is or if it's missing? >>>>>>> >>>>>>> Best regards, >>>>>>> Martin >>>>>>> From robbin.ehn at oracle.com Mon Mar 18 12:33:57 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 18 Mar 2019 13:33:57 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> <29eea53113104c9eaf4325c56d70dc11@sap.com> Message-ID: <5e8c3b7e-c8d4-4c88-abf0-2a1e6e50c081@oracle.com> For x86, this part in Intel/(AMD) the manual I'm following: " To write cross-modifying code and ensure that it is compliant with current and future versions of the IA-32 architecture, the following processor synchronization algorithm must be implemented: (* Action of Modifying Processor *) Memory_Flag ? 0; (* Set Memory_Flag to value other than 1 *) Store modified code (as data) into code segment; Memory_Flag ? 1; (* Action of Executing Processor *) WHILE (Memory_Flag ? 1) Wait for code to update; ELIHW; Execute serializing instruction; (* For example, CPUID instruction *) Begin executing modified code; " For this patch the poll is the Memory_Flag we are changing state on. There is no mentioning of another compliant way. Even if the pointer publishing would work on a current CPUs it is not future proof. On OS level, membarrier(...) do only guarantee smp_mb() semantics. If it would have stronger guarantees doing OS membar from the patcher would be a second method. /Robbin On 3/18/19 12:36 PM, Reingruber, Richard wrote: > Totally accepted. > > I was only trying to understand the rules for cross-modifying code on x86, which > is a prerequisite for the issue addressed by Robbins patch. > > Thanks, Richard. > > -----Original Message----- > From: Erik ?sterlund > Sent: Montag, 18. M?rz 2019 12:31 > To: Robbin Ehn ; Reingruber, Richard ; David Holmes ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi, > > Safely publishing JIT-compiled code is outside the scope of this patch, > and a completely different can of worms. This patch only intends to make > the code patching we do in HotSpot safer. To safely publish JIT-compiled > code, some more effort will be required, and I think that discussion can > wait. Just pretend it makes sense for now. > > Thanks, > /Erik > > On 2019-03-18 12:00, Robbin Ehn wrote: >>> Let's assume J2's modification makes it into J1's instruction stream. >>> How is it >>> guaranteed that the new compiled code for methodB from the JIT makes >>> it into >>> J1's instruction stream, too, if no serializing instruction is >>> executed by J1? >> >> In short, no, there are no such guarantee AFAIK. >> If methodB is generate in memory with a valid instruction stream, we >> may execute the old one. >> And J1 may see the unresolved call and also try to resolve it, seem >> much more likely. >> >> Erik? have looked more into these cases. (I don't plan to address them >> in this patch) >> >> /Robbin >> >>> My explanation was that the call instruction has the serializing >>> effect, just >>> like in OPTION 1. Note as well the difference between OPTION 1 and 2: >>> if the new >>> code is not reached by a jump then a serializing instruction is >>> required as >>> well. >>> >>> ?? > That's the "accidental success" that is being avoided by this fix >>> >>> Which part of the fix would that be? >>> >>> Thanks, Richard. >>> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Montag, 18. M?rz 2019 03:07 >>> To: Reingruber, Richard ; Robbin Ehn >>> ; Doerr, Martin ; Andrew >>> Haley ; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> On 15/03/2019 11:04 pm, Reingruber, Richard wrote: >>>> ??? > OPTION 1 is for self-modifying code, not cross-modifying code. >>>> >>>> I know. I wanted to point out that it seems to work for >>>> cross-modifying code, >>>> too, in the sense that patchee java threads don't execute a >>>> serializing instruction >>>> before jumping to compiled code or before executing a compiled call >>>> another java >>>> thread just resolved. >>> >>> That's the "accidental success" that is being avoided by this fix. Today >>> it all seems to work just fine because the "distance" between the >>> actions of the patching thread and the patchee thread that wants to use >>> the new code is "vast". >>> >>> Cheers, >>> David >>> >>>> Cheers, Richard. >>>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Freitag, 15. M?rz 2019 12:55 >>>> To: Reingruber, Richard ; Robbin Ehn >>>> ; Doerr, Martin ; >>>> Andrew Haley ; hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>> >>>> On 15/03/2019 9:08 pm, Reingruber, Richard wrote: >>>>> Hi Robbin, >>>>> >>>>> I'm referring to x86 here. >>>>> >>>>> My comment/question about the requirement of an ICache invalidation >>>>> in the >>>>> patcher thread was a little "undereducated". After reading recent >>>>> manuals [1]: >>>>> >>>>> ???? 1. I think your right that a serializing instruction after the >>>>> safepoint poll >>>>> ??????? is necessary. >>>>> >>>>> ???? 2. I doubt that cache flushing is required at all, according >>>>> to [1]. At least >>>>> ??????? SPECjvm2008 runs with an AbstractICache::_flush_icache_stub >>>>> that >>>>> ??????? immediately returns. >>>>> >>>>> In the sense of [1] JIT compilers and JIT runtimes are >>>>> cross-modifying code, >>>>> too, when they compile bytecode and resolve compiled calls. But >>>>> afaik the java >>>>> threads don't execute a serializing instruction before executing >>>>> that code. So, >>>>> if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee >>>>> threads jump to the >>>>> modified code is legal as well. >>>> >>>> OPTION 1 is for self-modifying code, not cross-modifying code. >>>> >>>> David >>>> ----- >>>> >>>>> So at least theoretically you should be able to replace the >>>>> serializing >>>>> instruction with a call to an address you would load from the >>>>> thread. The >>>>> address should lead to the runtime, where the thread would block if >>>>> the >>>>> safepoint is still active or just return if it is not. >>>>> >>>>> The last point was rather exploring the space of legal solutions >>>>> than a >>>>> suggestion to solve the issue. I'd favor your approach if you >>>>> manage to do the >>>>> serialization conditionally (BTW: a mechanism similar to >>>>> JavaThread::*deopt_suspend* >>>>> would be yet another way to do that). >>>>> >>>>> Cheers, Richard. >>>>> >>>>> [1] 8.1.3 Handling Self- and Cross-Modifying Code >>>>> ?????? in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's >>>>> Manual. Volume 3 >>>>> ?????? (3A, 3B, 3C & 3D): System Programming Guide >>>>> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >>>>> ?????? on https://software.intel.com/en-us/articles/intel-sdm >>>>> >>>>> -----Original Message----- >>>>> From: Robbin Ehn >>>>> Sent: Donnerstag, 14. M?rz 2019 13:24 >>>>> To: Reingruber, Richard ; Doerr, Martin >>>>> ; David Holmes ; >>>>> hotspot-dev at openjdk.java.net >>>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>>> >>>>> Hi Richard, >>>>> >>>>> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>>>>> Hi Robbin, >>>>>> >>>>>> ????? > the reason I heard for not doing invalidate_range/word in >>>>>> nmethod::oops_do >>>>>> ????? > is that it was deemed that a safepoint is 'long enough'. >>>>>> >>>>>> This doesn't seem to be a very good reason. I did a little >>>>>> modification to your >>>>>> initial figure: >>>>> >>>>> That's why we are adding the instruction barrier! >>>>> >>>>>> >>>>>> JavaThread:???????????? | VMThread >>>>>> StoreLoad?????????????? | >>>>>> ??????????????????????????? | update >>>>>> ??????????????????????????? | ? Global ICache Invalidate ? >>>>>> ??????????????????????????? | disarm >>>>>> load thread_poll??????? | >>>>>> >>>>>> Question would be: does an ICache flush happen after the immediate >>>>>> oop update? >>>>>> I would consider it problematic if not. Even if the safepoint was >>>>>> very long. >>>>> >>>>> The icache flush assumes it can stop speculation and flush >>>>> instruction cache in >>>>> patchee thread. >>>>> Correct me if anything is wrong: >>>>> - sparc needs flush in all patchee threads. >>>>> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee >>>>> threads. >>>>> - ppc needs isync in all patchee threads? >>>>> - s390 needs zarch_sync all in patchee threads? >>>>> - x86 needs cpuid in all in patchee threads. >>>>> >>>>> So a "Global ICache Invalidate" done by patcher thread (VMThread in >>>>> this case) >>>>> have no effect in patchee threads!? >>>>> >>>>> Adding a few words on x86 ICache::Invalidate/word. >>>>> We have TSO, so if the patchee thread can see poll is disarmed all >>>>> other stores >>>>> are visible. I have seen no case ICache::Invalidate/word would make >>>>> a difference >>>>> from a StoreStore/compiler barrier (still talking about platform >>>>> code for x86) ? >>>>> The problem is if we prefect an instruction over the load of the poll. >>>>> If poll is disarmed this perfected instruction can be fully executed. >>>>> If that happens to contain an immediate oop, we don't know if it's >>>>> the patched >>>>> oop or not. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Cheers Richard. >>>>>> >>>>>> -----Original Message----- >>>>>> From: Robbin Ehn >>>>>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>>>>> To: Doerr, Martin ; David Holmes >>>>>> ; hotspot-dev at openjdk.java.net >>>>>> Cc: Reingruber, Richard >>>>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>>>>> Hi Robbin, >>>>>>> >>>>>>>> But that is whole new subject and let's keep that separate from >>>>>>>> this :) >>>>>>> Sure, we can keep the modification side discussion separate. This >>>>>>> change is already difficult enough ?? >>>>>>> >>>>>>> >>>>>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here >>>>>>> they are: >>>>>>> >>>>>>> orderAccess_aix/linux_ppc.hpp: >>>>>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>>>>> >>>>>>> orderAccess_linux_s390.hpp: >>>>>>> inline void OrderAccess::instruction_pipeline() { >>>>>>> inlasm_zarch_sync(); } >>>>>>> >>>>>> >>>>>> Thanks! >>>>>> >>>>>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for >>>>>>>> oops encoded into the instruction stream. >>>>>>>> I couldn't find a call to AbstractICache::invalidate_range. >>>>>> >>>>>> I think I miss-understood you here, the reason I heard for not doing >>>>>> invalidate_range/word in nmethod::oops_do is that it was deemed >>>>>> that a >>>>>> safepoint is 'long enough'. >>>>>> >>>>>> /Robbin >>>>>> >>>>>>> >>>>>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>>>>> And one call to AbstractICache/ICache::invalidate_word. >>>>>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>>>>> But that is whole new subject and let's keep that separate from >>>>>>> this :) >>>>>>> >>>>>>> I'll create issues for some of these topics, which if nothing >>>>>>> else should be >>>>>>> discussed. >>>>>>> >>>>>>> /Robbin >>>>>>> >>>>>>>> >>>>>>>> Does anybody know where it is or if it's missing? >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Martin >>>>>>>> > From david.holmes at oracle.com Mon Mar 18 13:00:39 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Mar 2019 23:00:39 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <29eea53113104c9eaf4325c56d70dc11@sap.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <60c8545b-eb71-c8bd-1b9a-3b4740817da9@oracle.com> <29eea53113104c9eaf4325c56d70dc11@sap.com> Message-ID: I think we're starting to talk at cross-purposes here ... On 18/03/2019 8:37 pm, Reingruber, Richard wrote: > > That's the "accidental success" that is being avoided by this fix > > I don't think so, but maybe I'm missing something. So if you look at this example: You started by stating that the solution for self-modifying code currently seems to work in the case of cross-modifying code. I was simply stating that if it does "seem to work" then this accidental success and not something to be relied upon. I was expecting the "barrier" introduced by this patch to be (part of) the solution that could be relied upon. David ----- > > static void methodA() { > > // ... // <- J1 executes this > > methodB(); // <- J2 executes this > > // .. > > } > > Where > > * methodA is jit compiled with an unresolved call to methodB > > * Java thread J1 executes that compiled version of methodA up to just before the unresolved call > to methodB > > * JIT compiles methodB and registers the code. > > * Java thread J2 executes the unresolved call which takes it to SharedRuntime::resolve_static_call_C() > > * J2 finds the target methodB and the compiled code for it with entry point E. It patches the call > in the compiled code for methodA setting E as its target. This is done without safepoint. > (see SharedRuntime::resolve_sub_helper_internal()) > > * Now to the interesting part: J1 executes the patched call. Until here I see 2 > crossmodifications: (1) jit compilation of methodB: JIT is the patcher thread, J1 and J2 are the > patchees (2) patching the call: J2 is the patcher (this is actually also a self-modification) > and J1 is the patchee. > > Let's assume J2's modification makes it into J1's instruction stream. How is it > guaranteed that the new compiled code for methodB from the JIT makes it into > J1's instruction stream, too, if no serializing instruction is executed by J1? > My explanation was that the call instruction has the serializing effect, just > like in OPTION 1. Note as well the difference between OPTION 1 and 2: if the new > code is not reached by a jump then a serializing instruction is required as > well. > > > That's the "accidental success" that is being avoided by this fix > > Which part of the fix would that be? > > Thanks, Richard. > > -----Original Message----- > From: David Holmes > Sent: Montag, 18. M?rz 2019 03:07 > To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > > On 15/03/2019 11:04 pm, Reingruber, Richard wrote: >> > OPTION 1 is for self-modifying code, not cross-modifying code. >> >> I know. I wanted to point out that it seems to work for cross-modifying code, >> too, in the sense that patchee java threads don't execute a serializing instruction >> before jumping to compiled code or before executing a compiled call another java >> thread just resolved. > > That's the "accidental success" that is being avoided by this fix. Today > it all seems to work just fine because the "distance" between the > actions of the patching thread and the patchee thread that wants to use > the new code is "vast". > > Cheers, > David > >> Cheers, Richard. >> >> -----Original Message----- >> From: David Holmes >> Sent: Freitag, 15. M?rz 2019 12:55 >> To: Reingruber, Richard ; Robbin Ehn ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 15/03/2019 9:08 pm, Reingruber, Richard wrote: >>> Hi Robbin, >>> >>> I'm referring to x86 here. >>> >>> My comment/question about the requirement of an ICache invalidation in the >>> patcher thread was a little "undereducated". After reading recent manuals [1]: >>> >>> 1. I think your right that a serializing instruction after the safepoint poll >>> is necessary. >>> >>> 2. I doubt that cache flushing is required at all, according to [1]. At least >>> SPECjvm2008 runs with an AbstractICache::_flush_icache_stub that >>> immediately returns. >>> >>> In the sense of [1] JIT compilers and JIT runtimes are cross-modifying code, >>> too, when they compile bytecode and resolve compiled calls. But afaik the java >>> threads don't execute a serializing instruction before executing that code. So, >>> if I'm not mistaken, (* OPTION 1 *) of [1] where the patchee threads jump to the >>> modified code is legal as well. >> >> OPTION 1 is for self-modifying code, not cross-modifying code. >> >> David >> ----- >> >>> So at least theoretically you should be able to replace the serializing >>> instruction with a call to an address you would load from the thread. The >>> address should lead to the runtime, where the thread would block if the >>> safepoint is still active or just return if it is not. >>> >>> The last point was rather exploring the space of legal solutions than a >>> suggestion to solve the issue. I'd favor your approach if you manage to do the >>> serialization conditionally (BTW: a mechanism similar to JavaThread::*deopt_suspend* >>> would be yet another way to do that). >>> >>> Cheers, Richard. >>> >>> [1] 8.1.3 Handling Self- and Cross-Modifying Code >>> in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >>> (3A, 3B, 3C & 3D): System Programming Guide >>> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >>> on https://software.intel.com/en-us/articles/intel-sdm >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 13:24 >>> To: Reingruber, Richard ; Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Richard, >>> >>> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>>> Hi Robbin, >>>> >>>> > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>>> > is that it was deemed that a safepoint is 'long enough'. >>>> >>>> This doesn't seem to be a very good reason. I did a little modification to your >>>> initial figure: >>> >>> That's why we are adding the instruction barrier! >>> >>>> >>>> JavaThread: | VMThread >>>> StoreLoad | >>>> | update >>>> | ? Global ICache Invalidate ? >>>> | disarm >>>> load thread_poll | >>>> >>>> Question would be: does an ICache flush happen after the immediate oop update? >>>> I would consider it problematic if not. Even if the safepoint was very long. >>> >>> The icache flush assumes it can stop speculation and flush instruction cache in >>> patchee thread. >>> Correct me if anything is wrong: >>> - sparc needs flush in all patchee threads. >>> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >>> - ppc needs isync in all patchee threads? >>> - s390 needs zarch_sync all in patchee threads? >>> - x86 needs cpuid in all in patchee threads. >>> >>> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >>> have no effect in patchee threads!? >>> >>> Adding a few words on x86 ICache::Invalidate/word. >>> We have TSO, so if the patchee thread can see poll is disarmed all other stores >>> are visible. I have seen no case ICache::Invalidate/word would make a difference >>> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >>> The problem is if we prefect an instruction over the load of the poll. >>> If poll is disarmed this perfected instruction can be fully executed. >>> If that happens to contain an immediate oop, we don't know if it's the patched >>> oop or not. >>> >>> /Robbin >>> >>>> >>>> Cheers Richard. >>>> >>>> -----Original Message----- >>>> From: Robbin Ehn >>>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>>> To: Doerr, Martin ; David Holmes ; hotspot-dev at openjdk.java.net >>>> Cc: Reingruber, Richard >>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>> >>>> Hi Martin, >>>> >>>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>>> Hi Robbin, >>>>> >>>>>> But that is whole new subject and let's keep that separate from this :) >>>>> Sure, we can keep the modification side discussion separate. This change is already difficult enough ?? >>>>> >>>>> >>>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>>> >>>>> orderAccess_aix/linux_ppc.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>>> >>>>> orderAccess_linux_s390.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>>> >>>> >>>> Thanks! >>>> >>>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops encoded into the instruction stream. >>>>>> I couldn't find a call to AbstractICache::invalidate_range. >>>> >>>> I think I miss-understood you here, the reason I heard for not doing >>>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>>> safepoint is 'long enough'. >>>> >>>> /Robbin >>>> >>>>> >>>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>>> And one call to AbstractICache/ICache::invalidate_word. >>>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>>> But that is whole new subject and let's keep that separate from this :) >>>>> >>>>> I'll create issues for some of these topics, which if nothing else should be >>>>> discussed. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Does anybody know where it is or if it's missing? >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> From shade at redhat.com Mon Mar 18 15:41:01 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 18 Mar 2019 16:41:01 +0100 Subject: [11u] RFR (XS) 8220718: Missing ResourceMark in nmethod::metadata_do Message-ID: Hi, Please review a small fix in 11u. It cherry-picks the one-liner change that was pushed to 12u in a much larger one. See the rationale and details in the bug: https://bugs.openjdk.java.net/browse/JDK-8220718 Patch: http://cr.openjdk.java.net/~shade/8220718/webrev.11u.01/ Testing: tier {1,2,3} locally Thanks, -Aleksey From matthias.baesken at sap.com Mon Mar 18 15:41:58 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 18 Mar 2019 15:41:58 +0000 Subject: RFR [XS] : 8220781: linux-s390 : os::get_summary_cpu_info gives bad output Message-ID: Hello, please review the following small fix. Issue : currently the Host: information line in the hs_err file looks strange on linux s390x . Here is an example : Host: s : 6, 6 cores, 11G, SUSE Linux Enterprise Server 11 (s390x) The strange output after Host: ("s : 6") is caused by os::get_summary_cpu_info ; it scans for the string "processor" and runs into the second line of the file : more /proc/cpuinfo vendor_id : IBM/S390 # processors : 6 This leads to "s : 6" . Instead we can get the "machine" ID -number which is available in the /proc/cpuinfo file . This can be used to look up the machine+related processor model . Another question I had when looking into the issue - why is the hostname after "Host:" only printed in the non-product case ? It could be also helpful in the product build as well . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8220781 http://cr.openjdk.java.net/~mbaesken/webrevs/8220781.0/ Thanks, Matthias From martin.doerr at sap.com Mon Mar 18 15:57:12 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 18 Mar 2019 15:57:12 +0000 Subject: [11u] RFR (XS) 8220718: Missing ResourceMark in nmethod::metadata_do In-Reply-To: References: Message-ID: Hi Aleksey, looks good. Thanks for fixing this memory leak in 11u. Best regards, Martin -----Original Message----- From: jdk-updates-dev On Behalf Of Aleksey Shipilev Sent: Montag, 18. M?rz 2019 16:41 To: hotspot-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net Subject: [11u] RFR (XS) 8220718: Missing ResourceMark in nmethod::metadata_do Hi, Please review a small fix in 11u. It cherry-picks the one-liner change that was pushed to 12u in a much larger one. See the rationale and details in the bug: https://bugs.openjdk.java.net/browse/JDK-8220718 Patch: http://cr.openjdk.java.net/~shade/8220718/webrev.11u.01/ Testing: tier {1,2,3} locally Thanks, -Aleksey From serguei.spitsyn at oracle.com Mon Mar 18 16:33:21 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 18 Mar 2019 09:33:21 -0700 Subject: RFR (S) 8220512: Deoptimize redefinition functions that have dirty ICs In-Reply-To: <5682cb5a-4b23-4ea5-6b3b-1471a0bc6499@oracle.com> References: <00317af6-2a5f-5b32-408b-3bd888682c19@oracle.com> <15470b26-c400-1bc5-947f-985ee53cb002@oracle.com> <38cdb82d-c51e-b906-c1e4-2211715fa7b6@oracle.com> <5682cb5a-4b23-4ea5-6b3b-1471a0bc6499@oracle.com> Message-ID: Hi Coleen, Renaming looks good. Thanks, Serguei On 3/15/19 07:58, coleen.phillimore at oracle.com wrote: > > From some offline feedback, I changed the name of has_evol_ics => > has_evol_metadata and a couple of other small things.? I reran this > though builds and tier1 tests. > > Incremental: > http://cr.openjdk.java.net/~coleenp/2019/8220512.02.incr/webrev/index.html > Full: > http://cr.openjdk.java.net/~coleenp/2019/8220512.02/webrev/index.html > > Thanks, > Coleen > > On 3/14/19 3:09 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 3/14/19 2:24 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> It looks good to me. >>> >>> Just one minor suggestion: >>> >>> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev/src/hotspot/share/classfile/metadataOnStackMark.cpp.udiff.html >>> >>> - Threads::metadata_do(Metadata::mark_on_stack); >>> - CodeCache::metadata_do(Metadata::mark_on_stack); >>> + MetadataOnStackClosure mon_stack; >>> + Threads::metadata_do(&mon_stack); >>> + CodeCache::metadata_do(&mon_stack); >>> ?The 'mon_stack' can be associated with monitors. >>> ?How about to rename it to something like 'md_on_stack'? >> >> Okay, I'll change the name to md_on_stack. >> >> Thanks for the code review! >> Coleen >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 3/14/19 10:40, coleen.phillimore at oracle.com wrote: >>>> Summary: Walk ICs to determine whether nmethods are dependent on >>>> redefined classes. >>>> >>>> See bug for more details.? Tested with redefinition tests: >>>> >>>> #redefinition tests. >>>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jvmti >>>> >&jvmti.out >>>> make test TEST=open/test/hotspot/jtreg/vmTestbase/nsk/jdi >&jdi.out >>>> make test TEST=open/test/hotspot/jtreg/runtime/RedefineTests >>>> >&redefine.out >>>> make test TEST=open/test/jdk/java/lang/instrument >&instrument.out >>>> make test TEST=open/test/jdk/com/sun/jdi >&jtreg.jdi.out >>>> >>>> hs-tier1-6 as well as java/lang/instrument tests with -Xcomp. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2019/8220512.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8220512 >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>> >> > From daniel.daugherty at oracle.com Mon Mar 18 16:50:24 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 18 Mar 2019 12:50:24 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> Message-ID: <8e8b296b-18e4-663e-b098-dadb6dce6f90@oracle.com> On 3/15/19 7:03 PM, Kim Barrett wrote: > Please review this fix for intermittent Mac-only crashes involving the > OWSTTaskTerminator. > > This is joint work with Patricio Chilano Mateo. Very impressive bug hunt! > This turned out to be a bug in the macOSX kernel's pthread support > (which may be fixed in OSX 10.14 (Mojave)). When using a condvar, an > associated broadcast entry may be added to a kernel table and left > there after the last use of the condvar. If we then destroy and delete > the condvar and later happen to allocate and init a mutex at the same > address (this really does happen), that stale condvar entry in the > kernel table confuses operations on the mutex, causing crashes and > hangs. > > We have been able to reproduce those failure modes with standalone > programs (no JDK code involved) (see the CR for reproducers), and have > reported the problem to Apple. This reproducer didn't fail when > tested on Mojave, but did fail on all previous OS versions we tried. > > The OWSTTaskTerminator is subject to this problem because a new > HotSpot Monitor object is allocated for each terminator. Terminators > are allocated for various parallel tasks during garbage collection, so > there are quite a few Monitor objects (and their contained > PlatformMonitors) being created and deleted, so many opportunities to > get the kernel into the bad state and then later reuse a previous > condvar address for a mutex. That was enough for the stress test > (gc/stress/TestStressIHOPMultiThread.java) to hit the problem > occasionally. > > This problem would not have shown up before JDK-8210832, which was > made shortly before the first sighting of the stress test failure. > Before that change we re-used park events in the implementation of > HotSpot Mutex/Monitor. Because of that re-use, there wasn't an > opportunity to allocate a pthread_mutex_t at the same address as a > former pthread_cond_t. > > We work around this problem by allocating from a freelist the > mutex/condvar pairs needed by the macOSX PlatformMonitor. > > An alternative workaround that was explored is to (for macOSX only) > add a short timedwait on the condvar when deleting a PlatformMonitor. > The idea is that (1) if there is a lingering kernel table entry for the > condvar, the timedwait will eat it, and (2) if there isn't such an > entry the timedwait will (almost) immediately expire. This would use > pthread_cond_timedwait_relative_np to avoid unneeded clock accesses. > This approach isn't being taken because it might be sensitive to > implementation details that could vary between OS versions. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8218975 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8218975/open.01/ src/hotspot/os/posix/os_posix.hpp ??? Good job on making the freelist work around only apply ??? to MacOSX. src/hotspot/os/posix/os_posix.cpp ??? No comments. src/hotspot/os/posix/os_posix.inline.hpp ??? No comments. I have no objection to your use of the '_ptr' suffix. Thumbs up. Dan > > Testing: > mach5 tier1-5 > mach5 the specific test, 2000 times with no failures. Without this > change, typical failure rate seems to be on the order of 0.5-1.0%. > Performance testing on Mac found no regressions. > From daniel.daugherty at oracle.com Mon Mar 18 16:53:07 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 18 Mar 2019 12:53:07 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <8e8b296b-18e4-663e-b098-dadb6dce6f90@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <8e8b296b-18e4-663e-b098-dadb6dce6f90@oracle.com> Message-ID: I forgot to ask the meta question about how we are going to decide when this freelist work around can be retired so that MacOSX has the same code as the other platforms? Dan On 3/18/19 12:50 PM, Daniel D. Daugherty wrote: > On 3/15/19 7:03 PM, Kim Barrett wrote: >> Please review this fix for intermittent Mac-only crashes involving the >> OWSTTaskTerminator. >> >> This is joint work with Patricio Chilano Mateo. > > Very impressive bug hunt! > > >> This turned out to be a bug in the macOSX kernel's pthread support >> (which may be fixed in OSX 10.14 (Mojave)). When using a condvar, an >> associated broadcast entry may be added to a kernel table and left >> there after the last use of the condvar. If we then destroy and delete >> the condvar and later happen to allocate and init a mutex at the same >> address (this really does happen), that stale condvar entry in the >> kernel table confuses operations on the mutex, causing crashes and >> hangs. >> >> We have been able to reproduce those failure modes with standalone >> programs (no JDK code involved) (see the CR for reproducers), and have >> reported the problem to Apple.? This reproducer didn't fail when >> tested on Mojave, but did fail on all previous OS versions we tried. >> >> The OWSTTaskTerminator is subject to this problem because a new >> HotSpot Monitor object is allocated for each terminator. Terminators >> are allocated for various parallel tasks during garbage collection, so >> there are quite a few Monitor objects (and their contained >> PlatformMonitors) being created and deleted, so many opportunities to >> get the kernel into the bad state and then later reuse a previous >> condvar address for a mutex.? That was enough for the stress test >> (gc/stress/TestStressIHOPMultiThread.java) to hit the problem >> occasionally. >> >> This problem would not have shown up before JDK-8210832, which was >> made shortly before the first sighting of the stress test failure. >> Before that change we re-used park events in the implementation of >> HotSpot Mutex/Monitor.? Because of that re-use, there wasn't an >> opportunity to allocate a pthread_mutex_t at the same address as a >> former pthread_cond_t. >> >> We work around this problem by allocating from a freelist the >> mutex/condvar pairs needed by the macOSX PlatformMonitor. >> >> An alternative workaround that was explored is to (for macOSX only) >> add a short timedwait on the condvar when deleting a PlatformMonitor. >> The idea is that (1) if there is a lingering kernel table entry for the >> condvar, the timedwait will eat it, and (2) if there isn't such an >> entry the timedwait will (almost) immediately expire.? This would use >> pthread_cond_timedwait_relative_np to avoid unneeded clock accesses. >> This approach isn't being taken because it might be sensitive to >> implementation details that could vary between OS versions. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8218975 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8218975/open.01/ > > src/hotspot/os/posix/os_posix.hpp > ??? Good job on making the freelist work around only apply > ??? to MacOSX. > > src/hotspot/os/posix/os_posix.cpp > ??? No comments. > > src/hotspot/os/posix/os_posix.inline.hpp > ??? No comments. > > I have no objection to your use of the '_ptr' suffix. > > Thumbs up. > > Dan > > > >> >> Testing: >> mach5 tier1-5 >> mach5 the specific test, 2000 times with no failures.? Without this >> ?? change, typical failure rate seems to be on the order of 0.5-1.0%. >> Performance testing on Mac found no regressions. >> > > From david.holmes at oracle.com Mon Mar 18 21:24:14 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Mar 2019 07:24:14 +1000 Subject: RFR [XS] : 8220781: linux-s390 : os::get_summary_cpu_info gives bad output In-Reply-To: References: Message-ID: <87a2d6c3-5552-f592-5993-364350fa107a@oracle.com> Hi Matthias, On 19/03/2019 1:41 am, Baesken, Matthias wrote: > Hello, please review the following small fix. > Issue : currently the Host: information line in the hs_err file looks strange on linux s390x . > Here is an example : > > Host: s : 6, 6 cores, 11G, SUSE Linux Enterprise Server 11 (s390x) > > The strange output after Host: ("s : 6") is caused by os::get_summary_cpu_info ; it scans for the string "processor" and runs into the second line of the file : > > more /proc/cpuinfo > vendor_id : IBM/S390 > # processors : 6 > > This leads to "s : 6" . > Instead we can get the "machine" ID -number which is available in the /proc/cpuinfo file . This can be used to look up the machine+related processor model . That seems fine. You're in a better position to assess the correctness of this. > Another question I had when looking into the issue - why is the hostname after "Host:" only printed in the non-product case ? > It could be also helpful in the product build as well . The host name is considered "sensitive" information so is not exposed in product builds. It's provided in non-product builds to aid in test failure diagnosis. Thanks, David > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8220781 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8220781.0/ > > > > Thanks, Matthias > > From david.holmes at oracle.com Mon Mar 18 21:30:26 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Mar 2019 07:30:26 +1000 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <8e8b296b-18e4-663e-b098-dadb6dce6f90@oracle.com> Message-ID: On 19/03/2019 2:53 am, Daniel D. Daugherty wrote: > I forgot to ask the meta question about how we are going to decide when > this freelist work around can be retired so that MacOSX has the same > code as the other platforms? File a bug now: "Reevaluate Mac OS pthread bug workaround" and target to JDK 23. :) David > Dan > > > On 3/18/19 12:50 PM, Daniel D. Daugherty wrote: >> On 3/15/19 7:03 PM, Kim Barrett wrote: >>> Please review this fix for intermittent Mac-only crashes involving the >>> OWSTTaskTerminator. >>> >>> This is joint work with Patricio Chilano Mateo. >> >> Very impressive bug hunt! >> >> >>> This turned out to be a bug in the macOSX kernel's pthread support >>> (which may be fixed in OSX 10.14 (Mojave)). When using a condvar, an >>> associated broadcast entry may be added to a kernel table and left >>> there after the last use of the condvar. If we then destroy and delete >>> the condvar and later happen to allocate and init a mutex at the same >>> address (this really does happen), that stale condvar entry in the >>> kernel table confuses operations on the mutex, causing crashes and >>> hangs. >>> >>> We have been able to reproduce those failure modes with standalone >>> programs (no JDK code involved) (see the CR for reproducers), and have >>> reported the problem to Apple.? This reproducer didn't fail when >>> tested on Mojave, but did fail on all previous OS versions we tried. >>> >>> The OWSTTaskTerminator is subject to this problem because a new >>> HotSpot Monitor object is allocated for each terminator. Terminators >>> are allocated for various parallel tasks during garbage collection, so >>> there are quite a few Monitor objects (and their contained >>> PlatformMonitors) being created and deleted, so many opportunities to >>> get the kernel into the bad state and then later reuse a previous >>> condvar address for a mutex.? That was enough for the stress test >>> (gc/stress/TestStressIHOPMultiThread.java) to hit the problem >>> occasionally. >>> >>> This problem would not have shown up before JDK-8210832, which was >>> made shortly before the first sighting of the stress test failure. >>> Before that change we re-used park events in the implementation of >>> HotSpot Mutex/Monitor.? Because of that re-use, there wasn't an >>> opportunity to allocate a pthread_mutex_t at the same address as a >>> former pthread_cond_t. >>> >>> We work around this problem by allocating from a freelist the >>> mutex/condvar pairs needed by the macOSX PlatformMonitor. >>> >>> An alternative workaround that was explored is to (for macOSX only) >>> add a short timedwait on the condvar when deleting a PlatformMonitor. >>> The idea is that (1) if there is a lingering kernel table entry for the >>> condvar, the timedwait will eat it, and (2) if there isn't such an >>> entry the timedwait will (almost) immediately expire.? This would use >>> pthread_cond_timedwait_relative_np to avoid unneeded clock accesses. >>> This approach isn't being taken because it might be sensitive to >>> implementation details that could vary between OS versions. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8218975 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8218975/open.01/ >> >> src/hotspot/os/posix/os_posix.hpp >> ??? Good job on making the freelist work around only apply >> ??? to MacOSX. >> >> src/hotspot/os/posix/os_posix.cpp >> ??? No comments. >> >> src/hotspot/os/posix/os_posix.inline.hpp >> ??? No comments. >> >> I have no objection to your use of the '_ptr' suffix. >> >> Thumbs up. >> >> Dan >> >> >> >>> >>> Testing: >>> mach5 tier1-5 >>> mach5 the specific test, 2000 times with no failures.? Without this >>> ?? change, typical failure rate seems to be on the order of 0.5-1.0%. >>> Performance testing on Mac found no regressions. >>> >> >> > From david.holmes at oracle.com Mon Mar 18 21:43:55 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Mar 2019 07:43:55 +1000 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> Message-ID: <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> Hi Jie, On 14/03/2019 8:00 pm, Jie Fu wrote: > Hi all, > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 > Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ This seems a reasonable improvement, but I have a couple of suggestions. To avoid so much duplication you can change this structure: if (i > 0) { st->print("cpu_quota: %d\n", i); } else { st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); } to this: st->print("cpu_quota: "); if (i > 0) { st->print("%d\n", i); } else { st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); } Also I suggest replacing "failed" with "not supported". Finally one minor typo: s/"no share"/"no shares"/ Thanks, David ----- > In the VM's error log, the container information was dumpped as: > --------------------------------- > container (cgroup) information: > container_type: cgroupv1 > cpu_cpuset_cpus: 0-11 > cpu_memory_nodes: 0 > active_processor_count: 12 > cpu_quota: -1 > cpu_period: 100000 > cpu_shares: -1 > memory_limit_in_bytes: -1 > memory_and_swap_limit_in_bytes: -2 > memory_soft_limit_in_bytes: -1 > memory_usage_in_bytes: 11690164224 > memory_max_usage_in_bytes: 1806413824 > --------------------------------- > > Items such as "memory_limit_in_bytes" and > "memory_and_swap_limit_in_bytes" are difficult to understand. > > It might be more human-readable if we could dump it like this: > --------------------------------- > container (cgroup) information: > container_type: cgroupv1 > cpu_cpuset_cpus: 0-11 > cpu_memory_nodes: 0 > active_processor_count: 12 > cpu_quota: no quota > cpu_period: 100000 > cpu_shares: no share > memory_limit_in_bytes: unlimited > memory_and_swap_limit_in_bytes: failed > memory_soft_limit_in_bytes: unlimited > memory_usage_in_bytes: 11690164224 > memory_max_usage_in_bytes: 1806413824 > --------------------------------- > > I made this patch to improve the readability of container information in > the error log. > The change has been tested with hotspot:tier1 on Linux/x86-64. > > Could you please review it? > Thanks a lot. > > Best regards, > Jie > > From kim.barrett at oracle.com Mon Mar 18 23:04:47 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 18 Mar 2019 19:04:47 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <5ff8d1fc-8a3f-c270-6766-b38c40c1645a@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <5ff8d1fc-8a3f-c270-6766-b38c40c1645a@oracle.com> Message-ID: > On Mar 17, 2019, at 9:43 PM, David Holmes wrote: > > Hi Kim and Patricio > > On 16/03/2019 9:03 am, Kim Barrett wrote: >> Please review this fix for intermittent Mac-only crashes involving the >> OWSTTaskTerminator. >> This is joint work with Patricio Chilano Mateo. > > Fantastic job in pinning this one down! For the record, Patricio did the heavy lifting on getting to the bottom of this, including building and running patched versions of the macOSX kernel and pthread library, developing the standalone reproducer, and analyzing its behavior. > The fix generally looks good though I agree with Thomas that X_ptr() should just be X(). OK, _ptr suffix dropped. In my defence, that convention is used by the C++ standard, both in standard APIs and in examples. Though following "standard" conventions would probably use references in a lot of places where HotSpot code uses pointers. > There is one minor issue: > > + pthread_mutex_t os::PlatformMonitor::_freelist_lock = PTHREAD_MUTEX_INITIALIZER; > > on FreeBSD the default mutex initialization doesn't use PTHREAD_MUTEX_NORMAL and so we get a slow debug version IIRC. That's why we explicitly use non-static pthread_mutex initialization using a pthread_mutex_attr. You should be able to move the _freelist_lock initialization to pthread_init_common() with the PLATFORM_MONITOR_IMPL_INDIRECT guard. The workaround isn't in FreeBSD code, it is macOSX-specific, where the implemented default is "normal" rather than "errorcheck". But the documentation seems to be vague on that (as with FreeBSD and POSIX documentation, but not Linux, where the defaults are spelled out). Since the cost of explicitly ensuring we use "normal" mutexes is small, let's be paranoid and do so. New webrevs: full: http://cr.openjdk.java.net/~kbarrett/8218975/open.02/ incr: http://cr.openjdk.java.net/~kbarrett/8218975/open.02.inc/ From kim.barrett at oracle.com Mon Mar 18 23:08:16 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 18 Mar 2019 19:08:16 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <8e8b296b-18e4-663e-b098-dadb6dce6f90@oracle.com> Message-ID: <486D8332-7362-41F2-B2C3-99A884AE78E9@oracle.com> > On Mar 18, 2019, at 5:30 PM, David Holmes wrote: > > On 19/03/2019 2:53 am, Daniel D. Daugherty wrote: >> I forgot to ask the meta question about how we are going to decide when >> this freelist work around can be retired so that MacOSX has the same >> code as the other platforms? > > File a bug now: "Reevaluate Mac OS pthread bug workaround" and target to JDK 23. :) That?s what I was planning to do, though with target == ?tbd?. JDK 23 is only five-ish years away :) From kim.barrett at oracle.com Mon Mar 18 23:17:58 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 18 Mar 2019 19:17:58 -0400 Subject: RFR: 8188066: (ref) Examine the reachability of JNI WeakGlobalRef and interaction with phantom refs In-Reply-To: <4a589284-4e4d-5d4c-7441-6d00a9725fe3@oracle.com> References: <97572862-2C28-4ED7-ACEA-E23C6A148526@oracle.com> <6F6648D3-2536-4047-B73B-0A0411960F49@oracle.com> <4a589284-4e4d-5d4c-7441-6d00a9725fe3@oracle.com> Message-ID: <191DEA12-EFCB-4A5F-80BE-4B7F95DB22C7@oracle.com> > On Mar 14, 2019, at 12:40 AM, David Holmes wrote: > > Looks good. > > Thanks Kim! > > David Thanks. From david.holmes at oracle.com Mon Mar 18 23:19:52 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Mar 2019 09:19:52 +1000 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <5ff8d1fc-8a3f-c270-6766-b38c40c1645a@oracle.com> Message-ID: <3534f753-7add-86b9-ac70-2d730e629c36@oracle.com> Hi Kim, On 19/03/2019 9:04 am, Kim Barrett wrote: >> On Mar 17, 2019, at 9:43 PM, David Holmes wrote: >> >> Hi Kim and Patricio >> >> On 16/03/2019 9:03 am, Kim Barrett wrote: >>> Please review this fix for intermittent Mac-only crashes involving the >>> OWSTTaskTerminator. >>> This is joint work with Patricio Chilano Mateo. >> >> Fantastic job in pinning this one down! > > For the record, Patricio did the heavy lifting on getting to the > bottom of this, including building and running patched versions of the > macOSX kernel and pthread library, developing the standalone reproducer, > and analyzing its behavior. > >> The fix generally looks good though I agree with Thomas that X_ptr() should just be X(). > > OK, _ptr suffix dropped. In my defence, that convention is used by > the C++ standard, both in standard APIs and in examples. Though > following "standard" conventions would probably use references in a > lot of places where HotSpot code uses pointers. > >> There is one minor issue: >> >> + pthread_mutex_t os::PlatformMonitor::_freelist_lock = PTHREAD_MUTEX_INITIALIZER; >> >> on FreeBSD the default mutex initialization doesn't use PTHREAD_MUTEX_NORMAL and so we get a slow debug version IIRC. That's why we explicitly use non-static pthread_mutex initialization using a pthread_mutex_attr. You should be able to move the _freelist_lock initialization to pthread_init_common() with the PLATFORM_MONITOR_IMPL_INDIRECT guard. > > The workaround isn't in FreeBSD code, it is macOSX-specific, where the Yes my bad! Was thinking too broadly. > implemented default is "normal" rather than "errorcheck". But the > documentation seems to be vague on that (as with FreeBSD and POSIX > documentation, but not Linux, where the defaults are spelled out). > Since the cost of explicitly ensuring we use "normal" mutexes is > small, let's be paranoid and do so. Okay. That's consistent with the existing unconditional dynamic initialization of mutexes etc. > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8218975/open.02/ > incr: http://cr.openjdk.java.net/~kbarrett/8218975/open.02.inc/ Looks good! Thanks, David From david.holmes at oracle.com Mon Mar 18 23:20:53 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Mar 2019 09:20:53 +1000 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <486D8332-7362-41F2-B2C3-99A884AE78E9@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <8e8b296b-18e4-663e-b098-dadb6dce6f90@oracle.com> <486D8332-7362-41F2-B2C3-99A884AE78E9@oracle.com> Message-ID: On 19/03/2019 9:08 am, Kim Barrett wrote: >> On Mar 18, 2019, at 5:30 PM, David Holmes wrote: >> >> On 19/03/2019 2:53 am, Daniel D. Daugherty wrote: >>> I forgot to ask the meta question about how we are going to decide when >>> this freelist work around can be retired so that MacOSX has the same >>> code as the other platforms? >> >> File a bug now: "Reevaluate Mac OS pthread bug workaround" and target to JDK 23. :) > > That?s what I was planning to do, though with target == ?tbd?. JDK 23 is only five-ish years away :) I thought 5 years a good time to re-examine things :) tbd's might get overlooked for 10-20 years :) Cheers, David From kim.barrett at oracle.com Mon Mar 18 23:21:26 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 18 Mar 2019 19:21:26 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: <3534f753-7add-86b9-ac70-2d730e629c36@oracle.com> References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <5ff8d1fc-8a3f-c270-6766-b38c40c1645a@oracle.com> <3534f753-7add-86b9-ac70-2d730e629c36@oracle.com> Message-ID: <1B30BB9F-A925-443A-9DB3-384BB6250DC7@oracle.com> > On Mar 18, 2019, at 7:19 PM, David Holmes wrote: > > Hi Kim, > > On 19/03/2019 9:04 am, Kim Barrett wrote: > [?] >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8218975/open.02/ >> incr: http://cr.openjdk.java.net/~kbarrett/8218975/open.02.inc/ > > Looks good! > > Thanks, > David Thanks. From ysr1729 at gmail.com Mon Mar 18 23:37:20 2019 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Mon, 18 Mar 2019 16:37:20 -0700 Subject: Unused member field/dead code in hotspot/logging/log.hpp? Message-ID: Hello all, I was making some modifications to unified logging for optionally making it async -- basically porting some of our JDK 8 code changes over to OpenJDK 11's Unified Logging framework, and I noticed that in log.hpp there appears to be an unused static member in class LogImpl: class LogImpl { private: static const size_t LogBufferSize = 512; It doesn't seem to be used anywhere, at least in the OpenJDK 11 vintage I am looking at. Is it used elsewhere in code not visible in OpenJDK, or can it be deleted? (PS: please let me know if this question shd go to some other more specific email list.) thanks, -- ramki // TwitterVMTeam // openjdk: ysr From daniel.daugherty at oracle.com Mon Mar 18 23:58:14 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 18 Mar 2019 19:58:14 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <5ff8d1fc-8a3f-c270-6766-b38c40c1645a@oracle.com> Message-ID: On 3/18/19 7:04 PM, Kim Barrett wrote: >> On Mar 17, 2019, at 9:43 PM, David Holmes wrote: >> >> Hi Kim and Patricio >> >> On 16/03/2019 9:03 am, Kim Barrett wrote: >>> Please review this fix for intermittent Mac-only crashes involving the >>> OWSTTaskTerminator. >>> This is joint work with Patricio Chilano Mateo. >> Fantastic job in pinning this one down! > For the record, Patricio did the heavy lifting on getting to the > bottom of this, including building and running patched versions of the > macOSX kernel and pthread library, developing the standalone reproducer, > and analyzing its behavior. > >> The fix generally looks good though I agree with Thomas that X_ptr() should just be X(). > OK, _ptr suffix dropped. In my defence, that convention is used by > the C++ standard, both in standard APIs and in examples. Though > following "standard" conventions would probably use references in a > lot of places where HotSpot code uses pointers. > >> There is one minor issue: >> >> + pthread_mutex_t os::PlatformMonitor::_freelist_lock = PTHREAD_MUTEX_INITIALIZER; >> >> on FreeBSD the default mutex initialization doesn't use PTHREAD_MUTEX_NORMAL and so we get a slow debug version IIRC. That's why we explicitly use non-static pthread_mutex initialization using a pthread_mutex_attr. You should be able to move the _freelist_lock initialization to pthread_init_common() with the PLATFORM_MONITOR_IMPL_INDIRECT guard. > The workaround isn't in FreeBSD code, it is macOSX-specific, where the > implemented default is "normal" rather than "errorcheck". But the > documentation seems to be vague on that (as with FreeBSD and POSIX > documentation, but not Linux, where the defaults are spelled out). > Since the cost of explicitly ensuring we use "normal" mutexes is > small, let's be paranoid and do so. > > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8218975/open.02/ > incr: http://cr.openjdk.java.net/~kbarrett/8218975/open.02.inc/ > src/hotspot/os/posix/os_posix.cpp ??? No comments. src/hotspot/os/posix/os_posix.hpp ??? No comments. src/hotspot/os/posix/os_posix.inline.hpp ??? No comments. Thumbs up. Dan From david.holmes at oracle.com Tue Mar 19 00:16:10 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Mar 2019 10:16:10 +1000 Subject: Unused member field/dead code in hotspot/logging/log.hpp? In-Reply-To: References: Message-ID: <129f6fea-487e-eac6-f2c3-9f65ba168e37@oracle.com> Hi Ramki, On 19/03/2019 9:37 am, Srinivas Ramakrishna wrote: > Hello all, > > I was making some modifications to unified logging for optionally making it > async -- basically porting some of our JDK 8 > code changes over to OpenJDK 11's Unified Logging framework, and I noticed > that in log.hpp there appears to be an unused > static member in class LogImpl: > > class LogImpl { > private: > static const size_t LogBufferSize = 512; > > It doesn't seem to be used anywhere, at least in the OpenJDK 11 vintage I > am looking at. > > Is it used elsewhere in code not visible in OpenJDK, or can it be deleted? Looks dead to me. Hard to track if it may have been used in the past. > (PS: please let me know if this question shd go to some other more specific > email list.) hotspot-runtime-dev would probably be more specific, but hotspot-dev is fine. Cheers, David > thanks, > -- ramki > // TwitterVMTeam // openjdk: ysr > From kim.barrett at oracle.com Tue Mar 19 00:25:43 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 18 Mar 2019 20:25:43 -0400 Subject: RFR: 8218975: Bug in macOSX kernel's pthread support In-Reply-To: References: <161613CC-739F-4C65-8D0D-3D249F70D848@oracle.com> <5ff8d1fc-8a3f-c270-6766-b38c40c1645a@oracle.com> Message-ID: > On Mar 18, 2019, at 7:58 PM, Daniel D. Daugherty wrote: > > On 3/18/19 7:04 PM, Kim Barrett wrote: >> [?] >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8218975/open.02/ >> incr: http://cr.openjdk.java.net/~kbarrett/8218975/open.02.inc/ >> > > src/hotspot/os/posix/os_posix.cpp > No comments. > > src/hotspot/os/posix/os_posix.hpp > No comments. > > src/hotspot/os/posix/os_posix.inline.hpp > No comments. > > Thumbs up. > > Dan Thanks. From ysr1729 at gmail.com Tue Mar 19 00:26:41 2019 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Mon, 18 Mar 2019 17:26:41 -0700 Subject: Unused member field/dead code in hotspot/logging/log.hpp? In-Reply-To: <129f6fea-487e-eac6-f2c3-9f65ba168e37@oracle.com> References: <129f6fea-487e-eac6-f2c3-9f65ba168e37@oracle.com> Message-ID: Thanks for the prompt response/confirmation/ifnormation, David! -- ramki On Mon, Mar 18, 2019 at 5:16 PM David Holmes wrote: > Hi Ramki, > > On 19/03/2019 9:37 am, Srinivas Ramakrishna wrote: > > Hello all, > > > > I was making some modifications to unified logging for optionally making > it > > async -- basically porting some of our JDK 8 > > code changes over to OpenJDK 11's Unified Logging framework, and I > noticed > > that in log.hpp there appears to be an unused > > static member in class LogImpl: > > > > class LogImpl { > > private: > > static const size_t LogBufferSize = 512; > > > > It doesn't seem to be used anywhere, at least in the OpenJDK 11 vintage I > > am looking at. > > > > Is it used elsewhere in code not visible in OpenJDK, or can it be > deleted? > > Looks dead to me. Hard to track if it may have been used in the past. > > > (PS: please let me know if this question shd go to some other more > specific > > email list.) > > hotspot-runtime-dev would probably be more specific, but hotspot-dev is > fine. > > Cheers, > David > > > thanks, > > -- ramki > > // TwitterVMTeam // openjdk: ysr > > > From fujie at loongson.cn Tue Mar 19 07:49:29 2019 From: fujie at loongson.cn (Jie Fu) Date: Tue, 19 Mar 2019 15:49:29 +0800 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> Message-ID: <82980f85-0343-d783-baf3-0dd5a239b696@loongson.cn> Hi David, Thank you so much for your review and suggestions. Please review the second version: http://cr.openjdk.java.net/~jiefu/8220658/webrev.01/ Thanks a lot. Best regards, Jie On 2019/3/19 ??5:43, David Holmes wrote: > Hi Jie, > > On 14/03/2019 8:00 pm, Jie Fu wrote: >> Hi all, >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 >> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ > > This seems a reasonable improvement, but I have a couple of > suggestions. To avoid so much duplication you can change this structure: > > ?? if (i > 0) { > ???? st->print("cpu_quota: %d\n", i); > ?? } else { > ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" : > "no quota"); > ?? } > > to this: > > ?? st->print("cpu_quota: "); > ?? if (i > 0) { > ????? st->print("%d\n", i); > ?? } else { > ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); > ?? } > > Also I suggest replacing "failed" with "not supported". > > Finally one minor typo: s/"no share"/"no shares"/ > > Thanks, > David > ----- > >> In the VM's error log, the container information was dumpped as: >> --------------------------------- >> container (cgroup) information: >> container_type: cgroupv1 >> cpu_cpuset_cpus: 0-11 >> cpu_memory_nodes: 0 >> active_processor_count: 12 >> cpu_quota: -1 >> cpu_period: 100000 >> cpu_shares: -1 >> memory_limit_in_bytes: -1 >> memory_and_swap_limit_in_bytes: -2 >> memory_soft_limit_in_bytes: -1 >> memory_usage_in_bytes: 11690164224 >> memory_max_usage_in_bytes: 1806413824 >> --------------------------------- >> >> Items such as "memory_limit_in_bytes" and >> "memory_and_swap_limit_in_bytes" are difficult to understand. >> >> It might be more human-readable if we could dump it like this: >> --------------------------------- >> container (cgroup) information: >> container_type: cgroupv1 >> cpu_cpuset_cpus: 0-11 >> cpu_memory_nodes: 0 >> active_processor_count: 12 >> cpu_quota: no quota >> cpu_period: 100000 >> cpu_shares: no share >> memory_limit_in_bytes: unlimited >> memory_and_swap_limit_in_bytes: failed >> memory_soft_limit_in_bytes: unlimited >> memory_usage_in_bytes: 11690164224 >> memory_max_usage_in_bytes: 1806413824 >> --------------------------------- >> >> I made this patch to improve the readability of container information >> in the error log. >> The change has been tested with hotspot:tier1 on Linux/x86-64. >> >> Could you please review it? >> Thanks a lot. >> >> Best regards, >> Jie >> >> From stefan.karlsson at oracle.com Tue Mar 19 07:56:34 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 19 Mar 2019 08:56:34 +0100 Subject: Unused member field/dead code in hotspot/logging/log.hpp? In-Reply-To: References: Message-ID: <7072dad1-8a5c-d62a-392b-d0e8d3f7433c@oracle.com> Hi Ramki, Searching for LogBufferSize in the output of: hg log --follow -p src/hotspot/share/logging/log.hpp shows that its usage was removed in: changeset: 37247:bdbfe42632cb user: rehn date: Mon Apr 04 15:05:52 2016 +0200 summary: 8153254: Delegate (v)write from Log to LogTagSet You could also use hg grep, but I think that often takes longer time to perform: hg grep --all --follow "LogBufferSize" src/hotspot/share/logging/log.hpp src/hotspot/share/logging/log.hpp:47216:+: static const size_t LogBufferSize = 512; hotspot/src/share/vm/logging/log.hpp:37247:-: char buf[LogBufferSize]; hotspot/src/share/vm/logging/log.hpp:33097:+: static const size_t LogBufferSize = 512; hotspot/src/share/vm/logging/log.hpp:33097:+: char buf[LogBufferSize]; Cheers, StefanK On 2019-03-19 00:37, Srinivas Ramakrishna wrote: > Hello all, > > I was making some modifications to unified logging for optionally making it > async -- basically porting some of our JDK 8 > code changes over to OpenJDK 11's Unified Logging framework, and I noticed > that in log.hpp there appears to be an unused > static member in class LogImpl: > > class LogImpl { > private: > static const size_t LogBufferSize = 512; > > It doesn't seem to be used anywhere, at least in the OpenJDK 11 vintage I > am looking at. > > Is it used elsewhere in code not visible in OpenJDK, or can it be deleted? > > (PS: please let me know if this question shd go to some other more specific > email list.) > > thanks, > -- ramki > // TwitterVMTeam // openjdk: ysr > From shade at redhat.com Tue Mar 19 08:41:24 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 19 Mar 2019 09:41:24 +0100 Subject: RFR(XS): 8220690: ATTRIBUTE_ALIGNED requires GNU extensions enabled In-Reply-To: <2879CC12-CC8C-4ECA-9D7C-0D60F8AC8263@oracle.com> References: <2879CC12-CC8C-4ECA-9D7C-0D60F8AC8263@oracle.com> Message-ID: <7363200f-2709-61cf-a0e1-73c3dd7cba71@redhat.com> On 3/15/19 11:18 PM, Kim Barrett wrote: > Webrev: > http://cr.openjdk.java.net/~kbarrett/8220690/open.00/ I am fine with it. Severin, can you test jdk/jdk still builds on old RHEL with this patch? -Aleksey From ysr1729 at gmail.com Tue Mar 19 09:05:35 2019 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Tue, 19 Mar 2019 02:05:35 -0700 Subject: Leakage of build state from one flavor of build to another? Message-ID: I am still working with OpenJDK 11, so I am not sure if these comments apply to more recent vintages of, or the tip of OpenJDK. I noticed recently that if one had built a fastdebug (or release) image of the JDK and then built a slowdebug image,then it turns out that the slowdebug image fails with a bootstrap error such as the following: $repo/build/macosx-x86_64-normal-server-slowdebug/images/jdk/bin/java -version Error occurred during initialization of boot layer java.lang.LayerInstantiationException: Package sun.util in both module java.base and module jdk.localedata The precise details of the exception might vary a bit. A clean rebuild of slowdebug -- after cleaning out all vestiges of other flavors of builds -- seems to work around the problem. If one builds slowdebug, before the other builds then there isn't an issue. I have noticed this only on the mac, my primary dev platform these days, but will see if this also happens on Linux as well, and update. Not having looked in any detail at the build process, I was wondering if anyone had encountered this kind of error during development and if this is a known issue. thanks, -- ramki // TwitterVMTeam // openjdk: ysr From ysr1729 at gmail.com Tue Mar 19 09:07:39 2019 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Tue, 19 Mar 2019 02:07:39 -0700 Subject: Unused member field/dead code in hotspot/logging/log.hpp? In-Reply-To: <7072dad1-8a5c-d62a-392b-d0e8d3f7433c@oracle.com> References: <7072dad1-8a5c-d62a-392b-d0e8d3f7433c@oracle.com> Message-ID: Ah, nice, thanks! On Tue, Mar 19, 2019 at 12:56 AM Stefan Karlsson wrote: > Hi Ramki, > > Searching for LogBufferSize in the output of: > hg log --follow -p src/hotspot/share/logging/log.hpp > > shows that its usage was removed in: > changeset: 37247:bdbfe42632cb > user: rehn > date: Mon Apr 04 15:05:52 2016 +0200 > summary: 8153254: Delegate (v)write from Log to LogTagSet > > You could also use hg grep, but I think that often takes longer time to > perform: > hg grep --all --follow "LogBufferSize" src/hotspot/share/logging/log.hpp > src/hotspot/share/logging/log.hpp:47216:+: static const size_t > LogBufferSize = 512; > hotspot/src/share/vm/logging/log.hpp:37247:-: char buf[LogBufferSize]; > hotspot/src/share/vm/logging/log.hpp:33097:+: static const size_t > LogBufferSize = 512; > hotspot/src/share/vm/logging/log.hpp:33097:+: char buf[LogBufferSize]; > > Cheers, > StefanK > > On 2019-03-19 00:37, Srinivas Ramakrishna wrote: > > Hello all, > > > > I was making some modifications to unified logging for optionally making > it > > async -- basically porting some of our JDK 8 > > code changes over to OpenJDK 11's Unified Logging framework, and I > noticed > > that in log.hpp there appears to be an unused > > static member in class LogImpl: > > > > class LogImpl { > > private: > > static const size_t LogBufferSize = 512; > > > > It doesn't seem to be used anywhere, at least in the OpenJDK 11 vintage I > > am looking at. > > > > Is it used elsewhere in code not visible in OpenJDK, or can it be > deleted? > > > > (PS: please let me know if this question shd go to some other more > specific > > email list.) > > > > thanks, > > -- ramki > > // TwitterVMTeam // openjdk: ysr > > > From sgehwolf at redhat.com Tue Mar 19 09:23:42 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 19 Mar 2019 10:23:42 +0100 Subject: RFR(XS): 8220690: ATTRIBUTE_ALIGNED requires GNU extensions enabled In-Reply-To: <7363200f-2709-61cf-a0e1-73c3dd7cba71@redhat.com> References: <2879CC12-CC8C-4ECA-9D7C-0D60F8AC8263@oracle.com> <7363200f-2709-61cf-a0e1-73c3dd7cba71@redhat.com> Message-ID: <5ee9bdc759e3a057ed7c2222fb3d7c74b0e73443.camel@redhat.com> On Tue, 2019-03-19 at 09:41 +0100, Aleksey Shipilev wrote: > On 3/15/19 11:18 PM, Kim Barrett wrote: > > Webrev: > > http://cr.openjdk.java.net/~kbarrett/8220690/open.00/ > > I am fine with it. +1. I've added a comment to the bug as I've see that prior to the RFR. Sorry about this. > Severin, can you test jdk/jdk still builds on old RHEL with this patch? Sure. It'll take a bit. godbolt.org tells me that it should work for GCC 4.4.7, though. I'll reply once I have results. Thanks, Severin From lutz.schmidt at sap.com Tue Mar 19 10:35:02 2019 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 19 Mar 2019 10:35:02 +0000 Subject: RFR [XS] : 8220781: linux-s390 : os::get_summary_cpu_info gives bad output In-Reply-To: References: Message-ID: <4A8995A0-2656-46C7-AE6D-F836AD255938@sap.com> Hi Matthias, you change looks good (and trivial). Please note: I?m not a reviewer! Too bad we can?t print the hostname in PRODUCT builds. It has proven helpful when analyzing multiple hs_err* files (from distributed systems) in parallel. Thanks, Lutz From: "Baesken, Matthias" Date: Monday, 18. March 2019 at 16:41 To: "hotspot-dev at openjdk.java.net" Cc: Lutz Schmidt Subject: RFR [XS] : 8220781: linux-s390 : os::get_summary_cpu_info gives bad output Hello, please review the following small fix. Issue : currently the Host: information line in the hs_err file looks strange on linux s390x . Here is an example : Host: s : 6, 6 cores, 11G, SUSE Linux Enterprise Server 11 (s390x) The strange output after Host: (?s : 6?) is caused by os::get_summary_cpu_info ; it scans for the string ?processor? and runs into the second line of the file : more /proc/cpuinfo vendor_id : IBM/S390 # processors : 6 This leads to ?s : 6? . Instead we can get the ?machine? ID -number which is available in the /proc/cpuinfo file . This can be used to look up the machine+related processor model . Another question I had when looking into the issue - why is the hostname after ?Host:? only printed in the non-product case ? It could be also helpful in the product build as well . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8220781 http://cr.openjdk.java.net/~mbaesken/webrevs/8220781.0/ Thanks, Matthias From matthias.baesken at sap.com Tue Mar 19 11:51:15 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 19 Mar 2019 11:51:15 +0000 Subject: RFR [XS] : 8220781: linux-s390 : os::get_summary_cpu_info gives bad output In-Reply-To: <4A8995A0-2656-46C7-AE6D-F836AD255938@sap.com> References: <4A8995A0-2656-46C7-AE6D-F836AD255938@sap.com> Message-ID: David + Lucy , thanks for the reviews ! From: Schmidt, Lutz Sent: Dienstag, 19. M?rz 2019 11:35 To: Baesken, Matthias ; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR [XS] : 8220781: linux-s390 : os::get_summary_cpu_info gives bad output Hi Matthias, you change looks good (and trivial). Please note: I?m not a reviewer! Too bad we can?t print the hostname in PRODUCT builds. It has proven helpful when analyzing multiple hs_err* files (from distributed systems) in parallel. Thanks, Lutz From thomas.stuefe at gmail.com Tue Mar 19 11:59:32 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 19 Mar 2019 12:59:32 +0100 Subject: RFR [XS] : 8220781: linux-s390 : os::get_summary_cpu_info gives bad output In-Reply-To: References: Message-ID: +1 ..thomas On Mon, Mar 18, 2019 at 4:42 PM Baesken, Matthias wrote: > Hello, please review the following small fix. > Issue : currently the Host: information line in the hs_err file > looks strange on linux s390x . > Here is an example : > > Host: s : 6, 6 cores, 11G, SUSE Linux Enterprise Server 11 (s390x) > > The strange output after Host: ("s : 6") is caused by > os::get_summary_cpu_info ; it scans for the string "processor" and runs > into the second line of the file : > > more /proc/cpuinfo > vendor_id : IBM/S390 > # processors : 6 > > This leads to "s : 6" . > Instead we can get the "machine" ID -number which is available in > the /proc/cpuinfo file . This can be used to look up the machine+related > processor model . > > Another question I had when looking into the issue - why is the > hostname after "Host:" only printed in the non-product case ? > It could be also helpful in the product build as well . > > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8220781 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8220781.0/ > > > > Thanks, Matthias > > > From sgehwolf at redhat.com Tue Mar 19 12:28:41 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 19 Mar 2019 13:28:41 +0100 Subject: RFR(XS): 8220690: ATTRIBUTE_ALIGNED requires GNU extensions enabled In-Reply-To: <5ee9bdc759e3a057ed7c2222fb3d7c74b0e73443.camel@redhat.com> References: <2879CC12-CC8C-4ECA-9D7C-0D60F8AC8263@oracle.com> <7363200f-2709-61cf-a0e1-73c3dd7cba71@redhat.com> <5ee9bdc759e3a057ed7c2222fb3d7c74b0e73443.camel@redhat.com> Message-ID: <65a23b290a0572b2e261d0849602bbeeb8d9a015.camel@redhat.com> On Tue, 2019-03-19 at 10:23 +0100, Severin Gehwolf wrote: > > Sure. It'll take a bit. godbolt.org tells me that it should work for > GCC 4.4.7, though. I'll reply once I have results. Results are in. Good to go. Thanks, Severin From matthias.baesken at sap.com Tue Mar 19 13:53:39 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 19 Mar 2019 13:53:39 +0000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> Message-ID: Hello, here is a new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ This moved the ppc/s390 virtualization related info output into the platform/cpu - specific coding . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Montag, 11. M?rz 2019 08:36 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Cc: Doerr, Martin > Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > hs_error file on linux/windows x86_64 > > On 7/03/2019 3:13 am, Baesken, Matthias wrote: > > Hello , could I push the latest version of 8219241 ? > > Sorry based on your earlier email I thought you were looking at doing > the restructuring. I was then on vacation most of last week. > > > A change that moves the coding more into platform/cpu - specific coding > (if this is wanted) > > could be done when bringing in the AIX virtualization related info in > another patch which I plan to do . > > If that is imminent then okay - I don't like the code as it is. > > Thanks, > David > From martin.doerr at sap.com Tue Mar 19 14:04:05 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 19 Mar 2019 14:04:05 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <440ce735-add2-db60-b672-d929731304cd@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <440ce735-add2-db60-b672-d929731304cd@oracle.com> Message-ID: Hi Robbin, I just tried a small stress test [1] with (+UseNewCode) and without added CPUID instruction in native wrapper [2]. jdk/bin/java -XX:+UnlockDiagnosticVMOptions StressJVMEntries result: 2113929216, duration: 40.1181962 jdk/bin/java -XX:+UnlockDiagnosticVMOptions -XX:+UseNewCode StressJVMEntries result: 2113929216, duration: 76.0759127 Executed on 40 core 2-socket Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz. So I guess nobody wants CPUID on the regular path if it can be avoided. Best regards, Martin [1] StressJVMEntries.java: public class StressJVMEntries{ static long sum = 0; static final int loop_cnt = 10000000; public static void main(String args[]){ Runtime rt = Runtime.getRuntime(); long duration = System.nanoTime(); for (int x=0; x Sent: Freitag, 15. M?rz 2019 13:32 To: David Holmes ; Reingruber, Richard ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> The last point was rather exploring the space of legal solutions than a >> suggestion to solve the issue. I'd favor your approach if you manage to do the >> serialization conditionally (BTW: a mechanism similar to >> JavaThread::*deopt_suspend* >> would be yet another way to do that). FYI: In the long run I'm removing the suspend-flags and turning suspend/deopt/.. into handshakes instead. So we only have the poll to check. I'm testing a change where I removed the changes to the native wrappers, leaving native threads armed (as Martin suggested) and in the poll path notice that we are still armed (when returning from a safepoint/handshake we should be disarmed). Disarm yourself, re-check safepoint/handshake, execute instruction barrier. So we don't need any changes in any native wrapper, we handle the barrier in the polling code instead. I have just one issue to sort-out with that and benchmarking. The v2 patch (not yet published) didn't show to bad of a result when we always emit cpuid, but there is no super JNI heavy test in that suite. /Robbin >> >> Cheers, Richard. >> >> [1] 8.1.3 Handling Self- and Cross-Modifying Code >> ???? in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >> ???? (3A, 3B, 3C & 3D): System Programming Guide >> >> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >> >> ???? on https://software.intel.com/en-us/articles/intel-sdm >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Donnerstag, 14. M?rz 2019 13:24 >> To: Reingruber, Richard ; Doerr, Martin >> ; David Holmes ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> Hi Richard, >> >> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>> Hi Robbin, >>> >>> ??? > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>> ??? > is that it was deemed that a safepoint is 'long enough'. >>> >>> This doesn't seem to be a very good reason. I did a little modification to your >>> initial figure: >> >> That's why we are adding the instruction barrier! >> >>> >>> JavaThread:???????????? | VMThread >>> StoreLoad?????????????? | >>> ????????????????????????? | update >>> ????????????????????????? | ? Global ICache Invalidate ? >>> ????????????????????????? | disarm >>> load thread_poll??????? | >>> >>> Question would be: does an ICache flush happen after the immediate oop update? >>> I would consider it problematic if not. Even if the safepoint was very long. >> >> The icache flush assumes it can stop speculation and flush instruction cache in >> patchee thread. >> Correct me if anything is wrong: >> - sparc needs flush in all patchee threads. >> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >> - ppc needs isync in all patchee threads? >> - s390 needs zarch_sync all in patchee threads? >> - x86 needs cpuid in all in patchee threads. >> >> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >> have no effect in patchee threads!? >> >> Adding a few words on x86 ICache::Invalidate/word. >> We have TSO, so if the patchee thread can see poll is disarmed all other stores >> are visible. I have seen no case ICache::Invalidate/word would make a difference >> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >> The problem is if we prefect an instruction over the load of the poll. >> If poll is disarmed this perfected instruction can be fully executed. >> If that happens to contain an immediate oop, we don't know if it's the patched >> oop or not. >> >> /Robbin >> >>> >>> Cheers Richard. >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>> To: Doerr, Martin ; David Holmes >>> ; hotspot-dev at openjdk.java.net >>> Cc: Reingruber, Richard >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Martin, >>> >>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>> Hi Robbin, >>>> >>>>> But that is whole new subject and let's keep that separate from this :) >>>> Sure, we can keep the modification side discussion separate. This change is >>>> already difficult enough ?? >>>> >>>> >>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>> >>>> orderAccess_aix/linux_ppc.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>> >>>> orderAccess_linux_s390.hpp: >>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>> >>> >>> Thanks! >>> >>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops >>>>> encoded into the instruction stream. >>>>> I couldn't find a call to AbstractICache::invalidate_range. >>> >>> I think I miss-understood you here, the reason I heard for not doing >>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>> safepoint is 'long enough'. >>> >>> /Robbin >>> >>>> >>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>> And one call to AbstractICache/ICache::invalidate_word. >>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>> But that is whole new subject and let's keep that separate from this :) >>>> >>>> I'll create issues for some of these topics, which if nothing else should be >>>> discussed. >>>> >>>> /Robbin >>>> >>>>> >>>>> Does anybody know where it is or if it's missing? >>>>> >>>>> Best regards, >>>>> Martin >>>>> From robbin.ehn at oracle.com Tue Mar 19 14:11:29 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 19 Mar 2019 15:11:29 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <440ce735-add2-db60-b672-d929731304cd@oracle.com> Message-ID: <1bb9da30-4b40-b0ef-4f68-47f6c1b75670@oracle.com> Hi Martin, I'm running the final testing of v2 which includes the lazy disarm, since as you say, is costly. The numbers I'm seeing is (sps = safepoints per second): Baseline 1sps Trans.TotalMemory avgt 2 47.634 ns/op Trans.nativeMethod avgt 2 19.755 ns/op Baseline 100sps Trans.TotalMemory avgt 2 48.099 ns/op Trans.nativeMethod avgt 2 20.342 ns/op Always CPUID 1sps Trans.TotalMemory avgt 2 206.012 ns/op Trans.nativeMethod avgt 2 131.647 ns/op Always CPUID 100sps Trans.TotalMemory avgt 2 211.741 ns/op Trans.nativeMethod avgt 2 127.572 ns/op Lazy disarm 1sps Trans.TotalMemory avgt 2 49.111 ns/op Trans.nativeMethod avgt 2 19.734 ns/op Lazy disarm 100sps Trans.TotalMemory avgt 2 50.568 ns/op Trans.nativeMethod avgt 2 20.335 ns/op So hopefully it will be pretty unnoticeable. Thanks, Robbin On 3/19/19 3:04 PM, Doerr, Martin wrote: > Hi Robbin, > > I just tried a small stress test [1] with (+UseNewCode) and without added CPUID instruction in native wrapper [2]. > > jdk/bin/java -XX:+UnlockDiagnosticVMOptions StressJVMEntries > result: 2113929216, duration: 40.1181962 > jdk/bin/java -XX:+UnlockDiagnosticVMOptions -XX:+UseNewCode StressJVMEntries > result: 2113929216, duration: 76.0759127 > > Executed on 40 core 2-socket Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz. > So I guess nobody wants CPUID on the regular path if it can be avoided. > > Best regards, > Martin > > > > [1] StressJVMEntries.java: > public class StressJVMEntries{ > > static long sum = 0; > static final int loop_cnt = 10000000; > > public static void main(String args[]){ > Runtime rt = Runtime.getRuntime(); > long duration = System.nanoTime(); > for (int x=0; x duration = System.nanoTime() - duration; > System.out.println("result: " + sum/loop_cnt + ", duration: " + (double)duration/loop_cnt); > } > } > > > [2] patch: > diff -r 3827cd66e788 src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp > --- a/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Mon Mar 18 16:04:23 2019 +0100 > +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Tue Mar 19 15:02:31 2019 +0100 > @@ -2607,6 +2607,18 @@ > } > > __ bind(Continue); > + if (UseNewCode) { > + __ push(rax); > + __ push(rbx); > + __ push(rcx); > + __ push(rdx); > + __ xorl(rax, rax); > + __ cpuid(); > + __ pop(rdx); > + __ pop(rcx); > + __ pop(rbx); > + __ pop(rax); > + } > } > > // change thread state > > > > -----Original Message----- > From: Robbin Ehn > Sent: Freitag, 15. M?rz 2019 13:32 > To: David Holmes ; Reingruber, Richard ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > >>> >>> The last point was rather exploring the space of legal solutions than a >>> suggestion to solve the issue. I'd favor your approach if you manage to do the >>> serialization conditionally (BTW: a mechanism similar to >>> JavaThread::*deopt_suspend* >>> would be yet another way to do that). > > FYI: In the long run I'm removing the suspend-flags and turning suspend/deopt/.. > into handshakes instead. So we only have the poll to check. > > I'm testing a change where I removed the changes to the native wrappers, > leaving native threads armed (as Martin suggested) and in the poll path notice > that we are still armed (when returning from a safepoint/handshake we should be > disarmed). Disarm yourself, re-check safepoint/handshake, execute instruction > barrier. > So we don't need any changes in any native wrapper, we handle the barrier in the > polling code instead. > > I have just one issue to sort-out with that and benchmarking. > The v2 patch (not yet published) didn't show to bad of a result when we always > emit cpuid, but there is no super JNI heavy test in that suite. > > /Robbin > >>> >>> Cheers, Richard. >>> >>> [1] 8.1.3 Handling Self- and Cross-Modifying Code >>> ???? in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >>> ???? (3A, 3B, 3C & 3D): System Programming Guide >>> >>> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >>> >>> ???? on https://software.intel.com/en-us/articles/intel-sdm >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 13:24 >>> To: Reingruber, Richard ; Doerr, Martin >>> ; David Holmes ; >>> hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Richard, >>> >>> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>>> Hi Robbin, >>>> >>>> ??? > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>>> ??? > is that it was deemed that a safepoint is 'long enough'. >>>> >>>> This doesn't seem to be a very good reason. I did a little modification to your >>>> initial figure: >>> >>> That's why we are adding the instruction barrier! >>> >>>> >>>> JavaThread:???????????? | VMThread >>>> StoreLoad?????????????? | >>>> ????????????????????????? | update >>>> ????????????????????????? | ? Global ICache Invalidate ? >>>> ????????????????????????? | disarm >>>> load thread_poll??????? | >>>> >>>> Question would be: does an ICache flush happen after the immediate oop update? >>>> I would consider it problematic if not. Even if the safepoint was very long. >>> >>> The icache flush assumes it can stop speculation and flush instruction cache in >>> patchee thread. >>> Correct me if anything is wrong: >>> - sparc needs flush in all patchee threads. >>> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >>> - ppc needs isync in all patchee threads? >>> - s390 needs zarch_sync all in patchee threads? >>> - x86 needs cpuid in all in patchee threads. >>> >>> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >>> have no effect in patchee threads!? >>> >>> Adding a few words on x86 ICache::Invalidate/word. >>> We have TSO, so if the patchee thread can see poll is disarmed all other stores >>> are visible. I have seen no case ICache::Invalidate/word would make a difference >>> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >>> The problem is if we prefect an instruction over the load of the poll. >>> If poll is disarmed this perfected instruction can be fully executed. >>> If that happens to contain an immediate oop, we don't know if it's the patched >>> oop or not. >>> >>> /Robbin >>> >>>> >>>> Cheers Richard. >>>> >>>> -----Original Message----- >>>> From: Robbin Ehn >>>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>>> To: Doerr, Martin ; David Holmes >>>> ; hotspot-dev at openjdk.java.net >>>> Cc: Reingruber, Richard >>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>> >>>> Hi Martin, >>>> >>>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>>> Hi Robbin, >>>>> >>>>>> But that is whole new subject and let's keep that separate from this :) >>>>> Sure, we can keep the modification side discussion separate. This change is >>>>> already difficult enough ?? >>>>> >>>>> >>>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>>> >>>>> orderAccess_aix/linux_ppc.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>>> >>>>> orderAccess_linux_s390.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>>> >>>> >>>> Thanks! >>>> >>>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops >>>>>> encoded into the instruction stream. >>>>>> I couldn't find a call to AbstractICache::invalidate_range. >>>> >>>> I think I miss-understood you here, the reason I heard for not doing >>>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>>> safepoint is 'long enough'. >>>> >>>> /Robbin >>>> >>>>> >>>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>>> And one call to AbstractICache/ICache::invalidate_word. >>>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>>> But that is whole new subject and let's keep that separate from this :) >>>>> >>>>> I'll create issues for some of these topics, which if nothing else should be >>>>> discussed. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Does anybody know where it is or if it's missing? >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> From aph at redhat.com Tue Mar 19 14:29:05 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 19 Mar 2019 14:29:05 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <1bb9da30-4b40-b0ef-4f68-47f6c1b75670@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <440ce735-add2-db60-b672-d929731304cd@oracle.com> <1bb9da30-4b40-b0ef-4f68-47f6c1b75670@oracle.com> Message-ID: <378cf1c1-5221-f829-4f43-fe3cebf9020c@redhat.com> On 3/19/19 2:11 PM, Robbin Ehn wrote: > The numbers I'm seeing is (sps = safepoints per second): Nice! Is that benchmark available for me to try too? Thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Tue Mar 19 14:30:20 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 19 Mar 2019 14:30:20 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <1bb9da30-4b40-b0ef-4f68-47f6c1b75670@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <440ce735-add2-db60-b672-d929731304cd@oracle.com> <1bb9da30-4b40-b0ef-4f68-47f6c1b75670@oracle.com> Message-ID: Hi Robbin, the lazy disarm numbers look good. Looks like many safepoints don't make it slower. Thanks for changing and for sharing these numbers. I'm glad you can also use the lazy disarm for other purposes (removal of suspend flags). Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Dienstag, 19. M?rz 2019 15:11 To: Doerr, Martin ; David Holmes ; Reingruber, Richard ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(m): 8220351: Cross-modifying code Hi Martin, I'm running the final testing of v2 which includes the lazy disarm, since as you say, is costly. The numbers I'm seeing is (sps = safepoints per second): Baseline 1sps Trans.TotalMemory avgt 2 47.634 ns/op Trans.nativeMethod avgt 2 19.755 ns/op Baseline 100sps Trans.TotalMemory avgt 2 48.099 ns/op Trans.nativeMethod avgt 2 20.342 ns/op Always CPUID 1sps Trans.TotalMemory avgt 2 206.012 ns/op Trans.nativeMethod avgt 2 131.647 ns/op Always CPUID 100sps Trans.TotalMemory avgt 2 211.741 ns/op Trans.nativeMethod avgt 2 127.572 ns/op Lazy disarm 1sps Trans.TotalMemory avgt 2 49.111 ns/op Trans.nativeMethod avgt 2 19.734 ns/op Lazy disarm 100sps Trans.TotalMemory avgt 2 50.568 ns/op Trans.nativeMethod avgt 2 20.335 ns/op So hopefully it will be pretty unnoticeable. Thanks, Robbin On 3/19/19 3:04 PM, Doerr, Martin wrote: > Hi Robbin, > > I just tried a small stress test [1] with (+UseNewCode) and without added CPUID instruction in native wrapper [2]. > > jdk/bin/java -XX:+UnlockDiagnosticVMOptions StressJVMEntries > result: 2113929216, duration: 40.1181962 > jdk/bin/java -XX:+UnlockDiagnosticVMOptions -XX:+UseNewCode StressJVMEntries > result: 2113929216, duration: 76.0759127 > > Executed on 40 core 2-socket Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz. > So I guess nobody wants CPUID on the regular path if it can be avoided. > > Best regards, > Martin > > > > [1] StressJVMEntries.java: > public class StressJVMEntries{ > > static long sum = 0; > static final int loop_cnt = 10000000; > > public static void main(String args[]){ > Runtime rt = Runtime.getRuntime(); > long duration = System.nanoTime(); > for (int x=0; x duration = System.nanoTime() - duration; > System.out.println("result: " + sum/loop_cnt + ", duration: " + (double)duration/loop_cnt); > } > } > > > [2] patch: > diff -r 3827cd66e788 src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp > --- a/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Mon Mar 18 16:04:23 2019 +0100 > +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp Tue Mar 19 15:02:31 2019 +0100 > @@ -2607,6 +2607,18 @@ > } > > __ bind(Continue); > + if (UseNewCode) { > + __ push(rax); > + __ push(rbx); > + __ push(rcx); > + __ push(rdx); > + __ xorl(rax, rax); > + __ cpuid(); > + __ pop(rdx); > + __ pop(rcx); > + __ pop(rbx); > + __ pop(rax); > + } > } > > // change thread state > > > > -----Original Message----- > From: Robbin Ehn > Sent: Freitag, 15. M?rz 2019 13:32 > To: David Holmes ; Reingruber, Richard ; Doerr, Martin ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(m): 8220351: Cross-modifying code > >>> >>> The last point was rather exploring the space of legal solutions than a >>> suggestion to solve the issue. I'd favor your approach if you manage to do the >>> serialization conditionally (BTW: a mechanism similar to >>> JavaThread::*deopt_suspend* >>> would be yet another way to do that). > > FYI: In the long run I'm removing the suspend-flags and turning suspend/deopt/.. > into handshakes instead. So we only have the poll to check. > > I'm testing a change where I removed the changes to the native wrappers, > leaving native threads armed (as Martin suggested) and in the poll path notice > that we are still armed (when returning from a safepoint/handshake we should be > disarmed). Disarm yourself, re-check safepoint/handshake, execute instruction > barrier. > So we don't need any changes in any native wrapper, we handle the barrier in the > polling code instead. > > I have just one issue to sort-out with that and benchmarking. > The v2 patch (not yet published) didn't show to bad of a result when we always > emit cpuid, but there is no super JNI heavy test in that suite. > > /Robbin > >>> >>> Cheers, Richard. >>> >>> [1] 8.1.3 Handling Self- and Cross-Modifying Code >>> ???? in Intel(R) 64 and IA-32 ArchitecturesSoftware Developer's Manual. Volume 3 >>> ???? (3A, 3B, 3C & 3D): System Programming Guide >>> >>> https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf >>> >>> ???? on https://software.intel.com/en-us/articles/intel-sdm >>> >>> -----Original Message----- >>> From: Robbin Ehn >>> Sent: Donnerstag, 14. M?rz 2019 13:24 >>> To: Reingruber, Richard ; Doerr, Martin >>> ; David Holmes ; >>> hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>> >>> Hi Richard, >>> >>> On 3/14/19 11:12 AM, Reingruber, Richard wrote: >>>> Hi Robbin, >>>> >>>> ??? > the reason I heard for not doing invalidate_range/word in nmethod::oops_do >>>> ??? > is that it was deemed that a safepoint is 'long enough'. >>>> >>>> This doesn't seem to be a very good reason. I did a little modification to your >>>> initial figure: >>> >>> That's why we are adding the instruction barrier! >>> >>>> >>>> JavaThread:???????????? | VMThread >>>> StoreLoad?????????????? | >>>> ????????????????????????? | update >>>> ????????????????????????? | ? Global ICache Invalidate ? >>>> ????????????????????????? | disarm >>>> load thread_poll??????? | >>>> >>>> Question would be: does an ICache flush happen after the immediate oop update? >>>> I would consider it problematic if not. Even if the safepoint was very long. >>> >>> The icache flush assumes it can stop speculation and flush instruction cache in >>> patchee thread. >>> Correct me if anything is wrong: >>> - sparc needs flush in all patchee threads. >>> - arm/aarch64 needs isb + ___clear_cache(adr,..) in all patchee threads. >>> - ppc needs isync in all patchee threads? >>> - s390 needs zarch_sync all in patchee threads? >>> - x86 needs cpuid in all in patchee threads. >>> >>> So a "Global ICache Invalidate" done by patcher thread (VMThread in this case) >>> have no effect in patchee threads!? >>> >>> Adding a few words on x86 ICache::Invalidate/word. >>> We have TSO, so if the patchee thread can see poll is disarmed all other stores >>> are visible. I have seen no case ICache::Invalidate/word would make a difference >>> from a StoreStore/compiler barrier (still talking about platform code for x86) ? >>> The problem is if we prefect an instruction over the load of the poll. >>> If poll is disarmed this perfected instruction can be fully executed. >>> If that happens to contain an immediate oop, we don't know if it's the patched >>> oop or not. >>> >>> /Robbin >>> >>>> >>>> Cheers Richard. >>>> >>>> -----Original Message----- >>>> From: Robbin Ehn >>>> Sent: Donnerstag, 14. M?rz 2019 10:34 >>>> To: Doerr, Martin ; David Holmes >>>> ; hotspot-dev at openjdk.java.net >>>> Cc: Reingruber, Richard >>>> Subject: Re: RFR(m): 8220351: Cross-modifying code >>>> >>>> Hi Martin, >>>> >>>> On 3/14/19 10:25 AM, Doerr, Martin wrote: >>>>> Hi Robbin, >>>>> >>>>>> But that is whole new subject and let's keep that separate from this :) >>>>> Sure, we can keep the modification side discussion separate. This change is >>>>> already difficult enough ?? >>>>> >>>>> >>>>> Btw. I haven't sent the ppc/s390 instruction cache barriers. Here they are: >>>>> >>>>> orderAccess_aix/linux_ppc.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_isync(); } >>>>> >>>>> orderAccess_linux_s390.hpp: >>>>> inline void OrderAccess::instruction_pipeline() { inlasm_zarch_sync(); } >>>>> >>>> >>>> Thanks! >>>> >>>>>> nmethod::oops_do calls the closure f->do_oop(r->oop_addr()) for oops >>>>>> encoded into the instruction stream. >>>>>> I couldn't find a call to AbstractICache::invalidate_range. >>>> >>>> I think I miss-understood you here, the reason I heard for not doing >>>> invalidate_range/word in nmethod::oops_do is that it was deemed that a >>>> safepoint is 'long enough'. >>>> >>>> /Robbin >>>> >>>>> >>>>> Look for ICache::invalidate_range, I see 11 calls on x64. >>>>> And one call to AbstractICache/ICache::invalidate_word. >>>>> Btw I have spent some time looking at x86/icache_x86.cpp... >>>>> But that is whole new subject and let's keep that separate from this :) >>>>> >>>>> I'll create issues for some of these topics, which if nothing else should be >>>>> discussed. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Does anybody know where it is or if it's missing? >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> From robbin.ehn at oracle.com Tue Mar 19 14:53:14 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 19 Mar 2019 15:53:14 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <378cf1c1-5221-f829-4f43-fe3cebf9020c@redhat.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <440ce735-add2-db60-b672-d929731304cd@oracle.com> <1bb9da30-4b40-b0ef-4f68-47f6c1b75670@oracle.com> <378cf1c1-5221-f829-4f43-fe3cebf9020c@redhat.com> Message-ID: <7721b154-e474-8ccb-431b-67a58f0cb7b0@oracle.com> It's a really simple JMH benchmark. I put it here: http://cr.openjdk.java.net/~rehn/8220351/benchmark/jmh/ v2 (still testing) http://cr.openjdk.java.net/~rehn/v2/ /Robbin On 3/19/19 3:29 PM, Andrew Haley wrote: > On 3/19/19 2:11 PM, Robbin Ehn wrote: >> The numbers I'm seeing is (sps = safepoints per second): > > Nice! Is that benchmark available for me to try too? Thanks. > From robbin.ehn at oracle.com Tue Mar 19 14:59:20 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 19 Mar 2019 15:59:20 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <7721b154-e474-8ccb-431b-67a58f0cb7b0@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f25db7a-2c88-c76a-050e-9971e1153d2f@oracle.com> <36632504-4dda-3515-2360-9e026b90fcba@oracle.com> <440ce735-add2-db60-b672-d929731304cd@oracle.com> <1bb9da30-4b40-b0ef-4f68-47f6c1b75670@oracle.com> <378cf1c1-5221-f829-4f43-fe3cebf9020c@redhat.com> <7721b154-e474-8ccb-431b-67a58f0cb7b0@oracle.com> Message-ID: URL corrected: On 3/19/19 3:53 PM, Robbin Ehn wrote: > v2 (still testing) http://cr.openjdk.java.net/~rehn/8220351/v2/ > http://cr.openjdk.java.net/~rehn/v2/ > > /Robbin > > On 3/19/19 3:29 PM, Andrew Haley wrote: >> On 3/19/19 2:11 PM, Robbin Ehn wrote: >>> The numbers I'm seeing is (sps = safepoints per second): >> >> Nice! Is that benchmark available for me to try too? Thanks. >> From robbin.ehn at oracle.com Tue Mar 19 16:43:45 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 19 Mar 2019 17:43:45 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: Hi all, here is v2: Full: http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ Inc: http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ The deprecated non-TLH path is missing the correct cross-mod fence, but since 13 will be the last release containing that code path I think that is okay. We just leave native/native trans armed, and let the thread disarm on the transition. If it disarms a new safepoint, we re-arm and stop at next polling site, that avoids recursion. Passes stress and hs-t1-5. As in the previous mail, performance numbers look good. Thanks, Robbin On 3/8/19 4:24 PM, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state it can start > executing updated code. E.g. an oop in the instruction stream can have been > updated. > > Most hardware's require a barrier or that the code cross modified is far away to > guarantee that the thread executing the updated instruction stream sees the > modification. > > What far away is and how far an update instruction stream is from a safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe > state closer to a nmethod can increase changes of a crash. > > A few benchmarks will see a performance regression with the extra serializing, > such as Octane-Crypto -5% (x86). > > With a much more elaborate fix, such as changing the local poll to have more > than two states (armed/disarmed), it would be possible to only emit such > instruction stream barrier when the poll have been armed while the thread was > safepoint safe. > > Passes t1-3 on our platforms, and test built what I can. > > Thanks, Robbin From martin.doerr at sap.com Tue Mar 19 16:52:40 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 19 Mar 2019 16:52:40 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: Hi Robbin, some feedback on the platform code: orderAccess_aix_ppc.hpp should be same as for linux: inline void OrderAccess::cross_modify_fence() { inlasm_isync(); } Not sure if the inline assembler code on x86 necessarily needs a "clobber memory" effect. I don't know what a C++ compiler is allowed to do if it doesn't know that the code has some kind of memory effect. For ebx...edx, you could also use clobber if you want to make it shorter. E.g. with "+a" to use eax as input and output: int idx = 0; __asm__ volatile ("cpuid " : "+a" (idx) : : "ebx", "ecx", "edx", "memory"); Just a minor suggestion. Not important. My first impression of the shared changes is good, but I will need more time to think about it. Thanks again for implementing lazy disarm to get rid of the performance penalty. Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Dienstag, 19. M?rz 2019 17:44 To: hotspot-dev at openjdk.java.net Cc: David Holmes ; Andrew Haley ; Erik ?sterlund ; Doerr, Martin ; Reingruber, Richard Subject: Re: RFR(m): 8220351: Cross-modifying code Hi all, here is v2: Full: http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ Inc: http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ The deprecated non-TLH path is missing the correct cross-mod fence, but since 13 will be the last release containing that code path I think that is okay. We just leave native/native trans armed, and let the thread disarm on the transition. If it disarms a new safepoint, we re-arm and stop at next polling site, that avoids recursion. Passes stress and hs-t1-5. As in the previous mail, performance numbers look good. Thanks, Robbin On 3/8/19 4:24 PM, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state it can start > executing updated code. E.g. an oop in the instruction stream can have been > updated. > > Most hardware's require a barrier or that the code cross modified is far away to > guarantee that the thread executing the updated instruction stream sees the > modification. > > What far away is and how far an update instruction stream is from a safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe > state closer to a nmethod can increase changes of a crash. > > A few benchmarks will see a performance regression with the extra serializing, > such as Octane-Crypto -5% (x86). > > With a much more elaborate fix, such as changing the local poll to have more > than two states (armed/disarmed), it would be possible to only emit such > instruction stream barrier when the poll have been armed while the thread was > safepoint safe. > > Passes t1-3 on our platforms, and test built what I can. > > Thanks, Robbin From aph at redhat.com Tue Mar 19 17:08:51 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 19 Mar 2019 17:08:51 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <9f115d6b-9f08-f5b2-b7d8-161cdb8718bf@redhat.com> On 3/19/19 4:52 PM, Doerr, Martin wrote: > Not sure if the inline assembler code on x86 necessarily needs a "clobber memory" effect. > I don't know what a C++ compiler is allowed to do if it doesn't know that the code has some kind of memory effect. Leave the memory clobber. No memory clobber means that you are telling the compiler that this operation has no memory effect, which means that the compiler can move the asm *before* you wrote out the changes to memory. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Tue Mar 19 17:12:47 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 19 Mar 2019 17:12:47 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <9f115d6b-9f08-f5b2-b7d8-161cdb8718bf@redhat.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f115d6b-9f08-f5b2-b7d8-161cdb8718bf@redhat.com> Message-ID: Hi Andrew, sorry for my kind of confusing formulation. Current proposal doesn't have clobber. I also prefer adding it to be on the safe side. Best regards, Martin -----Original Message----- From: Andrew Haley Sent: Dienstag, 19. M?rz 2019 18:09 To: Doerr, Martin ; Robbin Ehn ; hotspot-dev at openjdk.java.net Cc: David Holmes ; Erik ?sterlund ; Reingruber, Richard Subject: Re: RFR(m): 8220351: Cross-modifying code On 3/19/19 4:52 PM, Doerr, Martin wrote: > Not sure if the inline assembler code on x86 necessarily needs a "clobber memory" effect. > I don't know what a C++ compiler is allowed to do if it doesn't know that the code has some kind of memory effect. Leave the memory clobber. No memory clobber means that you are telling the compiler that this operation has no memory effect, which means that the compiler can move the asm *before* you wrote out the changes to memory. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From kim.barrett at oracle.com Tue Mar 19 17:58:13 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 19 Mar 2019 13:58:13 -0400 Subject: RFR(XS): 8220690: ATTRIBUTE_ALIGNED requires GNU extensions enabled In-Reply-To: <7363200f-2709-61cf-a0e1-73c3dd7cba71@redhat.com> References: <2879CC12-CC8C-4ECA-9D7C-0D60F8AC8263@oracle.com> <7363200f-2709-61cf-a0e1-73c3dd7cba71@redhat.com> Message-ID: <11576B22-1C36-4EFA-B972-5B7DAFF4DC44@oracle.com> > On Mar 19, 2019, at 4:41 AM, Aleksey Shipilev wrote: > > On 3/15/19 11:18 PM, Kim Barrett wrote: >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8220690/open.00/ > > I am fine with it. > > Severin, can you test jdk/jdk still builds on old RHEL with this patch? > > -Aleksey Thanks. From kim.barrett at oracle.com Tue Mar 19 18:00:01 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 19 Mar 2019 14:00:01 -0400 Subject: RFR(XS): 8220690: ATTRIBUTE_ALIGNED requires GNU extensions enabled In-Reply-To: <65a23b290a0572b2e261d0849602bbeeb8d9a015.camel@redhat.com> References: <2879CC12-CC8C-4ECA-9D7C-0D60F8AC8263@oracle.com> <7363200f-2709-61cf-a0e1-73c3dd7cba71@redhat.com> <5ee9bdc759e3a057ed7c2222fb3d7c74b0e73443.camel@redhat.com> <65a23b290a0572b2e261d0849602bbeeb8d9a015.camel@redhat.com> Message-ID: > On Mar 19, 2019, at 8:28 AM, Severin Gehwolf wrote: > > On Tue, 2019-03-19 at 10:23 +0100, Severin Gehwolf wrote: >> >> Sure. It'll take a bit. godbolt.org tells me that it should work for >> GCC 4.4.7, though. I'll reply once I have results. > > Results are in. Good to go. > > Thanks, > Severin Thanks for the review, and thanks for checking the old compiler. I forget about godbolt.org, and should have checked that myself. From dthomson at google.com Tue Mar 19 18:08:53 2019 From: dthomson at google.com (Derek Thomson) Date: Tue, 19 Mar 2019 11:08:53 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool Message-ID: Changed uint to bool throughout. Could someone review my change please? Also, what tests should I run to validate this? Webrev: http://cr.openjdk.java.net/~jcbeyler/8157372/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8157372 From vladimir.kozlov at oracle.com Tue Mar 19 20:38:58 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 19 Mar 2019 13:38:58 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: Message-ID: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> Hi Derek, On 3/19/19 11:08 AM, Derek Thomson wrote: > Changed uint to bool throughout. Could someone review my change please? Looks good to me. > Also, what tests should I run to validate this? You can use submit repo to do build and tier1 testing on all platforms supported by Oracle: https://wiki.openjdk.java.net/display/Build/Submit+Repo > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8157372/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8157372 > Thanks, Vladimir From david.holmes at oracle.com Tue Mar 19 21:54:16 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 07:54:16 +1000 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> Message-ID: <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> CC'ing hotspot-dev. I agree this needs to be checked on every platform affected. I can't comment on the fix itself. David On 20/03/2019 2:36 am, Erik Joelsson wrote: > I think this needs to be reviewed by at least someone in hotspot who > regularly builds hsdis. I can't really comment on the validity of the > patch as I'm unfamiliar with both hsdis as well as this makefile. Have > you at least verified the build on all the platforms which you affect > with this change (which would be at least Macos, AIX and Windows in a > normal Cygwin VS env)? > > /Erik > > On 2019-03-18 17:56, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this change: >> >> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 >> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ >> >> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with >> gcc-mingw-w64-x86-64, but I saw error messages that some functions >> which are provided by zlib are unresolved. >> We need to link to zlib. >> >> >> Thanks, >> >> Yasumasa From yasuenag at gmail.com Wed Mar 20 01:07:36 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 20 Mar 2019 10:07:36 +0900 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> Message-ID: Hi Erik, David, I checked this change on Linux x64 and MinGW for Windows. According to hsdis README, we need to use MinGW cross compiler to build hsdis [1]. So I think Cygwin is not required. I do not have macOS and AIX. So I cannot check this change on them. BTW is hsdis included Java SE spec? hsdis seems not to be included jtreg tests, and it is not contained in OpenJDK binaries. I think it is allowed even if we lack some test for hsdis if hsdis is not required for Java SE. If not so, I want sponsor(s) for this change. Thanks, Yasumasa [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 2019?3?20?(?) 6:54 David Holmes : > > CC'ing hotspot-dev. I agree this needs to be checked on every platform > affected. I can't comment on the fix itself. > > David > > On 20/03/2019 2:36 am, Erik Joelsson wrote: > > I think this needs to be reviewed by at least someone in hotspot who > > regularly builds hsdis. I can't really comment on the validity of the > > patch as I'm unfamiliar with both hsdis as well as this makefile. Have > > you at least verified the build on all the platforms which you affect > > with this change (which would be at least Macos, AIX and Windows in a > > normal Cygwin VS env)? > > > > /Erik > > > > On 2019-03-18 17:56, Yasumasa Suenaga wrote: > >> Hi all, > >> > >> Please review this change: > >> > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 > >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ > >> > >> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with > >> gcc-mingw-w64-x86-64, but I saw error messages that some functions > >> which are provided by zlib are unresolved. > >> We need to link to zlib. > >> > >> > >> Thanks, > >> > >> Yasumasa From david.holmes at oracle.com Wed Mar 20 01:38:22 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 11:38:22 +1000 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> Message-ID: <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> Hi Yasumasa, I'm not familar with building hsdis, but if the only currnet problem is on Windows, why is the fix not restricted to only building on Windows? Otherwise we need people who build hsdis on other platforms to comment on the appropriateness of the fix. Thanks, David On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: > Hi Erik, David, > > I checked this change on Linux x64 and MinGW for Windows. > According to hsdis README, we need to use MinGW cross compiler to > build hsdis [1]. So I think Cygwin is not required. > > I do not have macOS and AIX. So I cannot check this change on them. > > BTW is hsdis included Java SE spec? > hsdis seems not to be included jtreg tests, and it is not contained in > OpenJDK binaries. > I think it is allowed even if we lack some test for hsdis if hsdis is > not required for Java SE. > If not so, I want sponsor(s) for this change. > > > Thanks, > > Yasumasa > > > [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 > > 2019?3?20?(?) 6:54 David Holmes : >> >> CC'ing hotspot-dev. I agree this needs to be checked on every platform >> affected. I can't comment on the fix itself. >> >> David >> >> On 20/03/2019 2:36 am, Erik Joelsson wrote: >>> I think this needs to be reviewed by at least someone in hotspot who >>> regularly builds hsdis. I can't really comment on the validity of the >>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have >>> you at least verified the build on all the platforms which you affect >>> with this change (which would be at least Macos, AIX and Windows in a >>> normal Cygwin VS env)? >>> >>> /Erik >>> >>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: >>>> Hi all, >>>> >>>> Please review this change: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 >>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ >>>> >>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with >>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions >>>> which are provided by zlib are unresolved. >>>> We need to link to zlib. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa From sanhong.lsh at alibaba-inc.com Wed Mar 20 01:49:18 2019 From: sanhong.lsh at alibaba-inc.com (=?UTF-8?B?5p2O5LiJ57qiKOS4iee6oik=?=) Date: Wed, 20 Mar 2019 09:49:18 +0800 Subject: RFC: JWarmup precompile java hot methods at application startup Message-ID: <020701d4debf$22110e90$66332bb0$@alibaba-inc.com> HI Alan, Thanks for comment. We can put com.alibaba.jwarmup in separate new module. Any further suggestion for the package name? Or anyone can give us some options for considering:-) Thanks! Sanhong -----????----- ???: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] ?? Alan Bateman ????: 2019?3?15? 15:30 ???: yumin qi ; hotspot-dev at openjdk.java.net ??: Re: RFC: JWarmup precompile java hot methods at application startup On 15/03/2019 05:59, yumin qi wrote: > Please review the patch for implementing JEP 8203832: > (*https://bugs.openjdk.java.net/browse/JDK-8203832 > )* > > Bug: *https://bugs.openjdk.java.net/browse/JDK-8220692 > * > Webrev: http://cr.openjdk.java.net/~minqi/8220692/webrev-01/ > > > Summary: This patch implements the first version of JIT warmup to > solve the java application peak time CPU contest between JIT compiler > threads and regular application java threads. The CPU contest isa > avoided by pre-run to collect comilation data and precompile the hot > java methods during application startup based on the pre-run collected > data. > com.alibaba.jwarmup will need to find a new home, it cannot be exported by java.base. I think you'll need to look at create a new module for this API if it's really needed, it will need a better package name too. -Alan. From yasuenag at gmail.com Wed Mar 20 01:54:13 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 20 Mar 2019 10:54:13 +0900 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> Message-ID: Hi David, 2019?3?20?(?) 10:38 David Holmes : > > Hi Yasumasa, > > I'm not familar with building hsdis, but if the only currnet problem is > on Windows, why is the fix not restricted to only building on Windows? For Solaris and Linux, -lz is not added to LDFLAGS. I think it means initial committer want to use zlib in binutils. And zlib in binutils will be built implicitly in build phase in binutils. > Otherwise we need people who build hsdis on other platforms to comment > on the appropriateness of the fix. If using different zlib is used by platforms is allowed, I will upload webrev which affects Windows only. What do you think? Thanks, Yasumasa > Thanks, > David > > On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: > > Hi Erik, David, > > > > I checked this change on Linux x64 and MinGW for Windows. > > According to hsdis README, we need to use MinGW cross compiler to > > build hsdis [1]. So I think Cygwin is not required. > > > > I do not have macOS and AIX. So I cannot check this change on them. > > > > BTW is hsdis included Java SE spec? > > hsdis seems not to be included jtreg tests, and it is not contained in > > OpenJDK binaries. > > I think it is allowed even if we lack some test for hsdis if hsdis is > > not required for Java SE. > > If not so, I want sponsor(s) for this change. > > > > > > Thanks, > > > > Yasumasa > > > > > > [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 > > > > 2019?3?20?(?) 6:54 David Holmes : > >> > >> CC'ing hotspot-dev. I agree this needs to be checked on every platform > >> affected. I can't comment on the fix itself. > >> > >> David > >> > >> On 20/03/2019 2:36 am, Erik Joelsson wrote: > >>> I think this needs to be reviewed by at least someone in hotspot who > >>> regularly builds hsdis. I can't really comment on the validity of the > >>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have > >>> you at least verified the build on all the platforms which you affect > >>> with this change (which would be at least Macos, AIX and Windows in a > >>> normal Cygwin VS env)? > >>> > >>> /Erik > >>> > >>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: > >>>> Hi all, > >>>> > >>>> Please review this change: > >>>> > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 > >>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ > >>>> > >>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with > >>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions > >>>> which are provided by zlib are unresolved. > >>>> We need to link to zlib. > >>>> > >>>> > >>>> Thanks, > >>>> > >>>> Yasumasa From david.holmes at oracle.com Wed Mar 20 02:25:14 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 12:25:14 +1000 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> Message-ID: <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: > Hi David, > > 2019?3?20?(?) 10:38 David Holmes : >> >> Hi Yasumasa, >> >> I'm not familar with building hsdis, but if the only currnet problem is >> on Windows, why is the fix not restricted to only building on Windows? > > For Solaris and Linux, -lz is not added to LDFLAGS. I think it means > initial committer want to use zlib in binutils. > And zlib in binutils will be built implicitly in build phase in binutils. > >> Otherwise we need people who build hsdis on other platforms to comment >> on the appropriateness of the fix. > > If using different zlib is used by platforms is allowed, I will upload > webrev which affects Windows only. > What do you think? That seems preferable/simpler. Then you only need to know that your change works for non-MinGW64 environments. Thanks, David > > Thanks, > > Yasumasa > > >> Thanks, >> David >> >> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: >>> Hi Erik, David, >>> >>> I checked this change on Linux x64 and MinGW for Windows. >>> According to hsdis README, we need to use MinGW cross compiler to >>> build hsdis [1]. So I think Cygwin is not required. >>> >>> I do not have macOS and AIX. So I cannot check this change on them. >>> >>> BTW is hsdis included Java SE spec? >>> hsdis seems not to be included jtreg tests, and it is not contained in >>> OpenJDK binaries. >>> I think it is allowed even if we lack some test for hsdis if hsdis is >>> not required for Java SE. >>> If not so, I want sponsor(s) for this change. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 >>> >>> 2019?3?20?(?) 6:54 David Holmes : >>>> >>>> CC'ing hotspot-dev. I agree this needs to be checked on every platform >>>> affected. I can't comment on the fix itself. >>>> >>>> David >>>> >>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: >>>>> I think this needs to be reviewed by at least someone in hotspot who >>>>> regularly builds hsdis. I can't really comment on the validity of the >>>>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have >>>>> you at least verified the build on all the platforms which you affect >>>>> with this change (which would be at least Macos, AIX and Windows in a >>>>> normal Cygwin VS env)? >>>>> >>>>> /Erik >>>>> >>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this change: >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 >>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ >>>>>> >>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with >>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions >>>>>> which are provided by zlib are unresolved. >>>>>> We need to link to zlib. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa From yasuenag at gmail.com Wed Mar 20 05:01:18 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 20 Mar 2019 14:01:18 +0900 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> Message-ID: Thanks David! I uploaded new webrev. This change affects Windows only, and I confirmed it works fine for Linux x64 and MinGW64. http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ Thanks, Yasumasa 2019?3?20?(?) 11:25 David Holmes : > > On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: > > Hi David, > > > > 2019?3?20?(?) 10:38 David Holmes : > >> > >> Hi Yasumasa, > >> > >> I'm not familar with building hsdis, but if the only currnet problem is > >> on Windows, why is the fix not restricted to only building on Windows? > > > > For Solaris and Linux, -lz is not added to LDFLAGS. I think it means > > initial committer want to use zlib in binutils. > > And zlib in binutils will be built implicitly in build phase in binutils. > > > >> Otherwise we need people who build hsdis on other platforms to comment > >> on the appropriateness of the fix. > > > > If using different zlib is used by platforms is allowed, I will upload > > webrev which affects Windows only. > > What do you think? > > That seems preferable/simpler. Then you only need to know that your > change works for non-MinGW64 environments. > > Thanks, > David > > > > > Thanks, > > > > Yasumasa > > > > > >> Thanks, > >> David > >> > >> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: > >>> Hi Erik, David, > >>> > >>> I checked this change on Linux x64 and MinGW for Windows. > >>> According to hsdis README, we need to use MinGW cross compiler to > >>> build hsdis [1]. So I think Cygwin is not required. > >>> > >>> I do not have macOS and AIX. So I cannot check this change on them. > >>> > >>> BTW is hsdis included Java SE spec? > >>> hsdis seems not to be included jtreg tests, and it is not contained in > >>> OpenJDK binaries. > >>> I think it is allowed even if we lack some test for hsdis if hsdis is > >>> not required for Java SE. > >>> If not so, I want sponsor(s) for this change. > >>> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>> [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 > >>> > >>> 2019?3?20?(?) 6:54 David Holmes : > >>>> > >>>> CC'ing hotspot-dev. I agree this needs to be checked on every platform > >>>> affected. I can't comment on the fix itself. > >>>> > >>>> David > >>>> > >>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: > >>>>> I think this needs to be reviewed by at least someone in hotspot who > >>>>> regularly builds hsdis. I can't really comment on the validity of the > >>>>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have > >>>>> you at least verified the build on all the platforms which you affect > >>>>> with this change (which would be at least Macos, AIX and Windows in a > >>>>> normal Cygwin VS env)? > >>>>> > >>>>> /Erik > >>>>> > >>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: > >>>>>> Hi all, > >>>>>> > >>>>>> Please review this change: > >>>>>> > >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 > >>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ > >>>>>> > >>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with > >>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions > >>>>>> which are provided by zlib are unresolved. > >>>>>> We need to link to zlib. > >>>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Yasumasa From david.holmes at oracle.com Wed Mar 20 05:25:40 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 15:25:40 +1000 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> Message-ID: <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> Hi Yasumasa, On 20/03/2019 3:01 pm, Yasumasa Suenaga wrote: > Thanks David! > > I uploaded new webrev. > This change affects Windows only, and I confirmed it works fine for > Linux x64 and MinGW64. > http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ I this is for MINGW only then shouldn't the change be made within this section: ## OS = Linux ## ifeq ($(OS),Linux) ifneq ($(MINGW),) LIB_EXT = .dll CPPFLAGS += -I$(TARGET_DIR)/include LDFLAGS += -L$(TARGET_DIR)/lib OS=windows ifneq ($(findstring x86_64-,$(MINGW)),) ARCH=amd64 else ARCH=i386 endif CC = $(MINGW)-gcc CONFIGURE_ARGS= --host=$(MINGW) --target=$(MINGW) else #linux ? David ----- > > Thanks, > > Yasumasa > > > 2019?3?20?(?) 11:25 David Holmes : >> >> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> 2019?3?20?(?) 10:38 David Holmes : >>>> >>>> Hi Yasumasa, >>>> >>>> I'm not familar with building hsdis, but if the only currnet problem is >>>> on Windows, why is the fix not restricted to only building on Windows? >>> >>> For Solaris and Linux, -lz is not added to LDFLAGS. I think it means >>> initial committer want to use zlib in binutils. >>> And zlib in binutils will be built implicitly in build phase in binutils. >>> >>>> Otherwise we need people who build hsdis on other platforms to comment >>>> on the appropriateness of the fix. >>> >>> If using different zlib is used by platforms is allowed, I will upload >>> webrev which affects Windows only. >>> What do you think? >> >> That seems preferable/simpler. Then you only need to know that your >> change works for non-MinGW64 environments. >> >> Thanks, >> David >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> David >>>> >>>> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: >>>>> Hi Erik, David, >>>>> >>>>> I checked this change on Linux x64 and MinGW for Windows. >>>>> According to hsdis README, we need to use MinGW cross compiler to >>>>> build hsdis [1]. So I think Cygwin is not required. >>>>> >>>>> I do not have macOS and AIX. So I cannot check this change on them. >>>>> >>>>> BTW is hsdis included Java SE spec? >>>>> hsdis seems not to be included jtreg tests, and it is not contained in >>>>> OpenJDK binaries. >>>>> I think it is allowed even if we lack some test for hsdis if hsdis is >>>>> not required for Java SE. >>>>> If not so, I want sponsor(s) for this change. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 >>>>> >>>>> 2019?3?20?(?) 6:54 David Holmes : >>>>>> >>>>>> CC'ing hotspot-dev. I agree this needs to be checked on every platform >>>>>> affected. I can't comment on the fix itself. >>>>>> >>>>>> David >>>>>> >>>>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: >>>>>>> I think this needs to be reviewed by at least someone in hotspot who >>>>>>> regularly builds hsdis. I can't really comment on the validity of the >>>>>>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have >>>>>>> you at least verified the build on all the platforms which you affect >>>>>>> with this change (which would be at least Macos, AIX and Windows in a >>>>>>> normal Cygwin VS env)? >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review this change: >>>>>>>> >>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 >>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ >>>>>>>> >>>>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with >>>>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions >>>>>>>> which are provided by zlib are unresolved. >>>>>>>> We need to link to zlib. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa From david.holmes at oracle.com Wed Mar 20 05:49:01 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 15:49:01 +1000 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <82980f85-0343-d783-baf3-0dd5a239b696@loongson.cn> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <82980f85-0343-d783-baf3-0dd5a239b696@loongson.cn> Message-ID: <484b7a37-d05c-79b6-0595-b311b4fc21e2@oracle.com> Hi Jie, On 19/03/2019 5:49 pm, Jie Fu wrote: > Hi David, > > Thank you so much for your review and suggestions. > > Please review the second version: > http://cr.openjdk.java.net/~jiefu/8220658/webrev.01/ This seems okay to me. I have to admit that when looking at the original diff, I did not notice that other sections already reported "failed". If I had noticed I may not have suggested using "not supported" instead. Lets see what others think. I've cc'd Bob to see if he can also review this. Thanks, David ----- > Thanks a lot. > > Best regards, > Jie > > > On 2019/3/19 ??5:43, David Holmes wrote: >> Hi Jie, >> >> On 14/03/2019 8:00 pm, Jie Fu wrote: >>> Hi all, >>> >>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 >>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >> >> This seems a reasonable improvement, but I have a couple of >> suggestions. To avoid so much duplication you can change this structure: >> >> ?? if (i > 0) { >> ???? st->print("cpu_quota: %d\n", i); >> ?? } else { >> ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" : >> "no quota"); >> ?? } >> >> to this: >> >> ?? st->print("cpu_quota: "); >> ?? if (i > 0) { >> ????? st->print("%d\n", i); >> ?? } else { >> ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); >> ?? } >> >> Also I suggest replacing "failed" with "not supported". >> >> Finally one minor typo: s/"no share"/"no shares"/ >> >> Thanks, >> David >> ----- >> >>> In the VM's error log, the container information was dumpped as: >>> --------------------------------- >>> container (cgroup) information: >>> container_type: cgroupv1 >>> cpu_cpuset_cpus: 0-11 >>> cpu_memory_nodes: 0 >>> active_processor_count: 12 >>> cpu_quota: -1 >>> cpu_period: 100000 >>> cpu_shares: -1 >>> memory_limit_in_bytes: -1 >>> memory_and_swap_limit_in_bytes: -2 >>> memory_soft_limit_in_bytes: -1 >>> memory_usage_in_bytes: 11690164224 >>> memory_max_usage_in_bytes: 1806413824 >>> --------------------------------- >>> >>> Items such as "memory_limit_in_bytes" and >>> "memory_and_swap_limit_in_bytes" are difficult to understand. >>> >>> It might be more human-readable if we could dump it like this: >>> --------------------------------- >>> container (cgroup) information: >>> container_type: cgroupv1 >>> cpu_cpuset_cpus: 0-11 >>> cpu_memory_nodes: 0 >>> active_processor_count: 12 >>> cpu_quota: no quota >>> cpu_period: 100000 >>> cpu_shares: no share >>> memory_limit_in_bytes: unlimited >>> memory_and_swap_limit_in_bytes: failed >>> memory_soft_limit_in_bytes: unlimited >>> memory_usage_in_bytes: 11690164224 >>> memory_max_usage_in_bytes: 1806413824 >>> --------------------------------- >>> >>> I made this patch to improve the readability of container information >>> in the error log. >>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>> >>> Could you please review it? >>> Thanks a lot. >>> >>> Best regards, >>> Jie >>> >>> > From yasuenag at gmail.com Wed Mar 20 06:03:15 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 20 Mar 2019 15:03:15 +0900 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> Message-ID: Hi David, 2019?3?20?(?) 14:25 David Holmes : > > Hi Yasumasa, > > On 20/03/2019 3:01 pm, Yasumasa Suenaga wrote: > > Thanks David! > > > > I uploaded new webrev. > > This change affects Windows only, and I confirmed it works fine for > > Linux x64 and MinGW64. > > http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ > > I this is for MINGW only then shouldn't the change be made within this > section: Did you say we should determine with $(MINGW) whether libz.a is added to LDFLAGS? webrev is here: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.02/ The code is added to outside of the section you pointed. $(LIBRARIES) is available after OS-specific code. So we need to change like webrev.02 to minimize changes. Yasumasa > ## OS = Linux ## > ifeq ($(OS),Linux) > ifneq ($(MINGW),) > LIB_EXT = .dll > CPPFLAGS += -I$(TARGET_DIR)/include > LDFLAGS += -L$(TARGET_DIR)/lib > OS=windows > ifneq ($(findstring x86_64-,$(MINGW)),) > ARCH=amd64 > else > ARCH=i386 > endif > CC = $(MINGW)-gcc > CONFIGURE_ARGS= --host=$(MINGW) --target=$(MINGW) > else #linux > ? > > David > ----- > > > > > Thanks, > > > > Yasumasa > > > > > > 2019?3?20?(?) 11:25 David Holmes : > >> > >> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: > >>> Hi David, > >>> > >>> 2019?3?20?(?) 10:38 David Holmes : > >>>> > >>>> Hi Yasumasa, > >>>> > >>>> I'm not familar with building hsdis, but if the only currnet problem is > >>>> on Windows, why is the fix not restricted to only building on Windows? > >>> > >>> For Solaris and Linux, -lz is not added to LDFLAGS. I think it means > >>> initial committer want to use zlib in binutils. > >>> And zlib in binutils will be built implicitly in build phase in binutils. > >>> > >>>> Otherwise we need people who build hsdis on other platforms to comment > >>>> on the appropriateness of the fix. > >>> > >>> If using different zlib is used by platforms is allowed, I will upload > >>> webrev which affects Windows only. > >>> What do you think? > >> > >> That seems preferable/simpler. Then you only need to know that your > >> change works for non-MinGW64 environments. > >> > >> Thanks, > >> David > >> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>>> Thanks, > >>>> David > >>>> > >>>> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: > >>>>> Hi Erik, David, > >>>>> > >>>>> I checked this change on Linux x64 and MinGW for Windows. > >>>>> According to hsdis README, we need to use MinGW cross compiler to > >>>>> build hsdis [1]. So I think Cygwin is not required. > >>>>> > >>>>> I do not have macOS and AIX. So I cannot check this change on them. > >>>>> > >>>>> BTW is hsdis included Java SE spec? > >>>>> hsdis seems not to be included jtreg tests, and it is not contained in > >>>>> OpenJDK binaries. > >>>>> I think it is allowed even if we lack some test for hsdis if hsdis is > >>>>> not required for Java SE. > >>>>> If not so, I want sponsor(s) for this change. > >>>>> > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Yasumasa > >>>>> > >>>>> > >>>>> [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 > >>>>> > >>>>> 2019?3?20?(?) 6:54 David Holmes : > >>>>>> > >>>>>> CC'ing hotspot-dev. I agree this needs to be checked on every platform > >>>>>> affected. I can't comment on the fix itself. > >>>>>> > >>>>>> David > >>>>>> > >>>>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: > >>>>>>> I think this needs to be reviewed by at least someone in hotspot who > >>>>>>> regularly builds hsdis. I can't really comment on the validity of the > >>>>>>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have > >>>>>>> you at least verified the build on all the platforms which you affect > >>>>>>> with this change (which would be at least Macos, AIX and Windows in a > >>>>>>> normal Cygwin VS env)? > >>>>>>> > >>>>>>> /Erik > >>>>>>> > >>>>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: > >>>>>>>> Hi all, > >>>>>>>> > >>>>>>>> Please review this change: > >>>>>>>> > >>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 > >>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ > >>>>>>>> > >>>>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with > >>>>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions > >>>>>>>> which are provided by zlib are unresolved. > >>>>>>>> We need to link to zlib. > >>>>>>>> > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Yasumasa From fujie at loongson.cn Wed Mar 20 06:07:55 2019 From: fujie at loongson.cn (Jie Fu) Date: Wed, 20 Mar 2019 14:07:55 +0800 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <484b7a37-d05c-79b6-0595-b311b4fc21e2@oracle.com> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <82980f85-0343-d783-baf3-0dd5a239b696@loongson.cn> <484b7a37-d05c-79b6-0595-b311b4fc21e2@oracle.com> Message-ID: <3bdcefba-c646-4abc-1bee-fcb215530f1e@loongson.cn> Thanks David. Actually, I prefer "not supported" to "failed". I'm also OK to change it back if someone has different ideas. Best regards, Jie On 2019/3/20 ??1:49, David Holmes wrote: > Hi Jie, > > On 19/03/2019 5:49 pm, Jie Fu wrote: >> Hi David, >> >> Thank you so much for your review and suggestions. >> >> Please review the second version: >> http://cr.openjdk.java.net/~jiefu/8220658/webrev.01/ > > This seems okay to me. > > I have to admit that when looking at the original diff, I did not > notice that other sections already reported "failed". If I had noticed > I may not have suggested using "not supported" instead. Lets see what > others think. I've cc'd Bob to see if he can also review this. > > Thanks, > David > ----- > >> Thanks a lot. >> >> Best regards, >> Jie >> >> >> On 2019/3/19 ??5:43, David Holmes wrote: >>> Hi Jie, >>> >>> On 14/03/2019 8:00 pm, Jie Fu wrote: >>>> Hi all, >>>> >>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 >>>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >>> >>> This seems a reasonable improvement, but I have a couple of >>> suggestions. To avoid so much duplication you can change this >>> structure: >>> >>> ?? if (i > 0) { >>> ???? st->print("cpu_quota: %d\n", i); >>> ?? } else { >>> ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" >>> : "no quota"); >>> ?? } >>> >>> to this: >>> >>> ?? st->print("cpu_quota: "); >>> ?? if (i > 0) { >>> ????? st->print("%d\n", i); >>> ?? } else { >>> ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); >>> ?? } >>> >>> Also I suggest replacing "failed" with "not supported". >>> >>> Finally one minor typo: s/"no share"/"no shares"/ >>> >>> Thanks, >>> David >>> ----- >>> >>>> In the VM's error log, the container information was dumpped as: >>>> --------------------------------- >>>> container (cgroup) information: >>>> container_type: cgroupv1 >>>> cpu_cpuset_cpus: 0-11 >>>> cpu_memory_nodes: 0 >>>> active_processor_count: 12 >>>> cpu_quota: -1 >>>> cpu_period: 100000 >>>> cpu_shares: -1 >>>> memory_limit_in_bytes: -1 >>>> memory_and_swap_limit_in_bytes: -2 >>>> memory_soft_limit_in_bytes: -1 >>>> memory_usage_in_bytes: 11690164224 >>>> memory_max_usage_in_bytes: 1806413824 >>>> --------------------------------- >>>> >>>> Items such as "memory_limit_in_bytes" and >>>> "memory_and_swap_limit_in_bytes" are difficult to understand. >>>> >>>> It might be more human-readable if we could dump it like this: >>>> --------------------------------- >>>> container (cgroup) information: >>>> container_type: cgroupv1 >>>> cpu_cpuset_cpus: 0-11 >>>> cpu_memory_nodes: 0 >>>> active_processor_count: 12 >>>> cpu_quota: no quota >>>> cpu_period: 100000 >>>> cpu_shares: no share >>>> memory_limit_in_bytes: unlimited >>>> memory_and_swap_limit_in_bytes: failed >>>> memory_soft_limit_in_bytes: unlimited >>>> memory_usage_in_bytes: 11690164224 >>>> memory_max_usage_in_bytes: 1806413824 >>>> --------------------------------- >>>> >>>> I made this patch to improve the readability of container >>>> information in the error log. >>>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>>> >>>> Could you please review it? >>>> Thanks a lot. >>>> >>>> Best regards, >>>> Jie >>>> >>>> >> From david.holmes at oracle.com Wed Mar 20 06:15:01 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 16:15:01 +1000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> Message-ID: <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> Hi Matthias, On 19/03/2019 11:53 pm, Baesken, Matthias wrote: > Hello, here is a new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ > > This moved the ppc/s390 virtualization related info output into the platform/cpu - specific coding . This is looking better though I'm struggling to keep all the dots connected in terms of the APIs - I can't quite see the pattern for what calls what and what overrides what. A couple of initial comments: 1. We already have a ton of cpuid related parsing in ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check virtualizations" and store the information for later use? 2. os_windows.cpp Why the indirection through os::win32::print_virtualization_info instead of just calling VM_Version::print_detected_virtualization directly? I'll keep trying to piece this together. :) Thanks, David ----- > > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 11. M?rz 2019 08:36 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Cc: Doerr, Martin >> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >> hs_error file on linux/windows x86_64 >> >> On 7/03/2019 3:13 am, Baesken, Matthias wrote: >>> Hello , could I push the latest version of 8219241 ? >> >> Sorry based on your earlier email I thought you were looking at doing >> the restructuring. I was then on vacation most of last week. >> >>> A change that moves the coding more into platform/cpu - specific coding >> (if this is wanted) >>> could be done when bringing in the AIX virtualization related info in >> another patch which I plan to do . >> >> If that is imminent then okay - I don't like the code as it is. >> >> Thanks, >> David >> > From david.holmes at oracle.com Wed Mar 20 06:18:49 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 16:18:49 +1000 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> Message-ID: On 20/03/2019 4:03 pm, Yasumasa Suenaga wrote: > Hi David, > > 2019?3?20?(?) 14:25 David Holmes : >> >> Hi Yasumasa, >> >> On 20/03/2019 3:01 pm, Yasumasa Suenaga wrote: >>> Thanks David! >>> >>> I uploaded new webrev. >>> This change affects Windows only, and I confirmed it works fine for >>> Linux x64 and MinGW64. >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ >> >> I this is for MINGW only then shouldn't the change be made within this >> section: > > Did you say we should determine with $(MINGW) whether libz.a is added > to LDFLAGS? > webrev is here: > http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.02/ > > The code is added to outside of the section you pointed. > > $(LIBRARIES) is available after OS-specific code. > So we need to change like webrev.02 to minimize changes. Or you could add LIBRARIES = .../libzip.a inside the existing MINGW section and then change the external LIBRARIES = ... to LIBRARIES += ... I'm unclear whether MINGW is only an issue on Linux when building for Windows or whether also of use on Windows itself? David > > Yasumasa > > >> ## OS = Linux ## >> ifeq ($(OS),Linux) >> ifneq ($(MINGW),) >> LIB_EXT = .dll >> CPPFLAGS += -I$(TARGET_DIR)/include >> LDFLAGS += -L$(TARGET_DIR)/lib >> OS=windows >> ifneq ($(findstring x86_64-,$(MINGW)),) >> ARCH=amd64 >> else >> ARCH=i386 >> endif >> CC = $(MINGW)-gcc >> CONFIGURE_ARGS= --host=$(MINGW) --target=$(MINGW) >> else #linux >> ? >> >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> 2019?3?20?(?) 11:25 David Holmes : >>>> >>>> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> 2019?3?20?(?) 10:38 David Holmes : >>>>>> >>>>>> Hi Yasumasa, >>>>>> >>>>>> I'm not familar with building hsdis, but if the only currnet problem is >>>>>> on Windows, why is the fix not restricted to only building on Windows? >>>>> >>>>> For Solaris and Linux, -lz is not added to LDFLAGS. I think it means >>>>> initial committer want to use zlib in binutils. >>>>> And zlib in binutils will be built implicitly in build phase in binutils. >>>>> >>>>>> Otherwise we need people who build hsdis on other platforms to comment >>>>>> on the appropriateness of the fix. >>>>> >>>>> If using different zlib is used by platforms is allowed, I will upload >>>>> webrev which affects Windows only. >>>>> What do you think? >>>> >>>> That seems preferable/simpler. Then you only need to know that your >>>> change works for non-MinGW64 environments. >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: >>>>>>> Hi Erik, David, >>>>>>> >>>>>>> I checked this change on Linux x64 and MinGW for Windows. >>>>>>> According to hsdis README, we need to use MinGW cross compiler to >>>>>>> build hsdis [1]. So I think Cygwin is not required. >>>>>>> >>>>>>> I do not have macOS and AIX. So I cannot check this change on them. >>>>>>> >>>>>>> BTW is hsdis included Java SE spec? >>>>>>> hsdis seems not to be included jtreg tests, and it is not contained in >>>>>>> OpenJDK binaries. >>>>>>> I think it is allowed even if we lack some test for hsdis if hsdis is >>>>>>> not required for Java SE. >>>>>>> If not so, I want sponsor(s) for this change. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 >>>>>>> >>>>>>> 2019?3?20?(?) 6:54 David Holmes : >>>>>>>> >>>>>>>> CC'ing hotspot-dev. I agree this needs to be checked on every platform >>>>>>>> affected. I can't comment on the fix itself. >>>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: >>>>>>>>> I think this needs to be reviewed by at least someone in hotspot who >>>>>>>>> regularly builds hsdis. I can't really comment on the validity of the >>>>>>>>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have >>>>>>>>> you at least verified the build on all the platforms which you affect >>>>>>>>> with this change (which would be at least Macos, AIX and Windows in a >>>>>>>>> normal Cygwin VS env)? >>>>>>>>> >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Please review this change: >>>>>>>>>> >>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 >>>>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ >>>>>>>>>> >>>>>>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with >>>>>>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions >>>>>>>>>> which are provided by zlib are unresolved. >>>>>>>>>> We need to link to zlib. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa From yasuenag at gmail.com Wed Mar 20 07:07:55 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 20 Mar 2019 16:07:55 +0900 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> Message-ID: Hi David, I tried to fix as you suggested, but it was failed. x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll -Ibuild/windows-amd64/include -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" -DLIB_EXT=\".dll\" -O hsdis.c -shared build/windows-amd64/zlib/libz.a build/windows-amd64/bfd/libbfd.a build/windows-amd64/opcodes/libopcodes.a build/windows-amd64/libiberty/libiberty.a build/windows-amd64/bfd/libbfd.a(compress.o):compress.c:(.text+0x5b): undefined reference to `inflateInit_' I tried to compile with moving libz.a to the last argument, it was succeeded. x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll -Ibuild/windows-amd64/include -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" -DLIB_EXT=\".dll\" -O hsdis.c -shared build/windows-amd64/bfd/libbfd.a build/windows-amd64/opcodes/libopcodes.a build/windows-amd64/libiberty/libiberty.a build/windows-amd64/zlib/libz.a I'm not unclear why it is failed. But order of libraries seems to be important. Yasumasa 2019?3?20?(?) 15:18 David Holmes : > > On 20/03/2019 4:03 pm, Yasumasa Suenaga wrote: > > Hi David, > > > > 2019?3?20?(?) 14:25 David Holmes : > >> > >> Hi Yasumasa, > >> > >> On 20/03/2019 3:01 pm, Yasumasa Suenaga wrote: > >>> Thanks David! > >>> > >>> I uploaded new webrev. > >>> This change affects Windows only, and I confirmed it works fine for > >>> Linux x64 and MinGW64. > >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ > >> > >> I this is for MINGW only then shouldn't the change be made within this > >> section: > > > > Did you say we should determine with $(MINGW) whether libz.a is added > > to LDFLAGS? > > webrev is here: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.02/ > > > > The code is added to outside of the section you pointed. > > > > $(LIBRARIES) is available after OS-specific code. > > So we need to change like webrev.02 to minimize changes. > > Or you could add > > LIBRARIES = .../libzip.a > > inside the existing MINGW section and then change the external > > LIBRARIES = ... > > to > > LIBRARIES += ... > > I'm unclear whether MINGW is only an issue on Linux when building for > Windows or whether also of use on Windows itself? > > David > > > > > Yasumasa > > > > > >> ## OS = Linux ## > >> ifeq ($(OS),Linux) > >> ifneq ($(MINGW),) > >> LIB_EXT = .dll > >> CPPFLAGS += -I$(TARGET_DIR)/include > >> LDFLAGS += -L$(TARGET_DIR)/lib > >> OS=windows > >> ifneq ($(findstring x86_64-,$(MINGW)),) > >> ARCH=amd64 > >> else > >> ARCH=i386 > >> endif > >> CC = $(MINGW)-gcc > >> CONFIGURE_ARGS= --host=$(MINGW) --target=$(MINGW) > >> else #linux > >> ? > >> > >> David > >> ----- > >> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>> 2019?3?20?(?) 11:25 David Holmes : > >>>> > >>>> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: > >>>>> Hi David, > >>>>> > >>>>> 2019?3?20?(?) 10:38 David Holmes : > >>>>>> > >>>>>> Hi Yasumasa, > >>>>>> > >>>>>> I'm not familar with building hsdis, but if the only currnet problem is > >>>>>> on Windows, why is the fix not restricted to only building on Windows? > >>>>> > >>>>> For Solaris and Linux, -lz is not added to LDFLAGS. I think it means > >>>>> initial committer want to use zlib in binutils. > >>>>> And zlib in binutils will be built implicitly in build phase in binutils. > >>>>> > >>>>>> Otherwise we need people who build hsdis on other platforms to comment > >>>>>> on the appropriateness of the fix. > >>>>> > >>>>> If using different zlib is used by platforms is allowed, I will upload > >>>>> webrev which affects Windows only. > >>>>> What do you think? > >>>> > >>>> That seems preferable/simpler. Then you only need to know that your > >>>> change works for non-MinGW64 environments. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Yasumasa > >>>>> > >>>>> > >>>>>> Thanks, > >>>>>> David > >>>>>> > >>>>>> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: > >>>>>>> Hi Erik, David, > >>>>>>> > >>>>>>> I checked this change on Linux x64 and MinGW for Windows. > >>>>>>> According to hsdis README, we need to use MinGW cross compiler to > >>>>>>> build hsdis [1]. So I think Cygwin is not required. > >>>>>>> > >>>>>>> I do not have macOS and AIX. So I cannot check this change on them. > >>>>>>> > >>>>>>> BTW is hsdis included Java SE spec? > >>>>>>> hsdis seems not to be included jtreg tests, and it is not contained in > >>>>>>> OpenJDK binaries. > >>>>>>> I think it is allowed even if we lack some test for hsdis if hsdis is > >>>>>>> not required for Java SE. > >>>>>>> If not so, I want sponsor(s) for this change. > >>>>>>> > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>> [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 > >>>>>>> > >>>>>>> 2019?3?20?(?) 6:54 David Holmes : > >>>>>>>> > >>>>>>>> CC'ing hotspot-dev. I agree this needs to be checked on every platform > >>>>>>>> affected. I can't comment on the fix itself. > >>>>>>>> > >>>>>>>> David > >>>>>>>> > >>>>>>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: > >>>>>>>>> I think this needs to be reviewed by at least someone in hotspot who > >>>>>>>>> regularly builds hsdis. I can't really comment on the validity of the > >>>>>>>>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have > >>>>>>>>> you at least verified the build on all the platforms which you affect > >>>>>>>>> with this change (which would be at least Macos, AIX and Windows in a > >>>>>>>>> normal Cygwin VS env)? > >>>>>>>>> > >>>>>>>>> /Erik > >>>>>>>>> > >>>>>>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: > >>>>>>>>>> Hi all, > >>>>>>>>>> > >>>>>>>>>> Please review this change: > >>>>>>>>>> > >>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 > >>>>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ > >>>>>>>>>> > >>>>>>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with > >>>>>>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions > >>>>>>>>>> which are provided by zlib are unresolved. > >>>>>>>>>> We need to link to zlib. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> Yasumasa From robbin.ehn at oracle.com Wed Mar 20 08:56:46 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 20 Mar 2019 09:56:46 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <9f115d6b-9f08-f5b2-b7d8-161cdb8718bf@redhat.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f115d6b-9f08-f5b2-b7d8-161cdb8718bf@redhat.com> Message-ID: <35fb1385-46c7-2545-d2c2-26d597a835d7@oracle.com> On 3/19/19 6:08 PM, Andrew Haley wrote: > On 3/19/19 4:52 PM, Doerr, Martin wrote: >> Not sure if the inline assembler code on x86 necessarily needs a "clobber memory" effect. >> I don't know what a C++ compiler is allowed to do if it doesn't know that the code has some kind of memory effect. > > Leave the memory clobber. No memory clobber means that you are telling the > compiler that this operation has no memory effect, which means that the > compiler can move the asm *before* you wrote out the changes to memory. > Thanks! /Robbin From robbin.ehn at oracle.com Wed Mar 20 08:58:55 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 20 Mar 2019 09:58:55 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <6fedfe57-1103-72e2-e55c-77f328d958ef@oracle.com> Hi Martin, On 3/19/19 5:52 PM, Doerr, Martin wrote: > Hi Robbin, > > some feedback on the platform code: > > orderAccess_aix_ppc.hpp should be same as for linux: > inline void OrderAccess::cross_modify_fence() > { inlasm_isync(); } Sorry, missed it, thanks. > > Not sure if the inline assembler code on x86 necessarily needs a "clobber memory" effect. > I don't know what a C++ compiler is allowed to do if it doesn't know that the code has some kind of memory effect. > > For ebx...edx, you could also use clobber if you want to make it shorter. > E.g. with "+a" to use eax as input and output: > int idx = 0; > __asm__ volatile ("cpuid " : "+a" (idx) : : "ebx", "ecx", "edx", "memory"); Yes, much better thanks! /Robbin > > Just a minor suggestion. Not important. > > My first impression of the shared changes is good, but I will need more time to think about it. > Thanks again for implementing lazy disarm to get rid of the performance penalty. > > Best regards, > Martin > > > > > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 19. M?rz 2019 17:44 > To: hotspot-dev at openjdk.java.net > Cc: David Holmes ; Andrew Haley ; Erik ?sterlund ; Doerr, Martin ; Reingruber, Richard > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi all, here is v2: > > Full: > http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ > Inc: > http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ > > The deprecated non-TLH path is missing the correct cross-mod fence, but since 13 > will be the last release containing that code path I think that is okay. > > We just leave native/native trans armed, and let the thread disarm on the > transition. If it disarms a new safepoint, we re-arm and stop at next polling > site, that avoids recursion. > > Passes stress and hs-t1-5. > As in the previous mail, performance numbers look good. > > Thanks, Robbin > > On 3/8/19 4:24 PM, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it can start >> executing updated code. E.g. an oop in the instruction stream can have been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is far away to >> guarantee that the thread executing the updated instruction stream sees the >> modification. >> >> What far away is and how far an update instruction stream is from a safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From robbin.ehn at oracle.com Wed Mar 20 09:31:59 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 20 Mar 2019 10:31:59 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> Hi, v3 updated with Martin's comments. Inc: http://cr.openjdk.java.net/~rehn/8220351/v3/inc/ Full: http://cr.openjdk.java.net/~rehn/8220351/v3/ David, are you okay with this model which does cross-mod fence on leaving a safe state? With the other model, always cross-mod fence going to java would mean we cannot disarm any thread since we don't know which state it's going to to next, if we want the same performance. Thanks, Robbin On 3/19/19 5:43 PM, Robbin Ehn wrote: > Hi all, here is v2: > > Full: > http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ > Inc: > http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ > > The deprecated non-TLH path is missing the correct cross-mod fence, but since 13 > will be the last release containing that code path I think that is okay. > > We just leave native/native trans armed, and let the thread disarm on the > transition. If it disarms a new safepoint, we re-arm and stop at next polling > site, that avoids recursion. > > Passes stress and hs-t1-5. > As in the previous mail, performance numbers look good. > > Thanks, Robbin > > On 3/8/19 4:24 PM, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it can start >> executing updated code. E.g. an oop in the instruction stream can have been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is far away to >> guarantee that the thread executing the updated instruction stream sees the >> modification. >> >> What far away is and how far an update instruction stream is from a safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From david.holmes at oracle.com Wed Mar 20 10:24:45 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Mar 2019 20:24:45 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> Message-ID: On 20/03/2019 7:31 pm, Robbin Ehn wrote: > Hi, v3 updated with Martin's comments. > > Inc: > http://cr.openjdk.java.net/~rehn/8220351/v3/inc/ > Full: > http://cr.openjdk.java.net/~rehn/8220351/v3/ > > David, are you okay with this model which does cross-mod fence on > leaving a safe state? With the other model, always cross-mod fence > going to java would mean we cannot disarm any thread since we don't > know which state it's going to to next, if we want the same > performance. Sorry Robbin but I can't follow the new approach just from the code changes. As I wrote earlier I'd like to see the stat-space enumerated here to show why pushing this down to the "safe states" is sufficient (and not introducing redundant fences) - e.g we fence when going from _thread_blocked to _thread_in_vm which covers the case if we then return to _thread_in_java; but if we're returning to native then we don't need the fence. I don't understand at all what your changes to the safepoint/handshake disarming logic are doing or how they relate to all this. Can you write up the design in the bug report please. Thanks, David ----- > > Thanks, Robbin > > On 3/19/19 5:43 PM, Robbin Ehn wrote: >> Hi all, here is v2: >> >> Full: >> http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ >> Inc: >> http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ >> >> The deprecated non-TLH path is missing the correct cross-mod fence, >> but since 13 >> will be the last release containing that code path I think that is okay. >> >> We just leave native/native trans armed, and let the thread disarm on the >> transition. If it disarms a new safepoint, we re-arm and stop at next >> polling >> site, that avoids recursion. >> >> Passes stress and hs-t1-5. >> As in the previous mail, performance numbers look good. >> >> Thanks, Robbin >> >> On 3/8/19 4:24 PM, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> After a JavaThread have been in a safepoint/(handshake) safe state it >>> can start >>> executing updated code. E.g. an oop in the instruction stream can >>> have been >>> updated. >>> >>> Most hardware's require a barrier or that the code cross modified is >>> far away to >>> guarantee that the thread executing the updated instruction stream >>> sees the >>> modification. >>> >>> What far away is and how far an update instruction stream is from a >>> safepoint >>> safe state is not clear. >>> >>> To be compliant with those architectures an instruction stream >>> barrier must be >>> added when leaving the safepoint safe state. >>> >>> There may be crashes today due to this missing barrier. >>> A new CPU with deeper pipeline or changes to the VM which moves a >>> safepoint safe >>> state closer to a nmethod can increase changes of a crash. >>> >>> A few benchmarks will see a performance regression with the extra >>> serializing, >>> such as Octane-Crypto -5% (x86). >>> >>> With a much more elaborate fix, such as changing the local poll to >>> have more >>> than two states (armed/disarmed), it would be possible to only emit such >>> instruction stream barrier when the poll have been armed while the >>> thread was >>> safepoint safe. >>> >>> Passes t1-3 on our platforms, and test built what I can. >>> >>> Thanks, Robbin From fw at deneb.enyo.de Wed Mar 20 14:14:29 2019 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed, 20 Mar 2019 15:14:29 +0100 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: (David Holmes's message of "Wed, 20 Mar 2019 16:18:49 +1000") References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> Message-ID: <87ftrh3a62.fsf@mid.deneb.enyo.de> * David Holmes: > I'm unclear whether MINGW is only an issue on Linux when building for > Windows or whether also of use on Windows itself? I suspect it depends on how binutils has been configured. If you have enabled the shared libbfd, you won't need to specify -lz because libbfd loads it automatically. From bob.vandette at oracle.com Wed Mar 20 14:40:56 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 20 Mar 2019 10:40:56 -0400 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> Message-ID: <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> I agree with David on the failed versus not supported. Make sure to run the container jtreg tests since these might rely on the current output formating. Bob. > On Mar 18, 2019, at 5:43 PM, David Holmes wrote: > > Hi Jie, > > On 14/03/2019 8:00 pm, Jie Fu wrote: >> Hi all, >> Bug: https://bugs.openjdk.java.net/browse/JDK-8220658 >> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ > > This seems a reasonable improvement, but I have a couple of suggestions. To avoid so much duplication you can change this structure: > > if (i > 0) { > st->print("cpu_quota: %d\n", i); > } else { > st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); > } > > to this: > > st->print("cpu_quota: "); > if (i > 0) { > st->print("%d\n", i); > } else { > st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); > } > > Also I suggest replacing "failed" with "not supported". > > Finally one minor typo: s/"no share"/"no shares"/ > > Thanks, > David > ----- > >> In the VM's error log, the container information was dumpped as: >> --------------------------------- >> container (cgroup) information: >> container_type: cgroupv1 >> cpu_cpuset_cpus: 0-11 >> cpu_memory_nodes: 0 >> active_processor_count: 12 >> cpu_quota: -1 >> cpu_period: 100000 >> cpu_shares: -1 >> memory_limit_in_bytes: -1 >> memory_and_swap_limit_in_bytes: -2 >> memory_soft_limit_in_bytes: -1 >> memory_usage_in_bytes: 11690164224 >> memory_max_usage_in_bytes: 1806413824 >> --------------------------------- >> Items such as "memory_limit_in_bytes" and "memory_and_swap_limit_in_bytes" are difficult to understand. >> It might be more human-readable if we could dump it like this: >> --------------------------------- >> container (cgroup) information: >> container_type: cgroupv1 >> cpu_cpuset_cpus: 0-11 >> cpu_memory_nodes: 0 >> active_processor_count: 12 >> cpu_quota: no quota >> cpu_period: 100000 >> cpu_shares: no share >> memory_limit_in_bytes: unlimited >> memory_and_swap_limit_in_bytes: failed >> memory_soft_limit_in_bytes: unlimited >> memory_usage_in_bytes: 11690164224 >> memory_max_usage_in_bytes: 1806413824 >> --------------------------------- >> I made this patch to improve the readability of container information in the error log. >> The change has been tested with hotspot:tier1 on Linux/x86-64. >> Could you please review it? >> Thanks a lot. >> Best regards, >> Jie From ioi.lam at oracle.com Wed Mar 20 16:11:05 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 20 Mar 2019 09:11:05 -0700 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> Message-ID: That's the problem with ld and static libraries. If libbfd.a depends on libz.a, you must put libbfd.a before libz.a. Since this is a rarely touched Makefile, I think what you're doing in webrev.02 is fine. We can refactor it later if it becomes more unwieldy. Thanks - Ioi On 3/20/19 12:07 AM, Yasumasa Suenaga wrote: > Hi David, > > I tried to fix as you suggested, but it was failed. > > x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll > -Ibuild/windows-amd64/include > -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include > -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd > -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" > -DLIB_EXT=\".dll\" -O hsdis.c -shared build/windows-amd64/zlib/libz.a > build/windows-amd64/bfd/libbfd.a > build/windows-amd64/opcodes/libopcodes.a > build/windows-amd64/libiberty/libiberty.a > build/windows-amd64/bfd/libbfd.a(compress.o):compress.c:(.text+0x5b): > undefined reference to `inflateInit_' > > I tried to compile with moving libz.a to the last argument, it was succeeded. > > x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll > -Ibuild/windows-amd64/include > -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include > -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd > -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" > -DLIB_EXT=\".dll\" -O hsdis.c -shared > build/windows-amd64/bfd/libbfd.a > build/windows-amd64/opcodes/libopcodes.a > build/windows-amd64/libiberty/libiberty.a > build/windows-amd64/zlib/libz.a > > I'm not unclear why it is failed. > But order of libraries seems to be important. > > > Yasumasa > > > 2019?3?20?(?) 15:18 David Holmes : >> On 20/03/2019 4:03 pm, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> 2019?3?20?(?) 14:25 David Holmes : >>>> Hi Yasumasa, >>>> >>>> On 20/03/2019 3:01 pm, Yasumasa Suenaga wrote: >>>>> Thanks David! >>>>> >>>>> I uploaded new webrev. >>>>> This change affects Windows only, and I confirmed it works fine for >>>>> Linux x64 and MinGW64. >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ >>>> I this is for MINGW only then shouldn't the change be made within this >>>> section: >>> Did you say we should determine with $(MINGW) whether libz.a is added >>> to LDFLAGS? >>> webrev is here: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.02/ >>> >>> The code is added to outside of the section you pointed. >>> >>> $(LIBRARIES) is available after OS-specific code. >>> So we need to change like webrev.02 to minimize changes. >> Or you could add >> >> LIBRARIES = .../libzip.a >> >> inside the existing MINGW section and then change the external >> >> LIBRARIES = ... >> >> to >> >> LIBRARIES += ... >> >> I'm unclear whether MINGW is only an issue on Linux when building for >> Windows or whether also of use on Windows itself? >> >> David >> >>> Yasumasa >>> >>> >>>> ## OS = Linux ## >>>> ifeq ($(OS),Linux) >>>> ifneq ($(MINGW),) >>>> LIB_EXT = .dll >>>> CPPFLAGS += -I$(TARGET_DIR)/include >>>> LDFLAGS += -L$(TARGET_DIR)/lib >>>> OS=windows >>>> ifneq ($(findstring x86_64-,$(MINGW)),) >>>> ARCH=amd64 >>>> else >>>> ARCH=i386 >>>> endif >>>> CC = $(MINGW)-gcc >>>> CONFIGURE_ARGS= --host=$(MINGW) --target=$(MINGW) >>>> else #linux >>>> ? >>>> >>>> David >>>> ----- >>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> 2019?3?20?(?) 11:25 David Holmes : >>>>>> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> 2019?3?20?(?) 10:38 David Holmes : >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> I'm not familar with building hsdis, but if the only currnet problem is >>>>>>>> on Windows, why is the fix not restricted to only building on Windows? >>>>>>> For Solaris and Linux, -lz is not added to LDFLAGS. I think it means >>>>>>> initial committer want to use zlib in binutils. >>>>>>> And zlib in binutils will be built implicitly in build phase in binutils. >>>>>>> >>>>>>>> Otherwise we need people who build hsdis on other platforms to comment >>>>>>>> on the appropriateness of the fix. >>>>>>> If using different zlib is used by platforms is allowed, I will upload >>>>>>> webrev which affects Windows only. >>>>>>> What do you think? >>>>>> That seems preferable/simpler. Then you only need to know that your >>>>>> change works for non-MinGW64 environments. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi Erik, David, >>>>>>>>> >>>>>>>>> I checked this change on Linux x64 and MinGW for Windows. >>>>>>>>> According to hsdis README, we need to use MinGW cross compiler to >>>>>>>>> build hsdis [1]. So I think Cygwin is not required. >>>>>>>>> >>>>>>>>> I do not have macOS and AIX. So I cannot check this change on them. >>>>>>>>> >>>>>>>>> BTW is hsdis included Java SE spec? >>>>>>>>> hsdis seems not to be included jtreg tests, and it is not contained in >>>>>>>>> OpenJDK binaries. >>>>>>>>> I think it is allowed even if we lack some test for hsdis if hsdis is >>>>>>>>> not required for Java SE. >>>>>>>>> If not so, I want sponsor(s) for this change. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 >>>>>>>>> >>>>>>>>> 2019?3?20?(?) 6:54 David Holmes : >>>>>>>>>> CC'ing hotspot-dev. I agree this needs to be checked on every platform >>>>>>>>>> affected. I can't comment on the fix itself. >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: >>>>>>>>>>> I think this needs to be reviewed by at least someone in hotspot who >>>>>>>>>>> regularly builds hsdis. I can't really comment on the validity of the >>>>>>>>>>> patch as I'm unfamiliar with both hsdis as well as this makefile. Have >>>>>>>>>>> you at least verified the build on all the platforms which you affect >>>>>>>>>>> with this change (which would be at least Macos, AIX and Windows in a >>>>>>>>>>> normal Cygwin VS env)? >>>>>>>>>>> >>>>>>>>>>> /Erik >>>>>>>>>>> >>>>>>>>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Please review this change: >>>>>>>>>>>> >>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 >>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with >>>>>>>>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some functions >>>>>>>>>>>> which are provided by zlib are unresolved. >>>>>>>>>>>> We need to link to zlib. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa From dthomson at google.com Wed Mar 20 16:20:36 2019 From: dthomson at google.com (Derek Thomson) Date: Wed, 20 Mar 2019 09:20:36 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> Message-ID: Thanks Vladimir, When I follow those instructions for testing I get a failure with the defpath extension. I'm going to ask JC Beyler here, too. I can't figure it out - it looks like it uses @command to me. I'm using Mercurial 4.7.2 *** failed to import extension defpath from /usr/local/google/home/dthomson/Downloads/defpath.py: missing attributes: norepo, optionalrepo, inferrepo *** (use @command decorator to register 'defpath') On Tue, Mar 19, 2019 at 1:39 PM Vladimir Kozlov wrote: > Hi Derek, > > On 3/19/19 11:08 AM, Derek Thomson wrote: > > Changed uint to bool throughout. Could someone review my change please? > > Looks good to me. > > > Also, what tests should I run to validate this? > > You can use submit repo to do build and tier1 testing on all platforms > supported by Oracle: > https://wiki.openjdk.java.net/display/Build/Submit+Repo > > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8157372/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8157372 > > > > Thanks, > Vladimir > From shade at redhat.com Wed Mar 20 16:27:53 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 20 Mar 2019 17:27:53 +0100 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> Message-ID: <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> On 3/20/19 5:20 PM, Derek Thomson wrote: > Thanks Vladimir, > > When I follow those instructions for testing I get a failure with the > defpath extension. I'm going to ask JC Beyler here, too. I can't figure it > out - it looks like it uses @command to me. I'm using Mercurial 4.7.2 defpath might have compatibility problems. What it does is putting default-push to .hgrc. It was tedious with forests, but with jdk10+ monorepo you can do it yourself once: $ cat .hg/hgrc [paths] default = https://hg.openjdk.java.net/jdk/submit default-push = ssh://shade at hg.openjdk.java.net/jdk/submit -Aleksey From dthomson at google.com Wed Mar 20 16:33:57 2019 From: dthomson at google.com (Derek Thomson) Date: Wed, 20 Mar 2019 09:33:57 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> Message-ID: Thanks Aleksey - does this mean I need a userid? I don't have one. I've been following this guide and can't find any mention of how to do that (or anywhere else): https://openjdk.java.net/contribute/ If I need it, can I get one? On Wed, Mar 20, 2019 at 9:28 AM Aleksey Shipilev wrote: > On 3/20/19 5:20 PM, Derek Thomson wrote: > > Thanks Vladimir, > > > > When I follow those instructions for testing I get a failure with the > > defpath extension. I'm going to ask JC Beyler here, too. I can't figure > it > > out - it looks like it uses @command to me. I'm using Mercurial 4.7.2 > > defpath might have compatibility problems. What it does is putting > default-push to .hgrc. It was > tedious with forests, but with jdk10+ monorepo you can do it yourself once: > > $ cat .hg/hgrc > [paths] > default = https://hg.openjdk.java.net/jdk/submit > default-push = ssh://shade at hg.openjdk.java.net/jdk/submit > > -Aleksey > > From shade at redhat.com Wed Mar 20 16:37:23 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 20 Mar 2019 17:37:23 +0100 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> Message-ID: <7e3b31a1-e0a0-b156-09f2-eef737268ca8@redhat.com> On 3/20/19 5:33 PM, Derek Thomson wrote: > Thanks Aleksey - does this mean I need a userid? I don't have one. I've been following this guide > and can't find any mention of how to do that (or anywhere else): > https://openjdk.java.net/contribute/ If I need it, can I get one? Yes, you need OpenJDK id (and maybe Committer privileges, not sure). JC certainly has it, you can ask him: http://openjdk.java.net/census#jcbeyler -Aleksey From dthomson at google.com Wed Mar 20 16:39:15 2019 From: dthomson at google.com (Derek Thomson) Date: Wed, 20 Mar 2019 09:39:15 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: <7e3b31a1-e0a0-b156-09f2-eef737268ca8@redhat.com> References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> <7e3b31a1-e0a0-b156-09f2-eef737268ca8@redhat.com> Message-ID: I figured, I'll bug him to run the tests then as I suspect you'll need Committer privileges. Thanks again, that saves me a lot of time trying to make this work. On Wed, Mar 20, 2019 at 9:37 AM Aleksey Shipilev wrote: > On 3/20/19 5:33 PM, Derek Thomson wrote: > > Thanks Aleksey - does this mean I need a userid? I don't have one. I've > been following this guide > > and can't find any mention of how to do that (or anywhere else): > > https://openjdk.java.net/contribute/ If I need it, can I get one? > > Yes, you need OpenJDK id (and maybe Committer privileges, not sure). JC > certainly has it, you can > ask him: http://openjdk.java.net/census#jcbeyler > > -Aleksey > > From martin.doerr at sap.com Wed Mar 20 17:42:55 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 20 Mar 2019 17:42:55 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> Message-ID: Hi David and Robbin, > As I wrote earlier I'd like to see the stat-space enumerated > here to show why pushing this down to the "safe states" is sufficient I was also struggling a bit with the placement of the cross_modify_fence. But there's the idea to replace the suspend flags by handshakes. After that, it seems to me that there will be very few cross_modify_fence usages left which look nicely placed to me. So if this is the plan, I should be happy with it ?? > I don't understand at all what your changes to the safepoint/handshake > disarming logic are doing or how they relate to all this. That was basically my suggestion. The first proposal was to introduce CPUID in native wrapper which is costly from performance point of view. Every Java thread returning from native would have to execute it even if there was no safepoint. And there was no way to detect if there was one. So my idea was to change the disarming logic such that every thread returning from native must detect if there was a safepoint. This is done by moving the task of disarming from the VM thread to the thread returning from native. I guess Robbin will also add a description. Best regards, Martin -----Original Message----- From: David Holmes Sent: Mittwoch, 20. M?rz 2019 11:25 To: Robbin Ehn ; hotspot-dev at openjdk.java.net; Doerr, Martin Cc: aph at redhat.com; Erik ?sterlund ; Reingruber, Richard Subject: Re: RFR(m): 8220351: Cross-modifying code On 20/03/2019 7:31 pm, Robbin Ehn wrote: > Hi, v3 updated with Martin's comments. > > Inc: > http://cr.openjdk.java.net/~rehn/8220351/v3/inc/ > Full: > http://cr.openjdk.java.net/~rehn/8220351/v3/ > > David, are you okay with this model which does cross-mod fence on > leaving a safe state? With the other model, always cross-mod fence > going to java would mean we cannot disarm any thread since we don't > know which state it's going to to next, if we want the same > performance. Sorry Robbin but I can't follow the new approach just from the code changes. As I wrote earlier I'd like to see the stat-space enumerated here to show why pushing this down to the "safe states" is sufficient (and not introducing redundant fences) - e.g we fence when going from _thread_blocked to _thread_in_vm which covers the case if we then return to _thread_in_java; but if we're returning to native then we don't need the fence. I don't understand at all what your changes to the safepoint/handshake disarming logic are doing or how they relate to all this. Can you write up the design in the bug report please. Thanks, David ----- > > Thanks, Robbin > > On 3/19/19 5:43 PM, Robbin Ehn wrote: >> Hi all, here is v2: >> >> Full: >> http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ >> Inc: >> http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ >> >> The deprecated non-TLH path is missing the correct cross-mod fence, >> but since 13 >> will be the last release containing that code path I think that is okay. >> >> We just leave native/native trans armed, and let the thread disarm on the >> transition. If it disarms a new safepoint, we re-arm and stop at next >> polling >> site, that avoids recursion. >> >> Passes stress and hs-t1-5. >> As in the previous mail, performance numbers look good. >> >> Thanks, Robbin >> >> On 3/8/19 4:24 PM, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> After a JavaThread have been in a safepoint/(handshake) safe state it >>> can start >>> executing updated code. E.g. an oop in the instruction stream can >>> have been >>> updated. >>> >>> Most hardware's require a barrier or that the code cross modified is >>> far away to >>> guarantee that the thread executing the updated instruction stream >>> sees the >>> modification. >>> >>> What far away is and how far an update instruction stream is from a >>> safepoint >>> safe state is not clear. >>> >>> To be compliant with those architectures an instruction stream >>> barrier must be >>> added when leaving the safepoint safe state. >>> >>> There may be crashes today due to this missing barrier. >>> A new CPU with deeper pipeline or changes to the VM which moves a >>> safepoint safe >>> state closer to a nmethod can increase changes of a crash. >>> >>> A few benchmarks will see a performance regression with the extra >>> serializing, >>> such as Octane-Crypto -5% (x86). >>> >>> With a much more elaborate fix, such as changing the local poll to >>> have more >>> than two states (armed/disarmed), it would be possible to only emit such >>> instruction stream barrier when the poll have been armed while the >>> thread was >>> safepoint safe. >>> >>> Passes t1-3 on our platforms, and test built what I can. >>> >>> Thanks, Robbin From thomas.stuefe at gmail.com Wed Mar 20 19:19:32 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 20 Mar 2019 20:19:32 +0100 Subject: RFC: Draft JEP: Improved Metaspace allocator Message-ID: Dear all, I would like to hear what you think about the following proposal: https://bugs.openjdk.java.net/browse/JDK-8221173 Do you consider this worthwhile? I debated with myself whether a JEP is the correct umbrella for it. In the past I did larger changes like this without JEP but in this case I feel better with a confirmed and sponsored JEP behind me - from a certain size on it just feels more appropriate. As usual, I am not sure where to put Metaspace-related proposals to catch all interested people, so I start with hs-dev. Please feel free to redirect if needed. Thanks a lot for your time, Thomas From david.holmes at oracle.com Wed Mar 20 22:23:43 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Mar 2019 08:23:43 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> Message-ID: <3c2145f8-c56b-e949-ac4e-8fc16a099b43@oracle.com> Hi Martin, On 21/03/2019 3:42 am, Doerr, Martin wrote: > Hi David and Robbin, > >> As I wrote earlier I'd like to see the stat-space enumerated >> here to show why pushing this down to the "safe states" is sufficient > I was also struggling a bit with the placement of the cross_modify_fence. > But there's the idea to replace the suspend flags by handshakes. > After that, it seems to me that there will be very few cross_modify_fence usages left which look nicely placed to me. > So if this is the plan, I should be happy with it ?? I'm really not sure how using handshakes for suspend might simplify the current problem. >> I don't understand at all what your changes to the safepoint/handshake >> disarming logic are doing or how they relate to all this. > That was basically my suggestion. > The first proposal was to introduce CPUID in native wrapper which is costly from performance point of view. > Every Java thread returning from native would have to execute it even if there was no safepoint. And there was no way to detect if there was one. > So my idea was to change the disarming logic such that every thread returning from native must detect if there was a safepoint. > This is done by moving the task of disarming from the VM thread to the thread returning from native. Doesn't the safepoint counter provide a means to detect if there has been a safepoint? I worry about making this kind of tweak to the safepoint mechanism. I now have to convince myself that a self-disarm can't race with a re-arm from the VMThread. Thanks, David ----- > I guess Robbin will also add a description. > > Best regards, > Martin > > > -----Original Message----- > From: David Holmes > Sent: Mittwoch, 20. M?rz 2019 11:25 > To: Robbin Ehn ; hotspot-dev at openjdk.java.net; Doerr, Martin > Cc: aph at redhat.com; Erik ?sterlund ; Reingruber, Richard > Subject: Re: RFR(m): 8220351: Cross-modifying code > > On 20/03/2019 7:31 pm, Robbin Ehn wrote: >> Hi, v3 updated with Martin's comments. >> >> Inc: >> http://cr.openjdk.java.net/~rehn/8220351/v3/inc/ >> Full: >> http://cr.openjdk.java.net/~rehn/8220351/v3/ >> >> David, are you okay with this model which does cross-mod fence on >> leaving a safe state? With the other model, always cross-mod fence >> going to java would mean we cannot disarm any thread since we don't >> know which state it's going to to next, if we want the same >> performance. > > Sorry Robbin but I can't follow the new approach just from the code > changes. As I wrote earlier I'd like to see the stat-space enumerated > here to show why pushing this down to the "safe states" is sufficient > (and not introducing redundant fences) - e.g we fence when going from > _thread_blocked to _thread_in_vm which covers the case if we then return > to _thread_in_java; but if we're returning to native then we don't need > the fence. > > I don't understand at all what your changes to the safepoint/handshake > disarming logic are doing or how they relate to all this. Can you write > up the design in the bug report please. > > Thanks, > David > ----- > > >> >> Thanks, Robbin >> >> On 3/19/19 5:43 PM, Robbin Ehn wrote: >>> Hi all, here is v2: >>> >>> Full: >>> http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ >>> Inc: >>> http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ >>> >>> The deprecated non-TLH path is missing the correct cross-mod fence, >>> but since 13 >>> will be the last release containing that code path I think that is okay. >>> >>> We just leave native/native trans armed, and let the thread disarm on the >>> transition. If it disarms a new safepoint, we re-arm and stop at next >>> polling >>> site, that avoids recursion. >>> >>> Passes stress and hs-t1-5. >>> As in the previous mail, performance numbers look good. >>> >>> Thanks, Robbin >>> >>> On 3/8/19 4:24 PM, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>> Changeset: >>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>> >>>> After a JavaThread have been in a safepoint/(handshake) safe state it >>>> can start >>>> executing updated code. E.g. an oop in the instruction stream can >>>> have been >>>> updated. >>>> >>>> Most hardware's require a barrier or that the code cross modified is >>>> far away to >>>> guarantee that the thread executing the updated instruction stream >>>> sees the >>>> modification. >>>> >>>> What far away is and how far an update instruction stream is from a >>>> safepoint >>>> safe state is not clear. >>>> >>>> To be compliant with those architectures an instruction stream >>>> barrier must be >>>> added when leaving the safepoint safe state. >>>> >>>> There may be crashes today due to this missing barrier. >>>> A new CPU with deeper pipeline or changes to the VM which moves a >>>> safepoint safe >>>> state closer to a nmethod can increase changes of a crash. >>>> >>>> A few benchmarks will see a performance regression with the extra >>>> serializing, >>>> such as Octane-Crypto -5% (x86). >>>> >>>> With a much more elaborate fix, such as changing the local poll to >>>> have more >>>> than two states (armed/disarmed), it would be possible to only emit such >>>> instruction stream barrier when the poll have been armed while the >>>> thread was >>>> safepoint safe. >>>> >>>> Passes t1-3 on our platforms, and test built what I can. >>>> >>>> Thanks, Robbin From fujie at loongson.cn Thu Mar 21 01:06:42 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 21 Mar 2019 09:06:42 +0800 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> Message-ID: Thanks Bob. I'll update the patch and run the container jtreg tests as soon as possible. Best regards, Jie On 2019/3/20 ??10:40, Bob Vandette wrote: > I agree with David on the failed versus not supported. > > Make sure to run the container jtreg tests since these might rely on the current output > formating. > > Bob. > > >> On Mar 18, 2019, at 5:43 PM, David Holmes wrote: >> >> Hi Jie, >> >> On 14/03/2019 8:00 pm, Jie Fu wrote: >>> Hi all, >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8220658 >>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >> This seems a reasonable improvement, but I have a couple of suggestions. To avoid so much duplication you can change this structure: >> >> if (i > 0) { >> st->print("cpu_quota: %d\n", i); >> } else { >> st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); >> } >> >> to this: >> >> st->print("cpu_quota: "); >> if (i > 0) { >> st->print("%d\n", i); >> } else { >> st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); >> } >> >> Also I suggest replacing "failed" with "not supported". >> >> Finally one minor typo: s/"no share"/"no shares"/ >> >> Thanks, >> David >> ----- >> >>> In the VM's error log, the container information was dumpped as: >>> --------------------------------- >>> container (cgroup) information: >>> container_type: cgroupv1 >>> cpu_cpuset_cpus: 0-11 >>> cpu_memory_nodes: 0 >>> active_processor_count: 12 >>> cpu_quota: -1 >>> cpu_period: 100000 >>> cpu_shares: -1 >>> memory_limit_in_bytes: -1 >>> memory_and_swap_limit_in_bytes: -2 >>> memory_soft_limit_in_bytes: -1 >>> memory_usage_in_bytes: 11690164224 >>> memory_max_usage_in_bytes: 1806413824 >>> --------------------------------- >>> Items such as "memory_limit_in_bytes" and "memory_and_swap_limit_in_bytes" are difficult to understand. >>> It might be more human-readable if we could dump it like this: >>> --------------------------------- >>> container (cgroup) information: >>> container_type: cgroupv1 >>> cpu_cpuset_cpus: 0-11 >>> cpu_memory_nodes: 0 >>> active_processor_count: 12 >>> cpu_quota: no quota >>> cpu_period: 100000 >>> cpu_shares: no share >>> memory_limit_in_bytes: unlimited >>> memory_and_swap_limit_in_bytes: failed >>> memory_soft_limit_in_bytes: unlimited >>> memory_usage_in_bytes: 11690164224 >>> memory_max_usage_in_bytes: 1806413824 >>> --------------------------------- >>> I made this patch to improve the readability of container information in the error log. >>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>> Could you please review it? >>> Thanks a lot. >>> Best regards, >>> Jie From david.holmes at oracle.com Thu Mar 21 01:25:55 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Mar 2019 11:25:55 +1000 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> Message-ID: On 21/03/2019 11:06 am, Jie Fu wrote: > Thanks Bob. > I'll update the patch and run the container jtreg tests as soon as > possible. Bob was commenting on my original suggestion to change failed to "not supported" and indicated agreement. So I don't think there's any update to the patch needed ?? David > Best regards, > > Jie > > > On 2019/3/20 ??10:40, Bob Vandette wrote: >> I agree with David on the failed versus not supported. >> >> Make sure to run the container jtreg tests since these might rely on >> the current output >> formating. >> >> Bob. >> >> >>> On Mar 18, 2019, at 5:43 PM, David Holmes >>> wrote: >>> >>> Hi Jie, >>> >>> On 14/03/2019 8:00 pm, Jie Fu wrote: >>>> Hi all, >>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 >>>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >>> This seems a reasonable improvement, but I have a couple of >>> suggestions. To avoid so much duplication you can change this structure: >>> >>> ?? if (i > 0) { >>> ???? st->print("cpu_quota: %d\n", i); >>> ?? } else { >>> ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" : >>> "no quota"); >>> ?? } >>> >>> to this: >>> >>> ?? st->print("cpu_quota: "); >>> ?? if (i > 0) { >>> ????? st->print("%d\n", i); >>> ?? } else { >>> ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no quota"); >>> ?? } >>> >>> Also I suggest replacing "failed" with "not supported". >>> >>> Finally one minor typo: s/"no share"/"no shares"/ >>> >>> Thanks, >>> David >>> ----- >>> >>>> In the VM's error log, the container information was dumpped as: >>>> --------------------------------- >>>> container (cgroup) information: >>>> container_type: cgroupv1 >>>> cpu_cpuset_cpus: 0-11 >>>> cpu_memory_nodes: 0 >>>> active_processor_count: 12 >>>> cpu_quota: -1 >>>> cpu_period: 100000 >>>> cpu_shares: -1 >>>> memory_limit_in_bytes: -1 >>>> memory_and_swap_limit_in_bytes: -2 >>>> memory_soft_limit_in_bytes: -1 >>>> memory_usage_in_bytes: 11690164224 >>>> memory_max_usage_in_bytes: 1806413824 >>>> --------------------------------- >>>> Items such as "memory_limit_in_bytes" and >>>> "memory_and_swap_limit_in_bytes" are difficult to understand. >>>> It might be more human-readable if we could dump it like this: >>>> --------------------------------- >>>> container (cgroup) information: >>>> container_type: cgroupv1 >>>> cpu_cpuset_cpus: 0-11 >>>> cpu_memory_nodes: 0 >>>> active_processor_count: 12 >>>> cpu_quota: no quota >>>> cpu_period: 100000 >>>> cpu_shares: no share >>>> memory_limit_in_bytes: unlimited >>>> memory_and_swap_limit_in_bytes: failed >>>> memory_soft_limit_in_bytes: unlimited >>>> memory_usage_in_bytes: 11690164224 >>>> memory_max_usage_in_bytes: 1806413824 >>>> --------------------------------- >>>> I made this patch to improve the readability of container >>>> information in the error log. >>>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>>> Could you please review it? >>>> Thanks a lot. >>>> Best regards, >>>> Jie > From fujie at loongson.cn Thu Mar 21 01:38:40 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 21 Mar 2019 09:38:40 +0800 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> Message-ID: <8ba2ea2e-04f3-14f3-00db-4a7cf632fdfd@loongson.cn> Oh, I'm very sorry for that. I am not good at English. Thanks David. On 2019/3/21 ??9:25, David Holmes wrote: > On 21/03/2019 11:06 am, Jie Fu wrote: >> Thanks Bob. >> I'll update the patch and run the container jtreg tests as soon as >> possible. > > Bob was commenting on my original suggestion to change failed to "not > supported" and indicated agreement. So I don't think there's any > update to the patch needed ?? > > David > >> Best regards, >> >> Jie >> >> >> On 2019/3/20 ??10:40, Bob Vandette wrote: >>> I agree with David on the failed versus not supported. >>> >>> Make sure to run the container jtreg tests since these might rely on >>> the current output >>> formating. >>> >>> Bob. >>> >>> >>>> On Mar 18, 2019, at 5:43 PM, David Holmes >>>> wrote: >>>> >>>> Hi Jie, >>>> >>>> On 14/03/2019 8:00 pm, Jie Fu wrote: >>>>> Hi all, >>>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 >>>>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >>>> This seems a reasonable improvement, but I have a couple of >>>> suggestions. To avoid so much duplication you can change this >>>> structure: >>>> >>>> ?? if (i > 0) { >>>> ???? st->print("cpu_quota: %d\n", i); >>>> ?? } else { >>>> ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? "failed" >>>> : "no quota"); >>>> ?? } >>>> >>>> to this: >>>> >>>> ?? st->print("cpu_quota: "); >>>> ?? if (i > 0) { >>>> ????? st->print("%d\n", i); >>>> ?? } else { >>>> ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no >>>> quota"); >>>> ?? } >>>> >>>> Also I suggest replacing "failed" with "not supported". >>>> >>>> Finally one minor typo: s/"no share"/"no shares"/ >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> In the VM's error log, the container information was dumpped as: >>>>> --------------------------------- >>>>> container (cgroup) information: >>>>> container_type: cgroupv1 >>>>> cpu_cpuset_cpus: 0-11 >>>>> cpu_memory_nodes: 0 >>>>> active_processor_count: 12 >>>>> cpu_quota: -1 >>>>> cpu_period: 100000 >>>>> cpu_shares: -1 >>>>> memory_limit_in_bytes: -1 >>>>> memory_and_swap_limit_in_bytes: -2 >>>>> memory_soft_limit_in_bytes: -1 >>>>> memory_usage_in_bytes: 11690164224 >>>>> memory_max_usage_in_bytes: 1806413824 >>>>> --------------------------------- >>>>> Items such as "memory_limit_in_bytes" and >>>>> "memory_and_swap_limit_in_bytes" are difficult to understand. >>>>> It might be more human-readable if we could dump it like this: >>>>> --------------------------------- >>>>> container (cgroup) information: >>>>> container_type: cgroupv1 >>>>> cpu_cpuset_cpus: 0-11 >>>>> cpu_memory_nodes: 0 >>>>> active_processor_count: 12 >>>>> cpu_quota: no quota >>>>> cpu_period: 100000 >>>>> cpu_shares: no share >>>>> memory_limit_in_bytes: unlimited >>>>> memory_and_swap_limit_in_bytes: failed >>>>> memory_soft_limit_in_bytes: unlimited >>>>> memory_usage_in_bytes: 11690164224 >>>>> memory_max_usage_in_bytes: 1806413824 >>>>> --------------------------------- >>>>> I made this patch to improve the readability of container >>>>> information in the error log. >>>>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>>>> Could you please review it? >>>>> Thanks a lot. >>>>> Best regards, >>>>> Jie >> From david.holmes at oracle.com Thu Mar 21 05:07:51 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Mar 2019 15:07:51 +1000 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> Message-ID: <643e59a4-2f7b-35ed-9114-332b8215c7f7@oracle.com> On 21/03/2019 2:11 am, Ioi Lam wrote: > That's the problem with ld and static libraries. If libbfd.a depends on > libz.a, you must put libbfd.a before libz.a. > > Since this is a rarely touched Makefile, I think what you're doing in > webrev.02 is fine. We can refactor it later if it becomes more unwieldy. Okay I can live with webrev.02 as well. Thanks, David ----- > Thanks > - Ioi > > On 3/20/19 12:07 AM, Yasumasa Suenaga wrote: >> Hi David, >> >> I tried to fix as you suggested, but it was failed. >> >> x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll >> -Ibuild/windows-amd64/include >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd >> -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" >> -DLIB_EXT=\".dll\" -O hsdis.c -shared? build/windows-amd64/zlib/libz.a >> build/windows-amd64/bfd/libbfd.a >> build/windows-amd64/opcodes/libopcodes.a >> build/windows-amd64/libiberty/libiberty.a >> build/windows-amd64/bfd/libbfd.a(compress.o):compress.c:(.text+0x5b): >> undefined reference to `inflateInit_' >> >> I tried to compile with moving libz.a to the last argument, it was >> succeeded. >> >> x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll >> -Ibuild/windows-amd64/include >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd >> -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" >> -DLIB_EXT=\".dll\" -O hsdis.c -shared >> build/windows-amd64/bfd/libbfd.a >> build/windows-amd64/opcodes/libopcodes.a >> build/windows-amd64/libiberty/libiberty.a >> build/windows-amd64/zlib/libz.a >> >> I'm not unclear why it is failed. >> But order of libraries seems to be important. >> >> >> Yasumasa >> >> >> 2019?3?20?(?) 15:18 David Holmes : >>> On 20/03/2019 4:03 pm, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>> 2019?3?20?(?) 14:25 David Holmes : >>>>> Hi Yasumasa, >>>>> >>>>> On 20/03/2019 3:01 pm, Yasumasa Suenaga wrote: >>>>>> Thanks David! >>>>>> >>>>>> I uploaded new webrev. >>>>>> This change affects Windows only, and I confirmed it works fine for >>>>>> Linux x64 and MinGW64. >>>>>> ???? http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ >>>>> I this is for MINGW only then shouldn't the change be made within this >>>>> section: >>>> Did you say we should determine with $(MINGW) whether libz.a is added >>>> to LDFLAGS? >>>> webrev is here: >>>> ??? http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.02/ >>>> >>>> The code is added to outside of the section you pointed. >>>> >>>> $(LIBRARIES) is available after OS-specific code. >>>> So we need to change like webrev.02 to minimize changes. >>> Or you could add >>> >>> LIBRARIES = .../libzip.a >>> >>> inside the existing MINGW section and then change the external >>> >>> LIBRARIES = ... >>> >>> to >>> >>> LIBRARIES += ... >>> >>> I'm unclear whether MINGW is only an issue on Linux when building for >>> Windows or whether also of use on Windows itself? >>> >>> David >>> >>>> Yasumasa >>>> >>>> >>>>> ???? ## OS = Linux ## >>>>> ???? ifeq???????? ($(OS),Linux) >>>>> ?????? ifneq?????????? ($(MINGW),) >>>>> ???????? LIB_EXT????????? = .dll >>>>> ???????? CPPFLAGS += -I$(TARGET_DIR)/include >>>>> ???????? LDFLAGS += -L$(TARGET_DIR)/lib >>>>> ???????? OS=windows >>>>> ???????? ifneq?????????? ($(findstring x86_64-,$(MINGW)),) >>>>> ?????????? ARCH=amd64 >>>>> ???????? else >>>>> ?????????? ARCH=i386 >>>>> ???????? endif >>>>> ???????? CC?????????????? = $(MINGW)-gcc >>>>> ???????? CONFIGURE_ARGS= --host=$(MINGW) --target=$(MINGW) >>>>> ?????? else?? #linux >>>>> ? >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> 2019?3?20?(?) 11:25 David Holmes : >>>>>>> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> 2019?3?20?(?) 10:38 David Holmes : >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> I'm not familar with building hsdis, but if the only currnet >>>>>>>>> problem is >>>>>>>>> on Windows, why is the fix not restricted to only building on >>>>>>>>> Windows? >>>>>>>> For Solaris and Linux, -lz is not added to LDFLAGS. I think it >>>>>>>> means >>>>>>>> initial committer want to use zlib in binutils. >>>>>>>> And zlib in binutils will be built implicitly in build phase in >>>>>>>> binutils. >>>>>>>> >>>>>>>>> Otherwise we need people who build hsdis on other platforms to >>>>>>>>> comment >>>>>>>>> on the appropriateness of the fix. >>>>>>>> If using different zlib is used by platforms is allowed, I will >>>>>>>> upload >>>>>>>> webrev which affects Windows only. >>>>>>>> What do you think? >>>>>>> That seems preferable/simpler. Then you only need to know that your >>>>>>> change works for non-MinGW64 environments. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Erik, David, >>>>>>>>>> >>>>>>>>>> I checked this change on Linux x64 and MinGW for Windows. >>>>>>>>>> According to hsdis README, we need to use MinGW cross compiler to >>>>>>>>>> build hsdis [1]. So I think Cygwin is not required. >>>>>>>>>> >>>>>>>>>> I do not have macOS and AIX. So I cannot check this change on >>>>>>>>>> them. >>>>>>>>>> >>>>>>>>>> BTW is hsdis included Java SE spec? >>>>>>>>>> hsdis seems not to be included jtreg tests, and it is not >>>>>>>>>> contained in >>>>>>>>>> OpenJDK binaries. >>>>>>>>>> I think it is allowed even if we lack some test for hsdis if >>>>>>>>>> hsdis is >>>>>>>>>> not required for Java SE. >>>>>>>>>> If not so, I want sponsor(s) for this change. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2019?3?20?(?) 6:54 David Holmes : >>>>>>>>>>> CC'ing hotspot-dev. I agree this needs to be checked on every >>>>>>>>>>> platform >>>>>>>>>>> affected. I can't comment on the fix itself. >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: >>>>>>>>>>>> I think this needs to be reviewed by at least someone in >>>>>>>>>>>> hotspot who >>>>>>>>>>>> regularly builds hsdis. I can't really comment on the >>>>>>>>>>>> validity of the >>>>>>>>>>>> patch as I'm unfamiliar with both hsdis as well as this >>>>>>>>>>>> makefile. Have >>>>>>>>>>>> you at least verified the build on all the platforms which >>>>>>>>>>>> you affect >>>>>>>>>>>> with this change (which would be at least Macos, AIX and >>>>>>>>>>>> Windows in a >>>>>>>>>>>> normal Cygwin VS env)? >>>>>>>>>>>> >>>>>>>>>>>> /Erik >>>>>>>>>>>> >>>>>>>>>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>> >>>>>>>>>>>>> ??????? JBS: https://bugs.openjdk.java.net/browse/JDK-8220784 >>>>>>>>>>>>> ??????? webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with >>>>>>>>>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some >>>>>>>>>>>>> functions >>>>>>>>>>>>> which are provided by zlib are unresolved. >>>>>>>>>>>>> We need to link to zlib. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa > From yasuenag at gmail.com Thu Mar 21 05:27:10 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 21 Mar 2019 14:27:10 +0900 Subject: RFR: 8220784: hsdis cannot be built with MinGW64 In-Reply-To: <643e59a4-2f7b-35ed-9114-332b8215c7f7@oracle.com> References: <85c6c262-9749-72f2-ca8b-30df91e178c8@oracle.com> <54f5b9eb-731d-592c-d358-0209cf910992@oracle.com> <7462f8fb-c28c-fed9-765c-b9f940082585@oracle.com> <300cd66e-8ccb-3f9f-9404-4f8f0b49ed04@oracle.com> <36f6d7c6-ed38-6ce5-daa7-77624032a461@oracle.com> <643e59a4-2f7b-35ed-9114-332b8215c7f7@oracle.com> Message-ID: Thanks Ioi, David! I will push webrev.02 later. Yasumasa 2019?3?21?(?) 14:08 David Holmes : > On 21/03/2019 2:11 am, Ioi Lam wrote: > > That's the problem with ld and static libraries. If libbfd.a depends on > > libz.a, you must put libbfd.a before libz.a. > > > > Since this is a rarely touched Makefile, I think what you're doing in > > webrev.02 is fine. We can refactor it later if it becomes more unwieldy. > > Okay I can live with webrev.02 as well. > > Thanks, > David > ----- > > > Thanks > > - Ioi > > > > On 3/20/19 12:07 AM, Yasumasa Suenaga wrote: > >> Hi David, > >> > >> I tried to fix as you suggested, but it was failed. > >> > >> x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll > >> -Ibuild/windows-amd64/include > >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include > >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd > >> -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" > >> -DLIB_EXT=\".dll\" -O hsdis.c -shared build/windows-amd64/zlib/libz.a > >> build/windows-amd64/bfd/libbfd.a > >> build/windows-amd64/opcodes/libopcodes.a > >> build/windows-amd64/libiberty/libiberty.a > >> build/windows-amd64/bfd/libbfd.a(compress.o):compress.c:(.text+0x5b): > >> undefined reference to `inflateInit_' > >> > >> I tried to compile with moving libz.a to the last argument, it was > >> succeeded. > >> > >> x86_64-w64-mingw32-gcc -o build/windows-amd64/hsdis-amd64.dll > >> -Ibuild/windows-amd64/include > >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/include > >> -I/home/ysuenaga/rpmbuild/BUILD/binutils-2.31.1/bfd > >> -Ibuild/windows-amd64/bfd -DLIBARCH_amd64 -DLIBARCH=\"amd64\" > >> -DLIB_EXT=\".dll\" -O hsdis.c -shared > >> build/windows-amd64/bfd/libbfd.a > >> build/windows-amd64/opcodes/libopcodes.a > >> build/windows-amd64/libiberty/libiberty.a > >> build/windows-amd64/zlib/libz.a > >> > >> I'm not unclear why it is failed. > >> But order of libraries seems to be important. > >> > >> > >> Yasumasa > >> > >> > >> 2019?3?20?(?) 15:18 David Holmes : > >>> On 20/03/2019 4:03 pm, Yasumasa Suenaga wrote: > >>>> Hi David, > >>>> > >>>> 2019?3?20?(?) 14:25 David Holmes : > >>>>> Hi Yasumasa, > >>>>> > >>>>> On 20/03/2019 3:01 pm, Yasumasa Suenaga wrote: > >>>>>> Thanks David! > >>>>>> > >>>>>> I uploaded new webrev. > >>>>>> This change affects Windows only, and I confirmed it works fine for > >>>>>> Linux x64 and MinGW64. > >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.01/ > >>>>> I this is for MINGW only then shouldn't the change be made within > this > >>>>> section: > >>>> Did you say we should determine with $(MINGW) whether libz.a is added > >>>> to LDFLAGS? > >>>> webrev is here: > >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.02/ > >>>> > >>>> The code is added to outside of the section you pointed. > >>>> > >>>> $(LIBRARIES) is available after OS-specific code. > >>>> So we need to change like webrev.02 to minimize changes. > >>> Or you could add > >>> > >>> LIBRARIES = .../libzip.a > >>> > >>> inside the existing MINGW section and then change the external > >>> > >>> LIBRARIES = ... > >>> > >>> to > >>> > >>> LIBRARIES += ... > >>> > >>> I'm unclear whether MINGW is only an issue on Linux when building for > >>> Windows or whether also of use on Windows itself? > >>> > >>> David > >>> > >>>> Yasumasa > >>>> > >>>> > >>>>> ## OS = Linux ## > >>>>> ifeq ($(OS),Linux) > >>>>> ifneq ($(MINGW),) > >>>>> LIB_EXT = .dll > >>>>> CPPFLAGS += -I$(TARGET_DIR)/include > >>>>> LDFLAGS += -L$(TARGET_DIR)/lib > >>>>> OS=windows > >>>>> ifneq ($(findstring x86_64-,$(MINGW)),) > >>>>> ARCH=amd64 > >>>>> else > >>>>> ARCH=i386 > >>>>> endif > >>>>> CC = $(MINGW)-gcc > >>>>> CONFIGURE_ARGS= --host=$(MINGW) --target=$(MINGW) > >>>>> else #linux > >>>>> ? > >>>>> > >>>>> David > >>>>> ----- > >>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> 2019?3?20?(?) 11:25 David Holmes : > >>>>>>> On 20/03/2019 11:54 am, Yasumasa Suenaga wrote: > >>>>>>>> Hi David, > >>>>>>>> > >>>>>>>> 2019?3?20?(?) 10:38 David Holmes : > >>>>>>>>> Hi Yasumasa, > >>>>>>>>> > >>>>>>>>> I'm not familar with building hsdis, but if the only currnet > >>>>>>>>> problem is > >>>>>>>>> on Windows, why is the fix not restricted to only building on > >>>>>>>>> Windows? > >>>>>>>> For Solaris and Linux, -lz is not added to LDFLAGS. I think it > >>>>>>>> means > >>>>>>>> initial committer want to use zlib in binutils. > >>>>>>>> And zlib in binutils will be built implicitly in build phase in > >>>>>>>> binutils. > >>>>>>>> > >>>>>>>>> Otherwise we need people who build hsdis on other platforms to > >>>>>>>>> comment > >>>>>>>>> on the appropriateness of the fix. > >>>>>>>> If using different zlib is used by platforms is allowed, I will > >>>>>>>> upload > >>>>>>>> webrev which affects Windows only. > >>>>>>>> What do you think? > >>>>>>> That seems preferable/simpler. Then you only need to know that your > >>>>>>> change works for non-MinGW64 environments. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> David > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Yasumasa > >>>>>>>> > >>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> David > >>>>>>>>> > >>>>>>>>> On 20/03/2019 11:07 am, Yasumasa Suenaga wrote: > >>>>>>>>>> Hi Erik, David, > >>>>>>>>>> > >>>>>>>>>> I checked this change on Linux x64 and MinGW for Windows. > >>>>>>>>>> According to hsdis README, we need to use MinGW cross compiler > to > >>>>>>>>>> build hsdis [1]. So I think Cygwin is not required. > >>>>>>>>>> > >>>>>>>>>> I do not have macOS and AIX. So I cannot check this change on > >>>>>>>>>> them. > >>>>>>>>>> > >>>>>>>>>> BTW is hsdis included Java SE spec? > >>>>>>>>>> hsdis seems not to be included jtreg tests, and it is not > >>>>>>>>>> contained in > >>>>>>>>>> OpenJDK binaries. > >>>>>>>>>> I think it is allowed even if we lack some test for hsdis if > >>>>>>>>>> hsdis is > >>>>>>>>>> not required for Java SE. > >>>>>>>>>> If not so, I want sponsor(s) for this change. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> Yasumasa > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> [1] > >>>>>>>>>> > http://hg.openjdk.java.net/jdk/jdk/file/ddfb658c8ce3/src/utils/hsdis/README#l91 > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> 2019?3?20?(?) 6:54 David Holmes : > >>>>>>>>>>> CC'ing hotspot-dev. I agree this needs to be checked on every > >>>>>>>>>>> platform > >>>>>>>>>>> affected. I can't comment on the fix itself. > >>>>>>>>>>> > >>>>>>>>>>> David > >>>>>>>>>>> > >>>>>>>>>>> On 20/03/2019 2:36 am, Erik Joelsson wrote: > >>>>>>>>>>>> I think this needs to be reviewed by at least someone in > >>>>>>>>>>>> hotspot who > >>>>>>>>>>>> regularly builds hsdis. I can't really comment on the > >>>>>>>>>>>> validity of the > >>>>>>>>>>>> patch as I'm unfamiliar with both hsdis as well as this > >>>>>>>>>>>> makefile. Have > >>>>>>>>>>>> you at least verified the build on all the platforms which > >>>>>>>>>>>> you affect > >>>>>>>>>>>> with this change (which would be at least Macos, AIX and > >>>>>>>>>>>> Windows in a > >>>>>>>>>>>> normal Cygwin VS env)? > >>>>>>>>>>>> > >>>>>>>>>>>> /Erik > >>>>>>>>>>>> > >>>>>>>>>>>> On 2019-03-18 17:56, Yasumasa Suenaga wrote: > >>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Please review this change: > >>>>>>>>>>>>> > >>>>>>>>>>>>> JBS: > https://bugs.openjdk.java.net/browse/JDK-8220784 > >>>>>>>>>>>>> webrev: > >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220784/webrev.00/ > >>>>>>>>>>>>> > >>>>>>>>>>>>> I attempt to build hsdis for Windows on WSL Ubuntu 18.04 with > >>>>>>>>>>>>> gcc-mingw-w64-x86-64, but I saw error messages that some > >>>>>>>>>>>>> functions > >>>>>>>>>>>>> which are provided by zlib are unresolved. > >>>>>>>>>>>>> We need to link to zlib. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Yasumasa > > > From robbin.ehn at oracle.com Thu Mar 21 08:04:25 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 21 Mar 2019 09:04:25 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <3c2145f8-c56b-e949-ac4e-8fc16a099b43@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> <3c2145f8-c56b-e949-ac4e-8fc16a099b43@oracle.com> Message-ID: <273bcd39-c527-5a75-385a-f825574b0ffc@oracle.com> Hi, > > I'm really not sure how using handshakes for suspend might simplify the current > problem. What I have in my head is leaving the thread on an active handshake blocked on SR_lock (some new features needed in handshakes code). So the suspend case would get the instruction barrier from the normal blocked on lock transition. So I think that would leave only 2 instruction barriers for blocked and with lazy disarm 1 in the should block which handles the native case. > >>> I don't understand at all what your changes to the safepoint/handshake >>> disarming logic are doing or how they relate to all this. >> That was basically my suggestion. >> The first proposal was to introduce CPUID in native wrapper which is costly >> from performance point of view. >> Every Java thread returning from native would have to execute it even if there >> was no safepoint. And there was no way to detect if there was one. >> So my idea was to change the disarming logic such that every thread returning >> from native must detect if there was a safepoint. >> This is done by moving the task of disarming from the VM thread to the thread >> returning from native. > > Doesn't the safepoint counter provide a means to detect if there has been a > safepoint? The problem is that we also need them for handshakes. Having a per thread handshake counter would work. (I think also Haley said this or similar) And update handshake counter on safepoint. (or just do the safepoint as a handshake all operation) But it would require updates to all assembly code adding more checks. > > I worry about making this kind of tweak to the safepoint mechanism. I now have > to convince myself that a self-disarm can't race with a re-arm from the VMThread. In safepoint case we first store global state and then arm: 319 // We are synchronizing 320 OrderAccess::storestore(); // Ordered with _safepoint_counter 321 _state = _synchronizing; 322 323 if (SafepointMechanism::uses_thread_local_poll()) { 324 // Arming the per thread poll while having _state != _not_synchronized means safepointing 325 log_trace(safepoint)("Setting thread local yield flag for threads"); 326 OrderAccess::storestore(); // storestore, global state -> local state 327 for (JavaThreadIteratorWithHandle jtiwh; JavaThread *cur = jtiwh.next(); ) { 328 // Make sure the threads start polling, it is time to yield. 329 SafepointMechanism::arm_local_poll(cur); In handshake case: void HandshakeState::set_operation(JavaThread* target, HandshakeOperation* op) { _operation = op; SafepointMechanism::arm_local_poll_release(target); } So in both cases _operation/_state are always stored before the arming. If we disarm, recheck with: if (uses_thread_local_poll() && local_poll_armed(thread)) { 108 disarm_local_poll_release(thread); 109 // We might have disarmed next safepoint/handshake 110 OrderAccess::storeload(); 111 if (global_poll() || thread->has_handshake()) { 112 arm_local_poll(thread); 113 } 114 } We might re-arm an already armed poll from VMThread, that is okey. We might re-arm a disarm from VMThread and hit an extra poll site, that is okey. Since we are in an unsafe state running this code, after re-arming we can continue to execute and stop at next polling site. Instead of rearming we can just call block_if_requested_slow again, this have the upside of faster re-safepointing and might be better. I did not find any issues with the recursion (but generally avoiding recursion is good). Hopefully there is no error in my logic :) /Robbin > > Thanks, > David > ----- > >> I guess Robbin will also add a description. >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Mittwoch, 20. M?rz 2019 11:25 >> To: Robbin Ehn ; hotspot-dev at openjdk.java.net; Doerr, >> Martin >> Cc: aph at redhat.com; Erik ?sterlund ; Reingruber, >> Richard >> Subject: Re: RFR(m): 8220351: Cross-modifying code >> >> On 20/03/2019 7:31 pm, Robbin Ehn wrote: >>> Hi, v3 updated with Martin's comments. >>> >>> Inc: >>> http://cr.openjdk.java.net/~rehn/8220351/v3/inc/ >>> Full: >>> http://cr.openjdk.java.net/~rehn/8220351/v3/ >>> >>> David, are you okay with this model which does cross-mod fence on >>> leaving a safe state? With the other model, always cross-mod fence >>> going to java would mean we cannot disarm any thread since we don't >>> know which state it's going to to next, if we want the same >>> performance. >> >> Sorry Robbin but I can't follow the new approach just from the code >> changes. As I wrote earlier I'd like to see the stat-space enumerated >> here to show why pushing this down to the "safe states" is sufficient >> (and not introducing redundant fences) - e.g we fence when going from >> _thread_blocked to _thread_in_vm which covers the case if we then return >> to _thread_in_java; but if we're returning to native then we don't need >> the fence. >> >> I don't understand at all what your changes to the safepoint/handshake >> disarming logic are doing or how they relate to all this. Can you write >> up the design in the bug report please. >> >> Thanks, >> David >> ----- >> >> >>> >>> Thanks, Robbin >>> >>> On 3/19/19 5:43 PM, Robbin Ehn wrote: >>>> Hi all, here is v2: >>>> >>>> Full: >>>> http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ >>>> Inc: >>>> http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ >>>> >>>> The deprecated non-TLH path is missing the correct cross-mod fence, >>>> but since 13 >>>> will be the last release containing that code path I think that is okay. >>>> >>>> We just leave native/native trans armed, and let the thread disarm on the >>>> transition. If it disarms a new safepoint, we re-arm and stop at next >>>> polling >>>> site, that avoids recursion. >>>> >>>> Passes stress and hs-t1-5. >>>> As in the previous mail, performance numbers look good. >>>> >>>> Thanks, Robbin >>>> >>>> On 3/8/19 4:24 PM, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>>> Changeset: >>>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>>> >>>>> After a JavaThread have been in a safepoint/(handshake) safe state it >>>>> can start >>>>> executing updated code. E.g. an oop in the instruction stream can >>>>> have been >>>>> updated. >>>>> >>>>> Most hardware's require a barrier or that the code cross modified is >>>>> far away to >>>>> guarantee that the thread executing the updated instruction stream >>>>> sees the >>>>> modification. >>>>> >>>>> What far away is and how far an update instruction stream is from a >>>>> safepoint >>>>> safe state is not clear. >>>>> >>>>> To be compliant with those architectures an instruction stream >>>>> barrier must be >>>>> added when leaving the safepoint safe state. >>>>> >>>>> There may be crashes today due to this missing barrier. >>>>> A new CPU with deeper pipeline or changes to the VM which moves a >>>>> safepoint safe >>>>> state closer to a nmethod can increase changes of a crash. >>>>> >>>>> A few benchmarks will see a performance regression with the extra >>>>> serializing, >>>>> such as Octane-Crypto -5% (x86). >>>>> >>>>> With a much more elaborate fix, such as changing the local poll to >>>>> have more >>>>> than two states (armed/disarmed), it would be possible to only emit such >>>>> instruction stream barrier when the poll have been armed while the >>>>> thread was >>>>> safepoint safe. >>>>> >>>>> Passes t1-3 on our platforms, and test built what I can. >>>>> >>>>> Thanks, Robbin From fujie at loongson.cn Thu Mar 21 08:12:25 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 21 Mar 2019 16:12:25 +0800 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <8ba2ea2e-04f3-14f3-00db-4a7cf632fdfd@loongson.cn> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> <8ba2ea2e-04f3-14f3-00db-4a7cf632fdfd@loongson.cn> Message-ID: <9f0025a6-a5e1-b3f7-582f-3c6d6f3a020c@loongson.cn> Hi Bob and David, The patch[1] has been tested with ----------------------------------------------- make test TEST="test/jdk/jdk/modules/scenarios/container test/hotspot/jtreg/vmTestbase/vm/gc/containers test/hotspot/jtreg/runtime/containers" ----------------------------------------------- on Linux/x86-64. All tests passed. ============================== Test summary ============================== ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL ERROR ?? jtreg:test/jdk/jdk/modules/scenarios/container??????? 1 1???? 0???? 0 ?? jtreg:test/hotspot/jtreg/vmTestbase/vm/gc/containers ??????????????????????????????????????????????????????? 12 12???? 0???? 0 ?? jtreg:test/hotspot/jtreg/runtime/containers?????????? 6 6???? 0???? 0 ============================== TEST SUCCESS Is it OK to be pushed? Thanks a lot. Best regards, Jie [1] http://cr.openjdk.java.net/~jiefu/8220658/webrev.01/ On 2019/3/21 ??9:38, Jie Fu wrote: > Oh, I'm very sorry for that. I am not good at English. > Thanks David. > > > On 2019/3/21 ??9:25, David Holmes wrote: >> On 21/03/2019 11:06 am, Jie Fu wrote: >>> Thanks Bob. >>> I'll update the patch and run the container jtreg tests as soon as >>> possible. >> >> Bob was commenting on my original suggestion to change failed to "not >> supported" and indicated agreement. So I don't think there's any >> update to the patch needed ?? >> >> David >> >>> Best regards, >>> >>> Jie >>> >>> >>> On 2019/3/20 ??10:40, Bob Vandette wrote: >>>> I agree with David on the failed versus not supported. >>>> >>>> Make sure to run the container jtreg tests since these might rely >>>> on the current output >>>> formating. >>>> >>>> Bob. >>>> >>>> >>>>> On Mar 18, 2019, at 5:43 PM, David Holmes >>>>> wrote: >>>>> >>>>> Hi Jie, >>>>> >>>>> On 14/03/2019 8:00 pm, Jie Fu wrote: >>>>>> Hi all, >>>>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 >>>>>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >>>>> This seems a reasonable improvement, but I have a couple of >>>>> suggestions. To avoid so much duplication you can change this >>>>> structure: >>>>> >>>>> ?? if (i > 0) { >>>>> ???? st->print("cpu_quota: %d\n", i); >>>>> ?? } else { >>>>> ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? >>>>> "failed" : "no quota"); >>>>> ?? } >>>>> >>>>> to this: >>>>> >>>>> ?? st->print("cpu_quota: "); >>>>> ?? if (i > 0) { >>>>> ????? st->print("%d\n", i); >>>>> ?? } else { >>>>> ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no >>>>> quota"); >>>>> ?? } >>>>> >>>>> Also I suggest replacing "failed" with "not supported". >>>>> >>>>> Finally one minor typo: s/"no share"/"no shares"/ >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> In the VM's error log, the container information was dumpped as: >>>>>> --------------------------------- >>>>>> container (cgroup) information: >>>>>> container_type: cgroupv1 >>>>>> cpu_cpuset_cpus: 0-11 >>>>>> cpu_memory_nodes: 0 >>>>>> active_processor_count: 12 >>>>>> cpu_quota: -1 >>>>>> cpu_period: 100000 >>>>>> cpu_shares: -1 >>>>>> memory_limit_in_bytes: -1 >>>>>> memory_and_swap_limit_in_bytes: -2 >>>>>> memory_soft_limit_in_bytes: -1 >>>>>> memory_usage_in_bytes: 11690164224 >>>>>> memory_max_usage_in_bytes: 1806413824 >>>>>> --------------------------------- >>>>>> Items such as "memory_limit_in_bytes" and >>>>>> "memory_and_swap_limit_in_bytes" are difficult to understand. >>>>>> It might be more human-readable if we could dump it like this: >>>>>> --------------------------------- >>>>>> container (cgroup) information: >>>>>> container_type: cgroupv1 >>>>>> cpu_cpuset_cpus: 0-11 >>>>>> cpu_memory_nodes: 0 >>>>>> active_processor_count: 12 >>>>>> cpu_quota: no quota >>>>>> cpu_period: 100000 >>>>>> cpu_shares: no share >>>>>> memory_limit_in_bytes: unlimited >>>>>> memory_and_swap_limit_in_bytes: failed >>>>>> memory_soft_limit_in_bytes: unlimited >>>>>> memory_usage_in_bytes: 11690164224 >>>>>> memory_max_usage_in_bytes: 1806413824 >>>>>> --------------------------------- >>>>>> I made this patch to improve the readability of container >>>>>> information in the error log. >>>>>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>>>>> Could you please review it? >>>>>> Thanks a lot. >>>>>> Best regards, >>>>>> Jie >>> > From david.holmes at oracle.com Thu Mar 21 08:42:32 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Mar 2019 18:42:32 +1000 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <9f0025a6-a5e1-b3f7-582f-3c6d6f3a020c@loongson.cn> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> <8ba2ea2e-04f3-14f3-00db-4a7cf632fdfd@loongson.cn> <9f0025a6-a5e1-b3f7-582f-3c6d6f3a020c@loongson.cn> Message-ID: <47f08df6-bb0b-d34b-9dfd-86a33d3c70c8@oracle.com> Hi Jie, On 21/03/2019 6:12 pm, Jie Fu wrote: > Hi Bob and David, > > The patch[1] has been tested with > ----------------------------------------------- > make test TEST="test/jdk/jdk/modules/scenarios/container > test/hotspot/jtreg/vmTestbase/vm/gc/containers > test/hotspot/jtreg/runtime/containers" > ----------------------------------------------- > on Linux/x86-64. > > All tests passed. > ============================== > Test summary > ============================== > ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL > ERROR > ?? jtreg:test/jdk/jdk/modules/scenarios/container??????? 1 1???? 0???? 0 > ?? jtreg:test/hotspot/jtreg/vmTestbase/vm/gc/containers > ??????????????????????????????????????????????????????? 12 12???? 0???? 0 > ?? jtreg:test/hotspot/jtreg/runtime/containers?????????? 6 6???? 0???? 0 > ============================== > TEST SUCCESS > > Is it OK to be pushed? Yes, I will sponsor it for you. Thanks, David ----- > Thanks a lot. > > Best regards, > Jie > > [1] http://cr.openjdk.java.net/~jiefu/8220658/webrev.01/ > > > On 2019/3/21 ??9:38, Jie Fu wrote: >> Oh, I'm very sorry for that. I am not good at English. >> Thanks David. >> >> >> On 2019/3/21 ??9:25, David Holmes wrote: >>> On 21/03/2019 11:06 am, Jie Fu wrote: >>>> Thanks Bob. >>>> I'll update the patch and run the container jtreg tests as soon as >>>> possible. >>> >>> Bob was commenting on my original suggestion to change failed to "not >>> supported" and indicated agreement. So I don't think there's any >>> update to the patch needed ?? >>> >>> David >>> >>>> Best regards, >>>> >>>> Jie >>>> >>>> >>>> On 2019/3/20 ??10:40, Bob Vandette wrote: >>>>> I agree with David on the failed versus not supported. >>>>> >>>>> Make sure to run the container jtreg tests since these might rely >>>>> on the current output >>>>> formating. >>>>> >>>>> Bob. >>>>> >>>>> >>>>>> On Mar 18, 2019, at 5:43 PM, David Holmes >>>>>> wrote: >>>>>> >>>>>> Hi Jie, >>>>>> >>>>>> On 14/03/2019 8:00 pm, Jie Fu wrote: >>>>>>> Hi all, >>>>>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8220658 >>>>>>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >>>>>> This seems a reasonable improvement, but I have a couple of >>>>>> suggestions. To avoid so much duplication you can change this >>>>>> structure: >>>>>> >>>>>> ?? if (i > 0) { >>>>>> ???? st->print("cpu_quota: %d\n", i); >>>>>> ?? } else { >>>>>> ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? >>>>>> "failed" : "no quota"); >>>>>> ?? } >>>>>> >>>>>> to this: >>>>>> >>>>>> ?? st->print("cpu_quota: "); >>>>>> ?? if (i > 0) { >>>>>> ????? st->print("%d\n", i); >>>>>> ?? } else { >>>>>> ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no >>>>>> quota"); >>>>>> ?? } >>>>>> >>>>>> Also I suggest replacing "failed" with "not supported". >>>>>> >>>>>> Finally one minor typo: s/"no share"/"no shares"/ >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> In the VM's error log, the container information was dumpped as: >>>>>>> --------------------------------- >>>>>>> container (cgroup) information: >>>>>>> container_type: cgroupv1 >>>>>>> cpu_cpuset_cpus: 0-11 >>>>>>> cpu_memory_nodes: 0 >>>>>>> active_processor_count: 12 >>>>>>> cpu_quota: -1 >>>>>>> cpu_period: 100000 >>>>>>> cpu_shares: -1 >>>>>>> memory_limit_in_bytes: -1 >>>>>>> memory_and_swap_limit_in_bytes: -2 >>>>>>> memory_soft_limit_in_bytes: -1 >>>>>>> memory_usage_in_bytes: 11690164224 >>>>>>> memory_max_usage_in_bytes: 1806413824 >>>>>>> --------------------------------- >>>>>>> Items such as "memory_limit_in_bytes" and >>>>>>> "memory_and_swap_limit_in_bytes" are difficult to understand. >>>>>>> It might be more human-readable if we could dump it like this: >>>>>>> --------------------------------- >>>>>>> container (cgroup) information: >>>>>>> container_type: cgroupv1 >>>>>>> cpu_cpuset_cpus: 0-11 >>>>>>> cpu_memory_nodes: 0 >>>>>>> active_processor_count: 12 >>>>>>> cpu_quota: no quota >>>>>>> cpu_period: 100000 >>>>>>> cpu_shares: no share >>>>>>> memory_limit_in_bytes: unlimited >>>>>>> memory_and_swap_limit_in_bytes: failed >>>>>>> memory_soft_limit_in_bytes: unlimited >>>>>>> memory_usage_in_bytes: 11690164224 >>>>>>> memory_max_usage_in_bytes: 1806413824 >>>>>>> --------------------------------- >>>>>>> I made this patch to improve the readability of container >>>>>>> information in the error log. >>>>>>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>>>>>> Could you please review it? >>>>>>> Thanks a lot. >>>>>>> Best regards, >>>>>>> Jie >>>> >> > From fujie at loongson.cn Thu Mar 21 08:48:22 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 21 Mar 2019 16:48:22 +0800 Subject: RFR: 8220658: Improve the readability of container information in the error log In-Reply-To: <47f08df6-bb0b-d34b-9dfd-86a33d3c70c8@oracle.com> References: <9094aea0-f241-686c-9977-520ba1f155cc@loongson.cn> <5cf0dbde-21c6-bf39-ccd0-428d59000028@oracle.com> <023A5F6E-5805-47B2-A85A-FD6C70C39A7A@oracle.com> <8ba2ea2e-04f3-14f3-00db-4a7cf632fdfd@loongson.cn> <9f0025a6-a5e1-b3f7-582f-3c6d6f3a020c@loongson.cn> <47f08df6-bb0b-d34b-9dfd-86a33d3c70c8@oracle.com> Message-ID: Thank you so much, David. On 2019/3/21 ??4:42, David Holmes wrote: > Hi Jie, > > On 21/03/2019 6:12 pm, Jie Fu wrote: >> Hi Bob and David, >> >> The patch[1] has been tested with >> ----------------------------------------------- >> make test TEST="test/jdk/jdk/modules/scenarios/container >> test/hotspot/jtreg/vmTestbase/vm/gc/containers >> test/hotspot/jtreg/runtime/containers" >> ----------------------------------------------- >> on Linux/x86-64. >> >> All tests passed. >> ============================== >> Test summary >> ============================== >> ??? TEST????????????????????????????????????????????? TOTAL PASS FAIL >> ERROR >> ??? jtreg:test/jdk/jdk/modules/scenarios/container??????? 1 1???? >> 0???? 0 >> ??? jtreg:test/hotspot/jtreg/vmTestbase/vm/gc/containers >> ???????????????????????????????????????????????????????? 12 12???? >> 0???? 0 >> ??? jtreg:test/hotspot/jtreg/runtime/containers?????????? 6 6???? >> 0???? 0 >> ============================== >> TEST SUCCESS >> >> Is it OK to be pushed? > > Yes, I will sponsor it for you. > > Thanks, > David > ----- > >> Thanks a lot. >> >> Best regards, >> Jie >> >> [1] http://cr.openjdk.java.net/~jiefu/8220658/webrev.01/ >> >> >> On 2019/3/21 ??9:38, Jie Fu wrote: >>> Oh, I'm very sorry for that. I am not good at English. >>> Thanks David. >>> >>> >>> On 2019/3/21 ??9:25, David Holmes wrote: >>>> On 21/03/2019 11:06 am, Jie Fu wrote: >>>>> Thanks Bob. >>>>> I'll update the patch and run the container jtreg tests as soon as >>>>> possible. >>>> >>>> Bob was commenting on my original suggestion to change failed to >>>> "not supported" and indicated agreement. So I don't think there's >>>> any update to the patch needed ?? >>>> >>>> David >>>> >>>>> Best regards, >>>>> >>>>> Jie >>>>> >>>>> >>>>> On 2019/3/20 ??10:40, Bob Vandette wrote: >>>>>> I agree with David on the failed versus not supported. >>>>>> >>>>>> Make sure to run the container jtreg tests since these might rely >>>>>> on the current output >>>>>> formating. >>>>>> >>>>>> Bob. >>>>>> >>>>>> >>>>>>> On Mar 18, 2019, at 5:43 PM, David Holmes >>>>>>> wrote: >>>>>>> >>>>>>> Hi Jie, >>>>>>> >>>>>>> On 14/03/2019 8:00 pm, Jie Fu wrote: >>>>>>>> Hi all, >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8220658 >>>>>>>> Webrev: http://cr.openjdk.java.net/~jiefu/8220658/webrev.00/ >>>>>>> This seems a reasonable improvement, but I have a couple of >>>>>>> suggestions. To avoid so much duplication you can change this >>>>>>> structure: >>>>>>> >>>>>>> ?? if (i > 0) { >>>>>>> ???? st->print("cpu_quota: %d\n", i); >>>>>>> ?? } else { >>>>>>> ???? st->print("cpu_quota: %s\n", i == OSCONTAINER_ERROR ? >>>>>>> "failed" : "no quota"); >>>>>>> ?? } >>>>>>> >>>>>>> to this: >>>>>>> >>>>>>> ?? st->print("cpu_quota: "); >>>>>>> ?? if (i > 0) { >>>>>>> ????? st->print("%d\n", i); >>>>>>> ?? } else { >>>>>>> ???? st->print("%s\n", i == OSCONTAINER_ERROR ? "failed" : "no >>>>>>> quota"); >>>>>>> ?? } >>>>>>> >>>>>>> Also I suggest replacing "failed" with "not supported". >>>>>>> >>>>>>> Finally one minor typo: s/"no share"/"no shares"/ >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> In the VM's error log, the container information was dumpped as: >>>>>>>> --------------------------------- >>>>>>>> container (cgroup) information: >>>>>>>> container_type: cgroupv1 >>>>>>>> cpu_cpuset_cpus: 0-11 >>>>>>>> cpu_memory_nodes: 0 >>>>>>>> active_processor_count: 12 >>>>>>>> cpu_quota: -1 >>>>>>>> cpu_period: 100000 >>>>>>>> cpu_shares: -1 >>>>>>>> memory_limit_in_bytes: -1 >>>>>>>> memory_and_swap_limit_in_bytes: -2 >>>>>>>> memory_soft_limit_in_bytes: -1 >>>>>>>> memory_usage_in_bytes: 11690164224 >>>>>>>> memory_max_usage_in_bytes: 1806413824 >>>>>>>> --------------------------------- >>>>>>>> Items such as "memory_limit_in_bytes" and >>>>>>>> "memory_and_swap_limit_in_bytes" are difficult to understand. >>>>>>>> It might be more human-readable if we could dump it like this: >>>>>>>> --------------------------------- >>>>>>>> container (cgroup) information: >>>>>>>> container_type: cgroupv1 >>>>>>>> cpu_cpuset_cpus: 0-11 >>>>>>>> cpu_memory_nodes: 0 >>>>>>>> active_processor_count: 12 >>>>>>>> cpu_quota: no quota >>>>>>>> cpu_period: 100000 >>>>>>>> cpu_shares: no share >>>>>>>> memory_limit_in_bytes: unlimited >>>>>>>> memory_and_swap_limit_in_bytes: failed >>>>>>>> memory_soft_limit_in_bytes: unlimited >>>>>>>> memory_usage_in_bytes: 11690164224 >>>>>>>> memory_max_usage_in_bytes: 1806413824 >>>>>>>> --------------------------------- >>>>>>>> I made this patch to improve the readability of container >>>>>>>> information in the error log. >>>>>>>> The change has been tested with hotspot:tier1 on Linux/x86-64. >>>>>>>> Could you please review it? >>>>>>>> Thanks a lot. >>>>>>>> Best regards, >>>>>>>> Jie >>>>> >>> >> From richard.reingruber at sap.com Thu Mar 21 09:00:39 2019 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 21 Mar 2019 09:00:39 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> Message-ID: <3dc92388d9a243fea056583de7bf154f@sap.com> Hi Robin, > Hopefully there is no error in my logic :) At least none that I would have been able to find :) To me the patch looks good. I'm not a reviewer, though. Thanks, Richard. -----Original Message----- From: Robbin Ehn Sent: Mittwoch, 20. M?rz 2019 10:32 To: hotspot-dev at openjdk.java.net; David Holmes ; Doerr, Martin Cc: aph at redhat.com; Erik ?sterlund ; Reingruber, Richard Subject: Re: RFR(m): 8220351: Cross-modifying code Hi, v3 updated with Martin's comments. Inc: http://cr.openjdk.java.net/~rehn/8220351/v3/inc/ Full: http://cr.openjdk.java.net/~rehn/8220351/v3/ David, are you okay with this model which does cross-mod fence on leaving a safe state? With the other model, always cross-mod fence going to java would mean we cannot disarm any thread since we don't know which state it's going to to next, if we want the same performance. Thanks, Robbin On 3/19/19 5:43 PM, Robbin Ehn wrote: > Hi all, here is v2: > > Full: > http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ > Inc: > http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ > > The deprecated non-TLH path is missing the correct cross-mod fence, but since 13 > will be the last release containing that code path I think that is okay. > > We just leave native/native trans armed, and let the thread disarm on the > transition. If it disarms a new safepoint, we re-arm and stop at next polling > site, that avoids recursion. > > Passes stress and hs-t1-5. > As in the previous mail, performance numbers look good. > > Thanks, Robbin > > On 3/8/19 4:24 PM, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it can start >> executing updated code. E.g. an oop in the instruction stream can have been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is far away to >> guarantee that the thread executing the updated instruction stream sees the >> modification. >> >> What far away is and how far an update instruction stream is from a safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From robbin.ehn at oracle.com Thu Mar 21 11:35:22 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 21 Mar 2019 12:35:22 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <3dc92388d9a243fea056583de7bf154f@sap.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <9f3e679a-8a87-d30b-eb31-c513a0f7a3bb@oracle.com> <3dc92388d9a243fea056583de7bf154f@sap.com> Message-ID: Hi Richard, On 2019-03-21 10:00, Reingruber, Richard wrote: > Hi Robin, > > > Hopefully there is no error in my logic :) > > At least none that I would have been able to find :) Good :) > > To me the patch looks good. I'm not a reviewer, though. Thanks! /Robbin > > Thanks, Richard. > > -----Original Message----- > From: Robbin Ehn > Sent: Mittwoch, 20. M?rz 2019 10:32 > To: hotspot-dev at openjdk.java.net; David Holmes ; Doerr, Martin > Cc: aph at redhat.com; Erik ?sterlund ; Reingruber, Richard > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi, v3 updated with Martin's comments. > > Inc: > http://cr.openjdk.java.net/~rehn/8220351/v3/inc/ > Full: > http://cr.openjdk.java.net/~rehn/8220351/v3/ > > David, are you okay with this model which does cross-mod fence on leaving a safe > state? > With the other model, always cross-mod fence going to java would mean we cannot > disarm any thread since we don't know which state it's going to to next, if we > want the same performance. > > Thanks, Robbin > > On 3/19/19 5:43 PM, Robbin Ehn wrote: >> Hi all, here is v2: >> >> Full: >> http://cr.openjdk.java.net/~rehn/8220351/v2/webrev/ >> Inc: >> http://cr.openjdk.java.net/~rehn/8220351/v2/inc/webrev/ >> >> The deprecated non-TLH path is missing the correct cross-mod fence, but since 13 >> will be the last release containing that code path I think that is okay. >> >> We just leave native/native trans armed, and let the thread disarm on the >> transition. If it disarms a new safepoint, we re-arm and stop at next polling >> site, that avoids recursion. >> >> Passes stress and hs-t1-5. >> As in the previous mail, performance numbers look good. >> >> Thanks, Robbin >> >> On 3/8/19 4:24 PM, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> After a JavaThread have been in a safepoint/(handshake) safe state it can start >>> executing updated code. E.g. an oop in the instruction stream can have been >>> updated. >>> >>> Most hardware's require a barrier or that the code cross modified is far away to >>> guarantee that the thread executing the updated instruction stream sees the >>> modification. >>> >>> What far away is and how far an update instruction stream is from a safepoint >>> safe state is not clear. >>> >>> To be compliant with those architectures an instruction stream barrier must be >>> added when leaving the safepoint safe state. >>> >>> There may be crashes today due to this missing barrier. >>> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >>> state closer to a nmethod can increase changes of a crash. >>> >>> A few benchmarks will see a performance regression with the extra serializing, >>> such as Octane-Crypto -5% (x86). >>> >>> With a much more elaborate fix, such as changing the local poll to have more >>> than two states (armed/disarmed), it would be possible to only emit such >>> instruction stream barrier when the poll have been armed while the thread was >>> safepoint safe. >>> >>> Passes t1-3 on our platforms, and test built what I can. >>> >>> Thanks, Robbin From bob.vandette at oracle.com Thu Mar 21 17:12:04 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 21 Mar 2019 13:12:04 -0400 Subject: RFR: 8220674: [TESTBUG] MetricsMemoryTester failcount test in docker container only works with debug JVMs Message-ID: Please review this fix for a container test that fails on some Linux distributions. The test fails to see the Memory Fail count metric value increase. This is caused by the fact that we are allowing ergonomics to select the amount of Heap size. This size varies depending on the amount of free memory that?s available on different docker implementations. The fix is to force the VM to always set the heap size to the size of the containers memory. This change also stops allocating once it sees the fail count increase. This is needed to keep the container from getting killed by the out of memory killer. The fix has been verified by the submitter along with a local run of the container tests. BUG: https://bugs.openjdk.java.net/browse/JDK-8220674 WEBREV: http://cr.openjdk.java.net/~bobv/8220674/webrev/ Bob. From sgehwolf at redhat.com Thu Mar 21 17:54:50 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 21 Mar 2019 18:54:50 +0100 Subject: RFR: 8220674: [TESTBUG] MetricsMemoryTester failcount test in docker container only works with debug JVMs In-Reply-To: References: Message-ID: Hi Bob, On Thu, 2019-03-21 at 13:12 -0400, Bob Vandette wrote: > WEBREV: > > http://cr.openjdk.java.net/~bobv/8220674/webrev/ This looks good to me. Thanks for fixing it! I'm not a Reviewer, though. Cheers, Severin From serguei.spitsyn at oracle.com Thu Mar 21 18:10:24 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 21 Mar 2019 11:10:24 -0700 Subject: RFR: 8220674: [TESTBUG] MetricsMemoryTester failcount test in docker container only works with debug JVMs In-Reply-To: References: Message-ID: <70f2d8c6-e3e6-7676-e4fb-6928588f0704@oracle.com> Hi Bob, It looks good to me. Thanks, Serguei On 3/21/19 10:12, Bob Vandette wrote: > Please review this fix for a container test that fails on some Linux distributions. > > The test fails to see the Memory Fail count metric value increase. This is caused by > the fact that we are allowing ergonomics to select the amount of Heap size. This > size varies depending on the amount of free memory that?s available on different > docker implementations. > > The fix is to force the VM to always set the heap size to the size of the containers > memory. This change also stops allocating once it sees the fail count increase. > This is needed to keep the container from getting killed by the out of memory > killer. > > The fix has been verified by the submitter along with a local run of the container tests. > > BUG: > > https://bugs.openjdk.java.net/browse/JDK-8220674 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8220674/webrev/ > > Bob. > > From kim.barrett at oracle.com Thu Mar 21 22:25:27 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 21 Mar 2019 18:25:27 -0400 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol Message-ID: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> Please review this change to the thread claiming protocol for parallel iteration over threads. Previously, the protocol used a 1bit claim counter plus zero as a "never claimed" marker. The new protocol uses a uintx counter, while retaining zero as a "never claimed" marker. JDK-8219613 changed use use of the claiming protocol, allowing all threads to be included and claimed by a parallel iteration. Doing so introduced a bug. Not all parallel iteration paths visit every thread. In particular, possibly_parallel_threads_do (PPTD) only visits Java threads and the VM thread. Using both all-thread parallel iterations and PPTD could lead to some threads being improperly skipped, because of the 1bit claim counter. For example, if an all-thread iteration is followed by an even number of PPTD then another all-thread iteration, the second all-thread iteration will incorrectly skip NonJavaThreads other than the VMThread. (Testing these changes with an instrumented claim function that detected cases that would be in trouble with the old 1bit counter found *many* occurrences. It is somewhat surprising that this doesn't seem to have produced lots of failures since JDK-8219613.) We address this by expanding the claim counter to a uintx-sized counter. There is still a theoretical possibility of collision, which we deal with by resetting the claim value of all threads back to zero when the global counter overflows. (On a 32bit platform, overflow of the claim counter probably takes months. And even that is likely more often than needed, since a collision needs MAX_UINTX-1 PPTD iterations between all-thread iterations, which isn't how iterations are used.) Some renaming was done to generalize from the old "parity" nomenclature, and because it's not just about "oops_do" (and wasn't before this change). Hopefully there won't be *too* much bike-shedding on the names. CR: https://bugs.openjdk.java.net/browse/JDK-8221102 Webrev: http://cr.openjdk.java.net/~kbarrett/8221102/open.00/ Testing: mach5 tier1-5. From rkennke at redhat.com Thu Mar 21 22:55:46 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 21 Mar 2019 23:55:46 +0100 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> Message-ID: Hi Kim, > Please review this change to the thread claiming protocol for parallel > iteration over threads. Previously, the protocol used a 1bit claim > counter plus zero as a "never claimed" marker. The new protocol uses a > uintx counter, while retaining zero as a "never claimed" marker. > > JDK-8219613 changed use use of the claiming protocol, allowing all > threads to be included and claimed by a parallel iteration. Doing so > introduced a bug. > > Not all parallel iteration paths visit every thread. In particular, > possibly_parallel_threads_do (PPTD) only visits Java threads and the > VM thread. Using both all-thread parallel iterations and PPTD could > lead to some threads being improperly skipped, because of the 1bit > claim counter. For example, if an all-thread iteration is followed by > an even number of PPTD then another all-thread iteration, the second > all-thread iteration will incorrectly skip NonJavaThreads other than > the VMThread. > > (Testing these changes with an instrumented claim function that > detected cases that would be in trouble with the old 1bit counter > found *many* occurrences. It is somewhat surprising that this doesn't > seem to have produced lots of failures since JDK-8219613.) > > We address this by expanding the claim counter to a uintx-sized > counter. There is still a theoretical possibility of collision, which > we deal with by resetting the claim value of all threads back to zero > when the global counter overflows. (On a 32bit platform, overflow of > the claim counter probably takes months. And even that is likely more > often than needed, since a collision needs MAX_UINTX-1 PPTD iterations > between all-thread iterations, which isn't how iterations are used.) > > Some renaming was done to generalize from the old "parity" > nomenclature, and because it's not just about "oops_do" (and wasn't > before this change). Hopefully there won't be *too* much bike-shedding > on the names. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8221102 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8221102/open.00/ > > Testing: > mach5 tier1-5. This would work. I am going back and forth with this in my head. I believe the simpler, more usable and perhaps less surprising approach might be to simply always iterate all threads, and do not even let other code use the claiming protocol, and care about its idiosyncrasies: http://cr.openjdk.java.net/~rkennke/JDK-8221102/webrev.01/ This way the claim token would always go in lock-step. The overhead of iterating a few more non-Java-threads (somewhat likely doing nothing) seems not very significant. This could probably be taken a little further and hiding more of the claiming protocol. Thread::claim_oops_do() doesn't exactly need to be public anymore. Thread::thread_claim_parity() doesn't appear to be used anywhere outside Thread, except for an assert which could be done somehow else, and change_thread_claim_parity() could be done via friend classes or so. I don't have a strong preference for your vs my solution, except that I want this bug fixed. :-) Just wanted to throw it out so that others can chime into the discussion. Roman From sgehwolf at redhat.com Fri Mar 22 10:43:47 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 22 Mar 2019 11:43:47 +0100 Subject: RFR: 8217338: [Containers] Improve systemd slice memory limit support Message-ID: <4a1ae4e0e3efd68818bfe868349972aa878de60b.camel@redhat.com> Hi, Please review this change which improves container detection support tin the JVM. While docker container detection works quite well the results for systemd slices with memory limits are mixed and depend on the Linux kernel version in use. With newer kernel versions becoming more widely used we should improve JVMs memory limit detection support as well. This should be entirely backwards compatible as the hierarchical limit will only be used if everything else is unlimited. Bug: https://bugs.openjdk.java.net/browse/JDK-8217338 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8217338/04/webrev/ Testing: Manual testing of -XshowSettings and -Xlog:os+container=trace with systemd slices on affected Linux distributions: Fedora 29, recent Ubuntu (18-10). Existing docker tests pass. I'm currently also running this through jdk/submit. Thoughts? Thanks, Severin From kuaiwei.kw at alibaba-inc.com Fri Mar 22 12:15:01 2019 From: kuaiwei.kw at alibaba-inc.com (Kuai Wei) Date: Fri, 22 Mar 2019 20:15:01 +0800 Subject: =?UTF-8?B?UmU6IFJGQzogSldhcm11cCBwcmVjb21waWxlIGphdmEgaG90IG1ldGhvZHMgYXQgYXBwbGlj?= =?UTF-8?B?YXRpb24gc3RhcnR1cA==?= In-Reply-To: References: , Message-ID: <1a9e9d8c-9a80-4cf7-9510-7de1cc9cbc4e.kuaiwei.kw@alibaba-inc.com> Hi Tobias, jwarmup is designed to work on linux x86-64 only. So we have not tested on other platform. We will check the GC options in tests. Thanks, Kuai Wei ------------------------------------------------------------------ From:Tobias Hartmann Send Time:2019?3?15?(???) 19:26 To:yumin qi ; hotspot-dev Subject:Re: RFC: JWarmup precompile java hot methods at application startup Hi Yumin, I haven't looked at the code yet but just executed some sanity testing. Many of the new jwarmup tests crash the VM on Sparc: # SIGBUS (0xa) at pc=0xffffffff7d865780, pid=27039, tid=8 # # JRE version: Java(TM) SE Runtime Environment (13.0) (fastdebug build 13-internal+0-2019-03-15-0750126.tobias.hartmann.jdkjdk) # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 13-internal+0-2019-03-15-0750126.tobias.hartmann.jdkjdk, compiled mode, compressed oops, g1 gc, solaris-sparc) # Problematic frame: # V [libjvm.so+0x1865780] void ProfileRecorder::write_header()+0xf0 Stack: [0xffffffff54600000,0xffffffff54700000], sp=0xffffffff546ff960, free space=1022k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x1865780] void ProfileRecorder::write_header()+0xf0 V [libjvm.so+0x1866800] void ProfileRecorder::flush()+0xa0 V [libjvm.so+0x186426c] JitWarmUp::JitWarmUpState JitWarmUp::flush_logfile()+0x1c V [libjvm.so+0x187059c] void JitWarmUpFlushThread::run()+0x9c V [libjvm.so+0x227c8f8] void Thread::call_run()+0x178 V [libjvm.so+0x1f4d32c] thread_native_entry+0x3ac And some tests fail when executed with a different GC due to: Error occurred during initialization of VM Multiple garbage collectors selected They should either not specify the GC or have a @requires vm.gc == "null" Also, these tests should be moved to hotspot/jtreg/compiler/ or ../runtime/ Best regards, Tobias On 15.03.19 06:59, yumin qi wrote: > Please review the patch for implementing JEP 8203832: > (*https://bugs.openjdk.java.net/browse/JDK-8203832 > )* > > Bug: *https://bugs.openjdk.java.net/browse/JDK-8220692 > * > Webrev: http://cr.openjdk.java.net/~minqi/8220692/webrev-01/ > > > Summary: This patch implements the first version of JIT warmup to > solve the java application peak time CPU contest between JIT compiler > threads and regular application java threads. The CPU contest is > avoided by pre-run to collect comilation data and precompile the hot > java methods during application startup based on the pre-run collected > data. > > The main flags (-XX:) used to control are: > CompilationWarmUpRecording: used for collecting compilation data. > Result stored in a file set by flag CompilationWarmUpLogfile. > CompilationWarmUp: use collected data to precompile the methods > recorded in above file. > CompilationWarmUpExclude: configure a file which contains the names to > be excluded for precompile. > CompilationWarmUpRecordTime: control the warmup recording time. > CompilationWarmUpDeoptTime: time (from start) to deoptimize precomiled methods. > > In this version, the profiled method data is not used at > precomilation, it will be addressed in followed bug fix. After the > first version integrated, will file bug for it. Also, it does not work > with CDS since the data added to ConstantPool data structure. It also > will be addressed in another separate bug followed. > > Test: Submitted to submit repo and passed all tests. > Newly added test suites: test/hotspot/jtreg/jwarmup include test > cases for most of the use cases and issues. > > Thanks > Yumin > From coleen.phillimore at oracle.com Fri Mar 22 16:32:37 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 22 Mar 2019 12:32:37 -0400 Subject: RFR (S) 8074355: make MutexLocker smarter about non-JavaThreads Message-ID: <80323289-ab58-6f81-d56c-4a6421d29374@oracle.com> Summary: Use safepoint_check_always/safepoint_check_never instead of safepoint_check_sometimes for locks that are taken by JavaThreads and non-JavaThreads This is a small cleanup to avoid having safepoint_check_sometimes. Threads_lock is the only one remaining.? 'Sometimes' was used for some locks because they were taken out by GC or other threads and JavaThreads, where the other threads weren't going to check for the safepoint anyway since they don't participate in the safepoint protocol.? There could be further cleanups here.? We could make the safepoint checking related to the rank of the lock in the future, or the safepoint checking could be a property of the lock so we don't have to pass in Mutex::_no_safepoint_check_flag (except Threads_lock).? I think these could be further enhancements after this. Tested with hs-tier1-6, runThese jck with G1, CMS, Serial and ParallelGC. 8074355: make MutexLocker smarter about non-JavaThreads open webrev at http://cr.openjdk.java.net/~coleenp/2019/8074355.01/webrev Thanks, Coleen From bob.vandette at oracle.com Fri Mar 22 18:25:50 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 22 Mar 2019 14:25:50 -0400 Subject: RFR: 8217338: [Containers] Improve systemd slice memory limit support In-Reply-To: <4a1ae4e0e3efd68818bfe868349972aa878de60b.camel@redhat.com> References: <4a1ae4e0e3efd68818bfe868349972aa878de60b.camel@redhat.com> Message-ID: Is there ever a situation where the memory.limit_in_bytes could be unlimited but the hierarchical_memory_limit is not? Could you maybe combine subsystem_file_contents with subsystem_file_line_contents by adding an additional argument? BTW: I found another problem with the mountinfo/cgroup parsing that impacts the container tests. I don?t know why it only caused a failure on one of my systems. I?m going to file another bug. You might want to test with these changes to make sure your looking at the correct subsystem files. diff --git a/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java b/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java --- a/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java +++ b/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java @@ -60,7 +60,7 @@ path = mountPoint; } else { - if (root.indexOf(cgroupPath) == 0) { + if (cgroupPath.indexOf(root) == 0) { if (cgroupPath.length() > root.length()) { String cgroupSubstr = cgroupPath.substring(root.length()); path = mountPoint + cgroupSubstr; diff --git a/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java b/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java --- a/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java +++ b/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java @@ -85,7 +85,7 @@ String mountPoint = paths[1]; if (root != null && cgroupPath != null) { if (root.equals("/")) { - if (cgroupPath.equals("/")) { + if (!cgroupPath.equals("/")) { finalPath = mountPoint + cgroupPath; } else { finalPath = mountPoint; @@ -94,7 +94,7 @@ if (root.equals(cgroupPath)) { finalPath = mountPoint; } else { - if (root.indexOf(cgroupPath) == 0) { + if (cgroupPath.indexOf(root) == 0) { if (cgroupPath.length() > root.length()) { String cgroupSubstr = cgroupPath.substring(root.length()); finalPath = mountPoint + cgroupSubstr; @@ -103,7 +103,7 @@ } } } - subSystemPaths.put(subSystem, new String[]{finalPath}); + subSystemPaths.put(subSystem, new String[]{finalPath, mountPoint}); } } } Bob. > On Mar 22, 2019, at 6:43 AM, Severin Gehwolf wrote: > > Hi, > > Please review this change which improves container detection support > tin the JVM. While docker container detection works quite well the > results for systemd slices with memory limits are mixed and depend on > the Linux kernel version in use. With newer kernel versions becoming > more widely used we should improve JVMs memory limit detection support > as well. This should be entirely backwards compatible as the > hierarchical limit will only be used if everything else is unlimited. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217338 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8217338/04/webrev/ > > Testing: Manual testing of -XshowSettings and -Xlog:os+container=trace > with systemd slices on affected Linux distributions: Fedora 29, > recent Ubuntu (18-10). Existing docker tests pass. I'm currently also > running this through jdk/submit. > > Thoughts? > > Thanks, > Severin > From coleen.phillimore at oracle.com Fri Mar 22 20:37:48 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 22 Mar 2019 16:37:48 -0400 Subject: RFR (T) 8218399: runtime/RedefineObject/TestRedefineObject.java timeout Message-ID: Summary: exclude this test for graal open webrev at http://cr.openjdk.java.net/~coleenp/2019/8218399.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8218399 See bug report for more details.? Tested that it's not run with: mach5 remote-build-and-test -b macosx-x64-debug --job hs-tier4-graal Thanks, Coleen From harold.seigel at oracle.com Fri Mar 22 20:39:44 2019 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 22 Mar 2019 16:39:44 -0400 Subject: RFR (T) 8218399: runtime/RedefineObject/TestRedefineObject.java timeout In-Reply-To: References: Message-ID: <01be72f9-a805-db6c-f353-e03251173e9a@oracle.com> Hi Coleen, Looks good, and trivial. Harold On 3/22/2019 4:37 PM, coleen.phillimore at oracle.com wrote: > Summary: exclude this test for graal > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8218399.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8218399 > > See bug report for more details.? Tested that it's not run with: > mach5 remote-build-and-test -b macosx-x64-debug --job hs-tier4-graal > > Thanks, > Coleen From lois.foltan at oracle.com Fri Mar 22 20:40:23 2019 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 22 Mar 2019 16:40:23 -0400 Subject: RFR (T) 8218399: runtime/RedefineObject/TestRedefineObject.java timeout In-Reply-To: References: Message-ID: Looks good and trivial. Lois On 3/22/2019 4:37 PM, coleen.phillimore at oracle.com wrote: > Summary: exclude this test for graal > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8218399.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8218399 > > See bug report for more details.? Tested that it's not run with: > mach5 remote-build-and-test -b macosx-x64-debug --job hs-tier4-graal > > Thanks, > Coleen From coleen.phillimore at oracle.com Fri Mar 22 20:41:34 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 22 Mar 2019 16:41:34 -0400 Subject: RFR (T) 8218399: runtime/RedefineObject/TestRedefineObject.java timeout In-Reply-To: References: Message-ID: <0a1cf069-6941-6378-8d2c-462084bcef88@oracle.com> Thanks Lois and Harold. Coleen On 3/22/19 4:40 PM, Lois Foltan wrote: > Looks good and trivial. > Lois > > On 3/22/2019 4:37 PM, coleen.phillimore at oracle.com wrote: >> Summary: exclude this test for graal >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8218399.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8218399 >> >> See bug report for more details.? Tested that it's not run with: >> mach5 remote-build-and-test -b macosx-x64-debug --job hs-tier4-graal >> >> Thanks, >> Coleen > From rkennke at redhat.com Fri Mar 22 22:00:48 2019 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 22 Mar 2019 23:00:48 +0100 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> Message-ID: >> Please review this change to the thread claiming protocol for parallel >> iteration over threads. Previously, the protocol used a 1bit claim >> counter plus zero as a "never claimed" marker. The new protocol uses a >> uintx counter, while retaining zero as a "never claimed" marker. >> >> JDK-8219613 changed use use of the claiming protocol, allowing all >> threads to be included and claimed by a parallel iteration.? Doing so >> introduced a bug. >> >> Not all parallel iteration paths visit every thread. In particular, >> possibly_parallel_threads_do (PPTD) only visits Java threads and the >> VM thread. Using both all-thread parallel iterations and PPTD could >> lead to some threads being improperly skipped, because of the 1bit >> claim counter. For example, if an all-thread iteration is followed by >> an even number of PPTD then another all-thread iteration, the second >> all-thread iteration will incorrectly skip NonJavaThreads other than >> the VMThread. >> >> (Testing these changes with an instrumented claim function that >> detected cases that would be in trouble with the old 1bit counter >> found *many* occurrences.? It is somewhat surprising that this doesn't >> seem to have produced lots of failures since JDK-8219613.) >> >> We address this by expanding the claim counter to a uintx-sized >> counter. There is still a theoretical possibility of collision, which >> we deal with by resetting the claim value of all threads back to zero >> when the global counter overflows. (On a 32bit platform, overflow of >> the claim counter probably takes months. And even that is likely more >> often than needed, since a collision needs MAX_UINTX-1 PPTD iterations >> between all-thread iterations, which isn't how iterations are used.) >> >> Some renaming was done to generalize from the old "parity" >> nomenclature, and because it's not just about "oops_do" (and wasn't >> before this change). Hopefully there won't be *too* much bike-shedding >> on the names. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8221102 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8221102/open.00/ >> >> Testing: >> mach5 tier1-5. > > This would work. > > I am going back and forth with this in my head. I believe the simpler, > more usable and perhaps less surprising approach might be to simply > always iterate all threads, and do not even let other code use the > claiming protocol, and care about its idiosyncrasies: > > http://cr.openjdk.java.net/~rkennke/JDK-8221102/webrev.01/ > > This way the claim token would always go in lock-step. The overhead of > iterating a few more non-Java-threads (somewhat likely doing nothing) > seems not very significant. > > This could probably be taken a little further and hiding more of the > claiming protocol. Thread::claim_oops_do() doesn't exactly need to be > public anymore. Thread::thread_claim_parity() doesn't appear to be used > anywhere outside Thread, except for an assert which could be done > somehow else, and change_thread_claim_parity() could be done via friend > classes or so. > > I don't have a strong preference for your vs my solution, except that I > want this bug fixed. :-) Just wanted to throw it out so that others can > chime into the discussion. And in-fact, both solutions could be combined, they are not mutually exclusive. Not sure what the point would be though ;-) I think it's nice to not expose the claiming logic to any outside code though. Roman From sgehwolf at redhat.com Mon Mar 25 13:01:51 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 25 Mar 2019 14:01:51 +0100 Subject: RFR: 8217338: [Containers] Improve systemd slice memory limit support In-Reply-To: References: <4a1ae4e0e3efd68818bfe868349972aa878de60b.camel@redhat.com> Message-ID: <9f743d16db44bc87ddb0a4aa275538ba3260fe9c.camel@redhat.com> On Fri, 2019-03-22 at 14:25 -0400, Bob Vandette wrote: > Is there ever a situation where the memory.limit_in_bytes could be unlimited but the > hierarchical_memory_limit is not? Yes. This is the very bug. Under a systemd slice with newer kernels the memory.limit_in_bytes file OpenJDK looks at has unlimited, but memory.stat on the same hierarchy has the correct setting. Patched JDK with -Xlog:os+container=trace [0.051s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf53fa46d5cc47ce857fbfaae04459c1.scope/memory.limit_in_bytes [0.051s][trace][os,container] Memory Limit is: 9223372036854771712 [0.051s][trace][os,container] Non-Hierarchical Memory Limit is: Unlimited [0.051s][trace][os,container] Path to /memory.stat is /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf53fa46d5cc47ce857fbfaae04459c1.scope/memory.stat [0.051s][trace][os,container] Hierarchical Memory Limit is: 10485760 $ cat /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf53fa46d5cc47ce857fbfaae04459c1.scope/memory.limit_in_bytes 9223372036854771712 $ cat /sys/fs/cgroup/memory/user.slice/user-cg.slice/memory.limit_in_bytes 10485760 $ grep hierarchical /sys/fs/cgroup/memory/user.slice/user-cg.slice/run-raf53fa46d5cc47ce857fbfaae04459c1.scope/memory.stat hierarchical_memory_limit 10485760 hierarchical_memsw_limit 9223372036854771712 This was mentioned in the bug comments as well. > Could you maybe combine subsystem_file_contents with subsystem_file_line_contents > by adding an additional argument? Sure, I could look into that. I'll post an updated webrev soon. Thanks for the review! Cheers, Severin > BTW: I found another problem with the mountinfo/cgroup parsing that impacts the > container tests. > > I don?t know why it only caused a failure on one of my systems. I?m going to > file another bug. You might want to test with these changes to make sure > your looking at the correct subsystem files. OK. Thanks for the heads-up. It looks unrelated, though. > > diff --git a/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java b/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java > --- a/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java > +++ b/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java > @@ -60,7 +60,7 @@ > path = mountPoint; > } > else { > - if (root.indexOf(cgroupPath) == 0) { > + if (cgroupPath.indexOf(root) == 0) { > if (cgroupPath.length() > root.length()) { > String cgroupSubstr = cgroupPath.substring(root.length()); > path = mountPoint + cgroupSubstr; > diff --git a/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java b/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java > --- a/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java > +++ b/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java > @@ -85,7 +85,7 @@ > String mountPoint = paths[1]; > if (root != null && cgroupPath != null) { > if (root.equals("/")) { > - if (cgroupPath.equals("/")) { > + if (!cgroupPath.equals("/")) { > finalPath = mountPoint + cgroupPath; > } else { > finalPath = mountPoint; > @@ -94,7 +94,7 @@ > if (root.equals(cgroupPath)) { > finalPath = mountPoint; > } else { > - if (root.indexOf(cgroupPath) == 0) { > + if (cgroupPath.indexOf(root) == 0) { > if (cgroupPath.length() > root.length()) { > String cgroupSubstr = cgroupPath.substring(root.length()); > finalPath = mountPoint + cgroupSubstr; > @@ -103,7 +103,7 @@ > } > } > } > - subSystemPaths.put(subSystem, new String[]{finalPath}); > + subSystemPaths.put(subSystem, new String[]{finalPath, mountPoint}); > } > } > } > > > Bob. > > > > > On Mar 22, 2019, at 6:43 AM, Severin Gehwolf wrote: > > > > Hi, > > > > Please review this change which improves container detection support > > tin the JVM. While docker container detection works quite well the > > results for systemd slices with memory limits are mixed and depend on > > the Linux kernel version in use. With newer kernel versions becoming > > more widely used we should improve JVMs memory limit detection support > > as well. This should be entirely backwards compatible as the > > hierarchical limit will only be used if everything else is unlimited. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217338 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8217338/04/webrev/ > > > > Testing: Manual testing of -XshowSettings and -Xlog:os+container=trace > > with systemd slices on affected Linux distributions: Fedora 29, > > recent Ubuntu (18-10). Existing docker tests pass. I'm currently also > > running this through jdk/submit. > > > > Thoughts? > > > > Thanks, > > Severin > > From sgehwolf at redhat.com Mon Mar 25 17:30:48 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 25 Mar 2019 18:30:48 +0100 Subject: RFR: 8217338: [Containers] Improve systemd slice memory limit support In-Reply-To: References: <4a1ae4e0e3efd68818bfe868349972aa878de60b.camel@redhat.com> Message-ID: <9a911da4b2a4a8eff42c7c2f1a19c06f6521da8d.camel@redhat.com> On Fri, 2019-03-22 at 14:25 -0400, Bob Vandette wrote: > Could you maybe combine subsystem_file_contents with subsystem_file_line_contents > by adding an additional argument? Done: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8217338/05/webrev/ Thanks, Severin From coleen.phillimore at oracle.com Mon Mar 25 17:49:20 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 25 Mar 2019 13:49:20 -0400 Subject: RFR (S) 8160539: Stack frame scanning acquires several mutexes Message-ID: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> Summary: Use CAS instead of OopMapCacheAlloc_lock Tested with hs-tier1-5 and specjbb2017. open webrev at http://cr.openjdk.java.net/~coleenp/2019/8160539.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8160539 Thanks, Coleen From coleen.phillimore at oracle.com Mon Mar 25 17:51:48 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 25 Mar 2019 13:51:48 -0400 Subject: RFR (S) 8160539: Stack frame scanning acquires several mutexes In-Reply-To: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> References: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> Message-ID: On 3/25/19 1:49 PM, coleen.phillimore at oracle.com wrote: > Summary: Use CAS instead of OopMapCacheAlloc_lock > > Tested with hs-tier1-5 and specjbb2017. I mean 2015. Coleen > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8160539.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8160539 > > Thanks, > Coleen From dthomson at google.com Mon Mar 25 18:10:56 2019 From: dthomson at google.com (Derek Thomson) Date: Mon, 25 Mar 2019 11:10:56 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> <7e3b31a1-e0a0-b156-09f2-eef737268ca8@redhat.com> Message-ID: Okay, JC ran the tests, with the following result: Job: mach5-one-jcbeyler-JDK-8157372-2-20190323-1437-1389995 BuildId: 2019-03-23-1433293.jcbeyler.source No failed tests Tasks Summary NA: 0 EXECUTED_WITH_FAILURE: 0 HARNESS_ERROR: 0 FAILED: 0 UNABLE_TO_RUN: 0 KILLED: 0 NOTHING_TO_RUN: 0 PASSED: 76 On Wed, Mar 20, 2019 at 9:39 AM Derek Thomson wrote: > I figured, I'll bug him to run the tests then as I suspect you'll need > Committer privileges. Thanks again, that saves me a lot of time trying to > make this work. > > On Wed, Mar 20, 2019 at 9:37 AM Aleksey Shipilev wrote: > >> On 3/20/19 5:33 PM, Derek Thomson wrote: >> > Thanks Aleksey - does this mean I need a userid? I don't have one. I've >> been following this guide >> > and can't find any mention of how to do that (or anywhere else): >> > https://openjdk.java.net/contribute/ If I need it, can I get one? >> >> Yes, you need OpenJDK id (and maybe Committer privileges, not sure). JC >> certainly has it, you can >> ask him: http://openjdk.java.net/census#jcbeyler >> >> -Aleksey >> >> From vladimir.kozlov at oracle.com Mon Mar 25 18:55:18 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 25 Mar 2019 11:55:18 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> <7e3b31a1-e0a0-b156-09f2-eef737268ca8@redhat.com> Message-ID: Good. I think you need second review to make sure nothing is missing. Note, I thought to suggest to fix Type::cmp() too but that is more complicated and outside this RFE. After reviews you need sponsor to push (person with Committer privilege). Thanks, Vladimir On 3/25/19 11:10 AM, Derek Thomson wrote: > Okay, JC ran the tests, with the following result: > > Job: mach5-one-jcbeyler-JDK-8157372-2-20190323-1437-1389995 > BuildId: 2019-03-23-1433293.jcbeyler.source > No failed tests > Tasks Summary > NA: 0 > EXECUTED_WITH_FAILURE: 0 > HARNESS_ERROR: 0 > FAILED: 0 > UNABLE_TO_RUN: 0 > KILLED: 0 > NOTHING_TO_RUN: 0 > PASSED: 76 > > > On Wed, Mar 20, 2019 at 9:39 AM Derek Thomson wrote: > >> I figured, I'll bug him to run the tests then as I suspect you'll need >> Committer privileges. Thanks again, that saves me a lot of time trying to >> make this work. >> >> On Wed, Mar 20, 2019 at 9:37 AM Aleksey Shipilev wrote: >> >>> On 3/20/19 5:33 PM, Derek Thomson wrote: >>>> Thanks Aleksey - does this mean I need a userid? I don't have one. I've >>> been following this guide >>>> and can't find any mention of how to do that (or anywhere else): >>>> https://openjdk.java.net/contribute/ If I need it, can I get one? >>> >>> Yes, you need OpenJDK id (and maybe Committer privileges, not sure). JC >>> certainly has it, you can >>> ask him: http://openjdk.java.net/census#jcbeyler >>> >>> -Aleksey >>> >>> From kim.barrett at oracle.com Mon Mar 25 19:42:18 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 25 Mar 2019 15:42:18 -0400 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> Message-ID: <43C7702A-C046-4E9A-9773-889E7741FBF1@oracle.com> > On Mar 22, 2019, at 6:00 PM, Roman Kennke wrote: > >>> Please review this change to the thread claiming protocol for parallel >>> iteration over threads. Previously, the protocol used a 1bit claim >>> counter plus zero as a "never claimed" marker. The new protocol uses a >>> uintx counter, while retaining zero as a "never claimed" marker. >>> >>> [?] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8221102 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8221102/open.00/ >>> >>> Testing: >>> mach5 tier1-5. >> >> This would work. >> >> I am going back and forth with this in my head. I believe the simpler, >> more usable and perhaps less surprising approach might be to simply >> always iterate all threads, and do not even let other code use the >> claiming protocol, and care about its idiosyncrasies: >> >> [?] >> I don't have a strong preference for your vs my solution, except that I >> want this bug fixed. :-) Just wanted to throw it out so that others can >> chime into the discussion. > > And in-fact, both solutions could be combined, they are not mutually > exclusive. Not sure what the point would be though ;-) I think it's nice > to not expose the claiming logic to any outside code though. > > Roman It *might* work. And it was one of the options I initially suggested. But all of the relevant closures need to be examined to ensure they don't have any assumptions that would make them not work properly when applied to some non-VM NJT. I don't want to do that audit as part of fixing the bug if that can be avoided. And I think extending the counter as I've proposed avoids the need for that audit. It also has the benefit of allowing subset iterations, which seems like it could be useful in other ways (though we presently only have the two sets). We could give less exposure to the claiming mechanism by providing a second function similar to the existing possibly_parallel_threads_do that does hit all the threads. The only problem I have with that is naming. Maybe the old function should be renamed to possibly_parallel_java_threads_and_vm_thread_do, for consistency with the serial function. From rkennke at redhat.com Mon Mar 25 20:19:16 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 25 Mar 2019 21:19:16 +0100 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: <43C7702A-C046-4E9A-9773-889E7741FBF1@oracle.com> References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> <43C7702A-C046-4E9A-9773-889E7741FBF1@oracle.com> Message-ID: <7571856a-5809-ab9d-c2be-5b6855be5c3c@redhat.com> Am 25.03.19 um 20:42 schrieb Kim Barrett: >> On Mar 22, 2019, at 6:00 PM, Roman Kennke wrote: >> >>>> Please review this change to the thread claiming protocol for parallel >>>> iteration over threads. Previously, the protocol used a 1bit claim >>>> counter plus zero as a "never claimed" marker. The new protocol uses a >>>> uintx counter, while retaining zero as a "never claimed" marker. >>>> >>>> [?] >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8221102 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~kbarrett/8221102/open.00/ >>>> >>>> Testing: >>>> mach5 tier1-5. >>> >>> This would work. >>> >>> I am going back and forth with this in my head. I believe the simpler, >>> more usable and perhaps less surprising approach might be to simply >>> always iterate all threads, and do not even let other code use the >>> claiming protocol, and care about its idiosyncrasies: >>> >>> [?] >>> I don't have a strong preference for your vs my solution, except that I >>> want this bug fixed. :-) Just wanted to throw it out so that others can >>> chime into the discussion. >> >> And in-fact, both solutions could be combined, they are not mutually >> exclusive. Not sure what the point would be though ;-) I think it's nice >> to not expose the claiming logic to any outside code though. >> >> Roman > > It *might* work. And it was one of the options I initially suggested. > But all of the relevant closures need to be examined to ensure they > don't have any assumptions that would make them not work properly when > applied to some non-VM NJT. I don't want to do that audit as part of > fixing the bug if that can be avoided. And I think extending the > counter as I've proposed avoids the need for that audit. It also has > the benefit of allowing subset iterations, which seems like it could > be useful in other ways (though we presently only have the two sets). > > We could give less exposure to the claiming mechanism by providing a > second function similar to the existing possibly_parallel_threads_do > that does hit all the threads. The only problem I have with that is > naming. Maybe the old function should be renamed to > possibly_parallel_java_threads_and_vm_thread_do, for consistency with > the serial function. Zhengyu found a problem with my approach to include all workers in the protocol: workers that would be created at safepoints would start out with 0 claim token, and the final verification would fail because it wants all threads to be at global thread_parity. It could be fixed fairly easily by letting all threads start with current global parity. But as you say, this might just be one case of assumptions and there might be more lurking around. So let's go with your approach for now. Roman From kim.barrett at oracle.com Mon Mar 25 22:16:11 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 25 Mar 2019 18:16:11 -0400 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: <7571856a-5809-ab9d-c2be-5b6855be5c3c@redhat.com> References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> <43C7702A-C046-4E9A-9773-889E7741FBF1@oracle.com> <7571856a-5809-ab9d-c2be-5b6855be5c3c@redhat.com> Message-ID: > On Mar 25, 2019, at 4:19 PM, Roman Kennke wrote: > > > > Am 25.03.19 um 20:42 schrieb Kim Barrett: >> >> We could give less exposure to the claiming mechanism by providing a >> second function similar to the existing possibly_parallel_threads_do >> that does hit all the threads. The only problem I have with that is >> naming. Maybe the old function should be renamed to >> possibly_parallel_java_threads_and_vm_thread_do, for consistency with >> the serial function. > > Zhengyu found a problem with my approach to include all workers in the > protocol: workers that would be created at safepoints would start out > with 0 claim token, and the final verification would fail because it > wants all threads to be at global thread_parity. Yes, that could be a problem. For the current cases where we do iterate over all threads, we get away with it because we don't do the final thread count verification; that verification is only associated with the subset iteration of possibly_parallel_oops_do. Interesting. The couple of places currently doing all-thread iteration look like they should be fine with missing some that are under construction. But that kind of brings us back around to the NJT initialization race question from the other thread. I'll continue this thought over there. > It could be fixed > fairly easily by letting all threads start with current global parity. I suspect that introduces a race to get the current global parity. > But as you say, this might just be one case of assumptions and there > might be more lurking around. So let's go with your approach for now. > > Roman OK. So does that count as a "review"? And I still need a second. From dthomson at google.com Mon Mar 25 22:50:03 2019 From: dthomson at google.com (Derek Thomson) Date: Mon, 25 Mar 2019 15:50:03 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: <57f48336-3476-4dd6-1413-af8595cd3d34@oracle.com> <65c4624d-05a9-6155-1da6-557d4554b3bd@redhat.com> <7e3b31a1-e0a0-b156-09f2-eef737268ca8@redhat.com> Message-ID: Thanks Vladimir - is there a process for asking for a second reviewer? On Mon, Mar 25, 2019 at 11:55 AM Vladimir Kozlov wrote: > Good. > > I think you need second review to make sure nothing is missing. Note, I > thought to suggest to fix Type::cmp() too but > that is more complicated and outside this RFE. > > After reviews you need sponsor to push (person with Committer privilege). > > Thanks, > Vladimir > > On 3/25/19 11:10 AM, Derek Thomson wrote: > > Okay, JC ran the tests, with the following result: > > > > Job: mach5-one-jcbeyler-JDK-8157372-2-20190323-1437-1389995 > > BuildId: 2019-03-23-1433293.jcbeyler.source > > No failed tests > > Tasks Summary > > NA: 0 > > EXECUTED_WITH_FAILURE: 0 > > HARNESS_ERROR: 0 > > FAILED: 0 > > UNABLE_TO_RUN: 0 > > KILLED: 0 > > NOTHING_TO_RUN: 0 > > PASSED: 76 > > > > > > On Wed, Mar 20, 2019 at 9:39 AM Derek Thomson > wrote: > > > >> I figured, I'll bug him to run the tests then as I suspect you'll need > >> Committer privileges. Thanks again, that saves me a lot of time trying > to > >> make this work. > >> > >> On Wed, Mar 20, 2019 at 9:37 AM Aleksey Shipilev > wrote: > >> > >>> On 3/20/19 5:33 PM, Derek Thomson wrote: > >>>> Thanks Aleksey - does this mean I need a userid? I don't have one. > I've > >>> been following this guide > >>>> and can't find any mention of how to do that (or anywhere else): > >>>> https://openjdk.java.net/contribute/ If I need it, can I get one? > >>> > >>> Yes, you need OpenJDK id (and maybe Committer privileges, not sure). JC > >>> certainly has it, you can > >>> ask him: http://openjdk.java.net/census#jcbeyler > >>> > >>> -Aleksey > >>> > >>> > From vladimir.x.ivanov at oracle.com Mon Mar 25 23:08:33 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 25 Mar 2019 16:08:33 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: Message-ID: Looks good. Best regards, Vladimir Ivanov On 19/03/2019 11:08, Derek Thomson wrote: > Changed uint to bool throughout. Could someone review my change please? > Also, what tests should I run to validate this? > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8157372/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8157372 > From dean.long at oracle.com Tue Mar 26 00:14:40 2019 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 25 Mar 2019 17:14:40 -0700 Subject: RFR (S) 8160539: Stack frame scanning acquires several mutexes In-Reply-To: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> References: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> Message-ID: <5b364241-d535-f2de-c1ae-d6c76c15088e@oracle.com> Is it OK to do a normal read of _oop_map_cache here: +??? if (!Atomic::replace_if_null(oop_map_cache, &_oop_map_cache)) { +????? // Some other thread won. +????? delete oop_map_cache; +????? oop_map_cache = _oop_map_cache; ???? } or does it need to be load-acquire or volatile?? We don't want the compiler to use the value from the earlier read: OopMapCache* oop_map_cache = OrderAccess::load_acquire(&_oop_map_cache); dl On 3/25/19 10:49 AM, coleen.phillimore at oracle.com wrote: > Summary: Use CAS instead of OopMapCacheAlloc_lock > > Tested with hs-tier1-5 and specjbb2017. > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8160539.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8160539 > > Thanks, > Coleen From david.holmes at oracle.com Tue Mar 26 00:27:10 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 26 Mar 2019 10:27:10 +1000 Subject: RFR (S) 8160539: Stack frame scanning acquires several mutexes In-Reply-To: <5b364241-d535-f2de-c1ae-d6c76c15088e@oracle.com> References: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> <5b364241-d535-f2de-c1ae-d6c76c15088e@oracle.com> Message-ID: Hi Dean, On 26/03/2019 10:14 am, dean.long at oracle.com wrote: > Is it OK to do a normal read of _oop_map_cache here: > > +??? if (!Atomic::replace_if_null(oop_map_cache, &_oop_map_cache)) { > +????? // Some other thread won. > +????? delete oop_map_cache; > +????? oop_map_cache = _oop_map_cache; > ???? } > > or does it need to be load-acquire or volatile?? We don't want the > compiler to use the value from the earlier read: From atomic.hpp: // All of the atomic operations that imply a read-modify-write action // guarantee a two-way memory barrier across that operation. So semantically the above should be correct (some platforms have bugs where the barrier is missing when the CAS fails). That said it would seem preferable to me to use cmpxchg if we actually want to use the value when we lose the race. Cheers, David ----- > > ?? OopMapCache* oop_map_cache = > OrderAccess::load_acquire(&_oop_map_cache); > > > dl > > On 3/25/19 10:49 AM, coleen.phillimore at oracle.com wrote: >> Summary: Use CAS instead of OopMapCacheAlloc_lock >> >> Tested with hs-tier1-5 and specjbb2017. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/2019/8160539.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8160539 >> >> Thanks, >> Coleen > From david.holmes at oracle.com Tue Mar 26 00:34:07 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 26 Mar 2019 10:34:07 +1000 Subject: RFR (S) 8160539: Stack frame scanning acquires several mutexes In-Reply-To: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> References: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> Message-ID: Hi Coleen, On 26/03/2019 3:49 am, coleen.phillimore at oracle.com wrote: > Summary: Use CAS instead of OopMapCacheAlloc_lock > > Tested with hs-tier1-5 and specjbb2017. > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8160539.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8160539 How expensive is it to create OopMapCache? Are we likely to encounter a lot of contention on this? As I mentioned in my reply to Dean I think it mildly preferable to use cmpxchg if you want to use the value in case you lose the race: OopMapCache * temp = Atomic::cmpxchg(oop_map_cache, &_oop_map_cache, NULL) if (temp != NULL) { // Some other thread won. delete oop_map_cache; oop_map_cache = temp; } Thanks, David > Thanks, > Coleen From kim.barrett at oracle.com Tue Mar 26 01:03:27 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 25 Mar 2019 21:03:27 -0400 Subject: RFR (S) 8160539: Stack frame scanning acquires several mutexes In-Reply-To: References: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> Message-ID: > On Mar 25, 2019, at 8:34 PM, David Holmes wrote: > > Hi Coleen, > > On 26/03/2019 3:49 am, coleen.phillimore at oracle.com wrote: >> Summary: Use CAS instead of OopMapCacheAlloc_lock >> Tested with hs-tier1-5 and specjbb2017. >> open webrev at http://cr.openjdk.java.net/~coleenp/2019/8160539.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8160539 > > How expensive is it to create OopMapCache? Are we likely to encounter a lot of contention on this? I had the same question. I suspect the added cognitive load of lock-free code (vs the existing DCLP) is not worthwhile here. From tobias.hartmann at oracle.com Tue Mar 26 07:52:55 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 26 Mar 2019 08:52:55 +0100 Subject: RFC: JWarmup precompile java hot methods at application startup In-Reply-To: <1a9e9d8c-9a80-4cf7-9510-7de1cc9cbc4e.kuaiwei.kw@alibaba-inc.com> References: <1a9e9d8c-9a80-4cf7-9510-7de1cc9cbc4e.kuaiwei.kw@alibaba-inc.com> Message-ID: On 22.03.19 13:15, Kuai Wei wrote: > jwarmup is designed to work on linux x86-64 only. So we have not tested on other platform.? Okay, in this case it should not be possible to enable the feature on any other platform (if the corresponding flag is set, a "... not supported on this platform" warning should be issued and the flag should be disabled, see examples in arguments.cpp). Also, the corresponding tests should only be executed on supported platforms. Best regards, Tobias From matthias.baesken at sap.com Tue Mar 26 08:32:45 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 26 Mar 2019 08:32:45 +0000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> Message-ID: Hi David, > This is looking better though I'm struggling to keep all the dots > connected in terms of the APIs - I can't quite see the pattern for what > calls what and what overrides what. I just followed here what is done at other places of VM_Version / Abstract_VM_Version , see for example : src/hotspot/share/runtime/vm_version.hpp 82 // ..... Platforms that 83 // need to specialize this define VM_Version::early_initialize(). 84 static void early_initialize() { } 85 86 // Called to initialize VM variables needing initialization 87 // after command line parsing. Platforms that need to specialize 88 // this should define VM_Version::init_before_ergo(). 89 static void init_before_ergo() {} > 2. os_windows.cpp > > Why the indirection through os::win32::print_virtualization_info instead > of just calling VM_Version::print_detected_virtualization directly? Yes that's a good point - I think I should change this . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Mittwoch, 20. M?rz 2019 07:15 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Cc: Doerr, Martin > Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > hs_error file on linux/windows x86_64 > > Hi Matthias, > > On 19/03/2019 11:53 pm, Baesken, Matthias wrote: > > Hello, here is a new webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ > > > > This moved the ppc/s390 virtualization related info output into the > platform/cpu - specific coding . > > This is looking better though I'm struggling to keep all the dots > connected in terms of the APIs - I can't quite see the pattern for what > calls what and what overrides what. A couple of initial comments: > > 1. We already have a ton of cpuid related parsing in > ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check > virtualizations" and store the information for later use? > > 2. os_windows.cpp > > Why the indirection through os::win32::print_virtualization_info instead > of just calling VM_Version::print_detected_virtualization directly? > > I'll keep trying to piece this together. :) > > Thanks, > David > ----- > > > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Montag, 11. M?rz 2019 08:36 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Cc: Doerr, Martin > >> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > >> hs_error file on linux/windows x86_64 > >> > >> On 7/03/2019 3:13 am, Baesken, Matthias wrote: > >>> Hello , could I push the latest version of 8219241 ? > >> > >> Sorry based on your earlier email I thought you were looking at doing > >> the restructuring. I was then on vacation most of last week. > >> > >>> A change that moves the coding more into platform/cpu - specific > coding > >> (if this is wanted) > >>> could be done when bringing in the AIX virtualization related info in > >> another patch which I plan to do . > >> > >> If that is imminent then okay - I don't like the code as it is. > >> > >> Thanks, > >> David > >> > > From Alan.Bateman at oracle.com Tue Mar 26 10:10:40 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Mar 2019 10:10:40 +0000 Subject: RFC: JWarmup precompile java hot methods at application startup In-Reply-To: <020701d4debf$22110e90$66332bb0$@alibaba-inc.com> References: <020701d4debf$22110e90$66332bb0$@alibaba-inc.com> Message-ID: On 20/03/2019 01:49, ???(??) wrote: > HI Alan, > Thanks for comment. We can put com.alibaba.jwarmup in separate new module. > Any further suggestion for the package name? > Or anyone can give us some options for considering:-) > If this feature goes ahead then I assume module and package name for the API will be discussed. Given that it's a JDK-specific feature then I assume it will be in the jdk.* name space. The more significant part to this is the need for an application to notify the API that it has completed its warm-up, something that will need to explained in the JEP. -Alan. From thomas.schatzl at oracle.com Tue Mar 26 10:12:32 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 26 Mar 2019 11:12:32 +0100 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> Message-ID: <44663bc09a529c415470ce609edc2e1f59f9b97b.camel@oracle.com> Hi, On Thu, 2019-03-21 at 18:25 -0400, Kim Barrett wrote: > Please review this change to the thread claiming protocol for > parallel iteration over threads. Previously, the protocol used a 1bit > claim counter plus zero as a "never claimed" marker. The new protocol > uses a uintx counter, while retaining zero as a "never claimed" > marker. > > [...] > We address this by expanding the claim counter to a uintx-sized > counter. There is still a theoretical possibility of collision, which > [...] > > CR: > https://bugs.openjdk.java.net/browse/JDK-8221102 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8221102/open.00/ > > Testing: > mach5 tier1-5. > looks good. One minor style nit: I would prefer that in thread.cpp: Threads::change_hread_claim_token():4521 the thread_claim_token would be set to a particular value (i.e. 1) directly instead of incremented from its current original value. This makes that assignment context- free. But I am fine with keeping it as is. Thanks, Thomas From rkennke at redhat.com Tue Mar 26 10:31:57 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 26 Mar 2019 11:31:57 +0100 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> <43C7702A-C046-4E9A-9773-889E7741FBF1@oracle.com> <7571856a-5809-ab9d-c2be-5b6855be5c3c@redhat.com> Message-ID: <6fcffd19-a198-9ee5-46a5-dcbf3c3ed2a9@redhat.com> Hi Kim, >>> We could give less exposure to the claiming mechanism by providing a >>> second function similar to the existing possibly_parallel_threads_do >>> that does hit all the threads. The only problem I have with that is >>> naming. Maybe the old function should be renamed to >>> possibly_parallel_java_threads_and_vm_thread_do, for consistency with >>> the serial function. >> >> Zhengyu found a problem with my approach to include all workers in the >> protocol: workers that would be created at safepoints would start out >> with 0 claim token, and the final verification would fail because it >> wants all threads to be at global thread_parity. > > Yes, that could be a problem. For the current cases where we do > iterate over all threads, we get away with it because we don't do the > final thread count verification; that verification is only associated > with the subset iteration of possibly_parallel_oops_do. Interesting. > The couple of places currently doing all-thread iteration look like > they should be fine with missing some that are under construction. > But that kind of brings us back around to the NJT initialization race > question from the other thread. I'll continue this thought over there. > >> It could be fixed >> fairly easily by letting all threads start with current global parity. > > I suspect that introduces a race to get the current global parity. > >> But as you say, this might just be one case of assumptions and there >> might be more lurking around. So let's go with your approach for now. >> >> Roman > > OK. So does that count as a "review"? And I still need a second. No, not yet. I've tested it against my failing stress test (together with your njt_lock patch) and it seems to fix it. I'm also testing it against Shenandoah test suite, and so far looks good. I don't quite understand the resetting logic though. It first increments the global counter. If it turns 0, it resets all thread tokens to 0, using the claim_oops_do/cmpxchg protocol, but that is now expecting 0 too? Why not simply assign 0 instead? Maybe that code just isn't obvious enough...? Also, the resetting logic warrants a test, it is so rare, it will otherwise not get executed, but if it fails, it would be catastrophic. Roman From david.holmes at oracle.com Tue Mar 26 12:01:04 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 26 Mar 2019 22:01:04 +1000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> Message-ID: Hi Matthias, You don't respond on the cpuid logic query. Thanks, David On 26/03/2019 6:32 pm, Baesken, Matthias wrote: > Hi David, > >> This is looking better though I'm struggling to keep all the dots >> connected in terms of the APIs - I can't quite see the pattern for what >> calls what and what overrides what. > > I just followed here what is done at other places of VM_Version / Abstract_VM_Version , see > for example : > > src/hotspot/share/runtime/vm_version.hpp > > 82 // ..... Platforms that > 83 // need to specialize this define VM_Version::early_initialize(). > 84 static void early_initialize() { } > 85 > 86 // Called to initialize VM variables needing initialization > 87 // after command line parsing. Platforms that need to specialize > 88 // this should define VM_Version::init_before_ergo(). > 89 static void init_before_ergo() {} > > >> 2. os_windows.cpp >> >> Why the indirection through os::win32::print_virtualization_info instead >> of just calling VM_Version::print_detected_virtualization directly? > > Yes that's a good point - I think I should change this . > > Best regards, Matthias > > > > >> -----Original Message----- >> From: David Holmes >> Sent: Mittwoch, 20. M?rz 2019 07:15 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Cc: Doerr, Martin >> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >> hs_error file on linux/windows x86_64 >> >> Hi Matthias, >> >> On 19/03/2019 11:53 pm, Baesken, Matthias wrote: >>> Hello, here is a new webrev : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ >>> >>> This moved the ppc/s390 virtualization related info output into the >> platform/cpu - specific coding . >> >> This is looking better though I'm struggling to keep all the dots >> connected in terms of the APIs - I can't quite see the pattern for what >> calls what and what overrides what. A couple of initial comments: >> >> 1. We already have a ton of cpuid related parsing in >> ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check >> virtualizations" and store the information for later use? >> >> 2. os_windows.cpp >> >> Why the indirection through os::win32::print_virtualization_info instead >> of just calling VM_Version::print_detected_virtualization directly? >> >> I'll keep trying to piece this together. :) >> >> Thanks, >> David >> ----- >> >>> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Montag, 11. M?rz 2019 08:36 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Cc: Doerr, Martin >>>> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >>>> hs_error file on linux/windows x86_64 >>>> >>>> On 7/03/2019 3:13 am, Baesken, Matthias wrote: >>>>> Hello , could I push the latest version of 8219241 ? >>>> >>>> Sorry based on your earlier email I thought you were looking at doing >>>> the restructuring. I was then on vacation most of last week. >>>> >>>>> A change that moves the coding more into platform/cpu - specific >> coding >>>> (if this is wanted) >>>>> could be done when bringing in the AIX virtualization related info in >>>> another patch which I plan to do . >>>> >>>> If that is imminent then okay - I don't like the code as it is. >>>> >>>> Thanks, >>>> David >>>> >>> From coleen.phillimore at oracle.com Tue Mar 26 12:35:25 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 26 Mar 2019 08:35:25 -0400 Subject: RFR (S) 8160539: Stack frame scanning acquires several mutexes In-Reply-To: References: <6f422d4c-bab7-500d-bba2-f3d59ba39593@oracle.com> Message-ID: <3475f45e-f823-9d1b-bc99-96f3e12c8667@oracle.com> On 3/25/19 8:34 PM, David Holmes wrote: > Hi Coleen, > > On 26/03/2019 3:49 am, coleen.phillimore at oracle.com wrote: >> Summary: Use CAS instead of OopMapCacheAlloc_lock >> >> Tested with hs-tier1-5 and specjbb2017. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8160539.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8160539 > > How expensive is it to create OopMapCache? Are we likely to encounter > a lot of contention on this? > > As I mentioned in my reply to Dean I think it mildly preferable to use > cmpxchg if you want to use the value in case you lose the race: > > OopMapCache * temp = Atomic::cmpxchg(oop_map_cache, &_oop_map_cache, > NULL) > if (temp != NULL) { > ? // Some other thread won. > ? delete oop_map_cache; > ? oop_map_cache = temp; > } I was curious if this was the case, or if I needed a load_acquire. That said, this is not expensive at all, and probably not worth doing.? I thought that the only benefit would be to remove one of our mutexes, particularly since this one can be used by both java threads and non-java threads. I'm going to withdraw this and make the original bug to work on the DerivedPointerTableGC_lock instead. thanks, Coleen > > Thanks, > David > >> Thanks, >> Coleen From matthias.baesken at sap.com Tue Mar 26 12:31:29 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 26 Mar 2019 12:31:29 +0000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> Message-ID: >> 1. We already have a ton of cpuid related parsing in >> ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check >> virtualizations" and store the information for later use? Hi David , probably you mean this one with "cpuid logic query " . Probably I could move some coding into ./cpu/x86/vm_version_x86.hpp but I did not like the idea to put more and more coding into the header file . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Dienstag, 26. M?rz 2019 13:01 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Cc: Doerr, Martin > Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > hs_error file on linux/windows x86_64 > > Hi Matthias, > > You don't respond on the cpuid logic query. > > Thanks, > David > > On 26/03/2019 6:32 pm, Baesken, Matthias wrote: > > Hi David, > > > >> This is looking better though I'm struggling to keep all the dots > >> connected in terms of the APIs - I can't quite see the pattern for what > >> calls what and what overrides what. > > > > I just followed here what is done at other places of VM_Version / > Abstract_VM_Version , see > > for example : > > > > src/hotspot/share/runtime/vm_version.hpp > > > > 82 // ..... Platforms that > > 83 // need to specialize this define VM_Version::early_initialize(). > > 84 static void early_initialize() { } > > 85 > > 86 // Called to initialize VM variables needing initialization > > 87 // after command line parsing. Platforms that need to specialize > > 88 // this should define VM_Version::init_before_ergo(). > > 89 static void init_before_ergo() {} > > > > > >> 2. os_windows.cpp > >> > >> Why the indirection through os::win32::print_virtualization_info instead > >> of just calling VM_Version::print_detected_virtualization directly? > > > > Yes that's a good point - I think I should change this . > > > > Best regards, Matthias > > > > > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Mittwoch, 20. M?rz 2019 07:15 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Cc: Doerr, Martin > >> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > >> hs_error file on linux/windows x86_64 > >> > >> Hi Matthias, > >> > >> On 19/03/2019 11:53 pm, Baesken, Matthias wrote: > >>> Hello, here is a new webrev : > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ > >>> > >>> This moved the ppc/s390 virtualization related info output into the > >> platform/cpu - specific coding . > >> > >> This is looking better though I'm struggling to keep all the dots > >> connected in terms of the APIs - I can't quite see the pattern for what > >> calls what and what overrides what. A couple of initial comments: > >> > >> 1. We already have a ton of cpuid related parsing in > >> ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check > >> virtualizations" and store the information for later use? > >> > >> 2. os_windows.cpp > >> > >> Why the indirection through os::win32::print_virtualization_info instead > >> of just calling VM_Version::print_detected_virtualization directly? > >> > >> I'll keep trying to piece this together. :) > >> > >> Thanks, > >> David > >> ----- > >> > >>> > >>> Best regards, Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Montag, 11. M?rz 2019 08:36 > >>>> To: Baesken, Matthias ; 'hotspot- > >>>> dev at openjdk.java.net' > >>>> Cc: Doerr, Martin > >>>> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > >>>> hs_error file on linux/windows x86_64 > >>>> > >>>> On 7/03/2019 3:13 am, Baesken, Matthias wrote: > >>>>> Hello , could I push the latest version of 8219241 ? > >>>> > >>>> Sorry based on your earlier email I thought you were looking at doing > >>>> the restructuring. I was then on vacation most of last week. > >>>> > >>>>> A change that moves the coding more into platform/cpu - specific > >> coding > >>>> (if this is wanted) > >>>>> could be done when bringing in the AIX virtualization related info > in > >>>> another patch which I plan to do . > >>>> > >>>> If that is imminent then okay - I don't like the code as it is. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>> From robbin.ehn at oracle.com Tue Mar 26 12:51:12 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 26 Mar 2019 13:51:12 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> Message-ID: <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> Hi all, please review. This is v4, which is just an rebase on-top of "RFR: 8221207: Redo JDK-8218446 - SuspendAtExit hangs". The commit simplified S/R code a bit, making this patch simpler. Full: http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ Inc (not complete since v3 did not apply proper, one of the reject wasn't needed): http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ To summarize, before 8221207 we had 4 safe states: *Off threads_list (_thread_new) *Blocked (_thread_blocked) *In native (_thread_in_native) *Suspend (*ANY*) *After handshake (X) After 8221207 we have: *Off threads_list (_thread_new) (far from cross-mod code) *Blocked (_thread_blocked) *In native (_thread_in_native) *After handshake (X) -We have one instruction barrier for when going from _thread_new. -We have four instruction barrier for when going from blocked: - ThreadBlockInVM - ThreadBlockInVMWithDeadlockCheck - The manual blocked transition for suspended blocked case. - Blocked in SS:block() uses the barrier in block_if_requested_slow. -We let threads doing their own handshake hit the barrier in block_if_requested_slow. -We let threads in native stay armed so they hit the barrier in block_if_requested_slow. Not disarming threads in native works because, false positives already happens today and are handled. Both handshakes and safepoints disarms in correct order, so a JavaThread can notice that is should be armed, but with false positives. Before first check of handshake/safepoint the JavaThread is in a unsafe state. After that check a new handshake/safepoint cannot start/progress since the JavaThread is unsafe. Thus re-arming and continue to next polling site is okay. Patch only adds barrier for cross-modifying code that happens during a handshake/safepoint and implements this barrier for those platforms which have a simple implementaion for accomplish it. sparc/arm32/aarch64 needs additional work. The deprecated global poll path does not contain proper barriers, and will be obsoleted in JDK 14. Thanks, Robbin On 2019-03-08 16:24, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state it can start > executing updated code. E.g. an oop in the instruction stream can have been > updated. > > Most hardware's require a barrier or that the code cross modified is far away to > guarantee that the thread executing the updated instruction stream sees the > modification. > > What far away is and how far an update instruction stream is from a safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe > state closer to a nmethod can increase changes of a crash. > > A few benchmarks will see a performance regression with the extra serializing, > such as Octane-Crypto -5% (x86). > > With a much more elaborate fix, such as changing the local poll to have more > than two states (armed/disarmed), it would be possible to only emit such > instruction stream barrier when the poll have been armed while the thread was > safepoint safe. > > Passes t1-3 on our platforms, and test built what I can. > > Thanks, Robbin From martin.doerr at sap.com Tue Mar 26 13:45:22 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 26 Mar 2019 13:45:22 +0000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> Message-ID: Hi Robbin, the rebased version looks good to me. I'm only missing barrier implementations for aarch64 and SPARC. I'd expect isb on aarch64 and at least a compiler barrier on SPARC. But I'm not an expert for these platforms. Best regards, Martin -----Original Message----- From: Robbin Ehn Sent: Dienstag, 26. M?rz 2019 13:51 To: hotspot-dev at openjdk.java.net; Doerr, Martin ; David Holmes ; Erik Osterlund ; Reingruber, Richard ; Andrew Haley Subject: Re: RFR(m): 8220351: Cross-modifying code Hi all, please review. This is v4, which is just an rebase on-top of "RFR: 8221207: Redo JDK-8218446 - SuspendAtExit hangs". The commit simplified S/R code a bit, making this patch simpler. Full: http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ Inc (not complete since v3 did not apply proper, one of the reject wasn't needed): http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ To summarize, before 8221207 we had 4 safe states: *Off threads_list (_thread_new) *Blocked (_thread_blocked) *In native (_thread_in_native) *Suspend (*ANY*) *After handshake (X) After 8221207 we have: *Off threads_list (_thread_new) (far from cross-mod code) *Blocked (_thread_blocked) *In native (_thread_in_native) *After handshake (X) -We have one instruction barrier for when going from _thread_new. -We have four instruction barrier for when going from blocked: - ThreadBlockInVM - ThreadBlockInVMWithDeadlockCheck - The manual blocked transition for suspended blocked case. - Blocked in SS:block() uses the barrier in block_if_requested_slow. -We let threads doing their own handshake hit the barrier in block_if_requested_slow. -We let threads in native stay armed so they hit the barrier in block_if_requested_slow. Not disarming threads in native works because, false positives already happens today and are handled. Both handshakes and safepoints disarms in correct order, so a JavaThread can notice that is should be armed, but with false positives. Before first check of handshake/safepoint the JavaThread is in a unsafe state. After that check a new handshake/safepoint cannot start/progress since the JavaThread is unsafe. Thus re-arming and continue to next polling site is okay. Patch only adds barrier for cross-modifying code that happens during a handshake/safepoint and implements this barrier for those platforms which have a simple implementaion for accomplish it. sparc/arm32/aarch64 needs additional work. The deprecated global poll path does not contain proper barriers, and will be obsoleted in JDK 14. Thanks, Robbin On 2019-03-08 16:24, Robbin Ehn wrote: > Hi all, please review. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220351 > Changeset: > http://cr.openjdk.java.net/~rehn/8220351/webrev/ > > After a JavaThread have been in a safepoint/(handshake) safe state it can start > executing updated code. E.g. an oop in the instruction stream can have been > updated. > > Most hardware's require a barrier or that the code cross modified is far away to > guarantee that the thread executing the updated instruction stream sees the > modification. > > What far away is and how far an update instruction stream is from a safepoint > safe state is not clear. > > To be compliant with those architectures an instruction stream barrier must be > added when leaving the safepoint safe state. > > There may be crashes today due to this missing barrier. > A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe > state closer to a nmethod can increase changes of a crash. > > A few benchmarks will see a performance regression with the extra serializing, > such as Octane-Crypto -5% (x86). > > With a much more elaborate fix, such as changing the local poll to have more > than two states (armed/disarmed), it would be possible to only emit such > instruction stream barrier when the poll have been armed while the thread was > safepoint safe. > > Passes t1-3 on our platforms, and test built what I can. > > Thanks, Robbin From matthias.baesken at sap.com Tue Mar 26 15:16:57 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 26 Mar 2019 15:16:57 +0000 Subject: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? Message-ID: Hello, it seems that the fopen calls in CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file do not have a corresponding fclose. Or do I miss something ? We could also add a "true" for the second argument of the fileStream constructor calls ( fileStream(FILE* file, bool need_close = false) ) to get the closing. coding : ------------ jdk/src/hotspot/share/compiler/compilerOracle.cpp 741void CompilerOracle::append_comment_to_file(const char* message) { 742 assert(has_command_file(), "command file must be specified"); 743 fileStream stream(fopen(cc_file(), "at")); 744 stream.print("# "); 745 for (int index = 0; message[index] != '\0'; index++) { 746 stream.put(message[index]); 747 if (message[index] == '\n') stream.print("# "); 748 } 749 stream.cr(); 750} 751 752void CompilerOracle::append_exclude_to_file(const methodHandle& method) { 753 assert(has_command_file(), "command file must be specified"); 754 fileStream stream(fopen(cc_file(), "at")); 755 stream.print("exclude "); 756 method->method_holder()->name()->print_symbol_on(&stream); 757 stream.print("."); 758 method->name()->print_symbol_on(&stream); 759 method->signature()->print_symbol_on(&stream); 760 stream.cr(); 761 stream.cr(); 762} Best regards, Matthias From gunter.haug at sap.com Tue Mar 26 15:30:59 2019 From: gunter.haug at sap.com (Haug, Gunter) Date: Tue, 26 Mar 2019 15:30:59 +0000 Subject: [RFR]: [S390] Fix JFR profiling Message-ID: Hi all, can I please have reviews and a sponsor for the following tiny fix which only affects one s390 platform file: https://bugs.openjdk.java.net/browse/JDK-8221479 http://cr.openjdk.java.net/~ghaug/webrevs/8221479/ It detects addresses which are totally off and would have been dereferenced otherwise. This caused very rare crashes on s390. Thanks, Gunter From martin.doerr at sap.com Tue Mar 26 15:45:12 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 26 Mar 2019 15:45:12 +0000 Subject: [RFR]: [S390] Fix JFR profiling In-Reply-To: References: Message-ID: Hi Gunter, looks good and trivial. I can push it tomorrow if there are no objections. Thanks for fixing. Martin -----Original Message----- From: hotspot-dev On Behalf Of Haug, Gunter Sent: Dienstag, 26. M?rz 2019 16:31 To: hotspot-dev at openjdk.java.net Subject: [RFR]: [S390] Fix JFR profiling Hi all, can I please have reviews and a sponsor for the following tiny fix which only affects one s390 platform file: https://bugs.openjdk.java.net/browse/JDK-8221479 http://cr.openjdk.java.net/~ghaug/webrevs/8221479/ It detects addresses which are totally off and would have been dereferenced otherwise. This caused very rare crashes on s390. Thanks, Gunter From volker.simonis at gmail.com Tue Mar 26 17:23:35 2019 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 26 Mar 2019 18:23:35 +0100 Subject: RFR(XXS): 8221483: TestOopCmp.java fails due to "Multiple garbage collectors selected" Message-ID: Hi, sorry, but I broke tier 3 suite with my test from "8221083 [ppc64] Wrong oop compare in C1-generated code". The fix is trivial: http://cr.openjdk.java.net/~simonis/webrevs/2019/8221483/ https://bugs.openjdk.java.net/browse/JDK-8221483 Because the test is only meaningful if run with SerialGC it uses "main/othervm" mode with "-XX:+UseSerialGC" as command line option. However this may break if JTreg is called with another, explicit GC setting in "vmoptions" (what the tier3 tests are apparently doing). Adding "@requires vm.gc.Serial" will only run the test if no explicit GC was set outside of the test or if the GC was explicitly set to "UseSerial". Thanks, Volker From vladimir.kozlov at oracle.com Tue Mar 26 17:42:36 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 26 Mar 2019 10:42:36 -0700 Subject: RFR(XXS): 8221483: TestOopCmp.java fails due to "Multiple garbage collectors selected" In-Reply-To: References: Message-ID: <8b42d2ea-eb53-ae89-3ac3-ef819d1ecec4@oracle.com> Looks good. Thanks, Vladimir On 3/26/19 10:23 AM, Volker Simonis wrote: > Hi, > > sorry, but I broke tier 3 suite with my test from "8221083 [ppc64] > Wrong oop compare in C1-generated code". The fix is trivial: > > http://cr.openjdk.java.net/~simonis/webrevs/2019/8221483/ > https://bugs.openjdk.java.net/browse/JDK-8221483 > > Because the test is only meaningful if run with SerialGC it uses > "main/othervm" mode with "-XX:+UseSerialGC" as command line option. > However this may break if JTreg is called with another, explicit GC > setting in "vmoptions" (what the tier3 tests are apparently doing). > > Adding "@requires vm.gc.Serial" will only run the test if no explicit > GC was set outside of the test or if the GC was explicitly set to > "UseSerial". > > Thanks, > Volker > From martin.doerr at sap.com Tue Mar 26 17:43:27 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 26 Mar 2019 17:43:27 +0000 Subject: RFR(XXS): 8221483: TestOopCmp.java fails due to "Multiple garbage collectors selected" In-Reply-To: References: Message-ID: Hi Volker, I had already thought about that while reviewing the original change. But then I thought it wouldn't be needed because Serial should be always available. I was not aware of the problem when explicitly setting another GC. Looks good. Thanks, Martin -----Original Message----- From: hotspot-dev On Behalf Of Volker Simonis Sent: Dienstag, 26. M?rz 2019 18:24 To: HotSpot Open Source Developers Cc: Vladimir Kozlov Subject: RFR(XXS): 8221483: TestOopCmp.java fails due to "Multiple garbage collectors selected" Hi, sorry, but I broke tier 3 suite with my test from "8221083 [ppc64] Wrong oop compare in C1-generated code". The fix is trivial: http://cr.openjdk.java.net/~simonis/webrevs/2019/8221483/ https://bugs.openjdk.java.net/browse/JDK-8221483 Because the test is only meaningful if run with SerialGC it uses "main/othervm" mode with "-XX:+UseSerialGC" as command line option. However this may break if JTreg is called with another, explicit GC setting in "vmoptions" (what the tier3 tests are apparently doing). Adding "@requires vm.gc.Serial" will only run the test if no explicit GC was set outside of the test or if the GC was explicitly set to "UseSerial". Thanks, Volker From volker.simonis at gmail.com Tue Mar 26 17:52:40 2019 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 26 Mar 2019 18:52:40 +0100 Subject: RFR(XXS): 8221483: TestOopCmp.java fails due to "Multiple garbage collectors selected" In-Reply-To: References: Message-ID: Thanks Vladimir and Martin. I've pushed the fix. On Tue, Mar 26, 2019 at 6:43 PM Doerr, Martin wrote: > > Hi Volker, > > I had already thought about that while reviewing the original change. But then I thought it wouldn't be needed because Serial should be always available. I was not aware of the problem when explicitly setting another GC. > > Looks good. > > Thanks, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of Volker Simonis > Sent: Dienstag, 26. M?rz 2019 18:24 > To: HotSpot Open Source Developers > Cc: Vladimir Kozlov > Subject: RFR(XXS): 8221483: TestOopCmp.java fails due to "Multiple garbage collectors selected" > > Hi, > > sorry, but I broke tier 3 suite with my test from "8221083 [ppc64] > Wrong oop compare in C1-generated code". The fix is trivial: > > http://cr.openjdk.java.net/~simonis/webrevs/2019/8221483/ > https://bugs.openjdk.java.net/browse/JDK-8221483 > > Because the test is only meaningful if run with SerialGC it uses > "main/othervm" mode with "-XX:+UseSerialGC" as command line option. > However this may break if JTreg is called with another, explicit GC > setting in "vmoptions" (what the tier3 tests are apparently doing). > > Adding "@requires vm.gc.Serial" will only run the test if no explicit > GC was set outside of the test or if the GC was explicitly set to > "UseSerial". > > Thanks, > Volker From aph at redhat.com Tue Mar 26 18:26:03 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 26 Mar 2019 18:26:03 +0000 Subject: 8219993: AArch64: Compiled CI stubs are unsafely modified In-Reply-To: <3abc4b04-a6d4-62f7-4cd9-36a89fe020e4@redhat.com> References: <3abc4b04-a6d4-62f7-4cd9-36a89fe020e4@redhat.com> Message-ID: <2fa5fcb0-3bcc-f490-4948-405e23c10e40@redhat.com> Here's a simpler patch. All I do this time is insert an ISB between the call and the load instructions so that we don't have a stale instruction buffer. I also took the opportunity to unify some code which was duplicated in C1 and compiledIC_aarch64. Andrew Dinn, please approve this one. :-) http://cr.openjdk.java.net/~aph/8219993.2/ -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From jcbeyler at google.com Tue Mar 26 19:37:49 2019 From: jcbeyler at google.com (Jean Christophe Beyler) Date: Tue, 26 Mar 2019 12:37:49 -0700 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <25a50bc3-222c-a915-5517-37a2f9375c42@oracle.com> References: <45341168-e7e0-90d1-449f-210500882b8f@oracle.com> <55283958-de3d-07f2-51e3-ad34c5046a96@oracle.com> <31613f88-5f7d-938d-e9f6-69cdaf857268@oracle.com> <839301b7-c247-df3b-e485-283e8bb7388b@oracle.com> <95fe277d-ba6e-4fec-77aa-d1f1051751aa@oracle.com> <72bf2f4a-5bf7-98de-5f00-68485072923d@oracle.com> <25a50bc3-222c-a915-5517-37a2f9375c42@oracle.com> Message-ID: Hi all, My apologies again for this one. This has been a bit tricky to get in and has fell off my priority list due to other issues/commitments but I'm back to finishing this up and the next webrevs regarding this work :) Problem is that we change years so the copyrights changed on some of these and there were a few problems with various architectures/build systems that made the testing fail on the submit repo. So I offer two webrevs: - The incremental from the last LGTM stamped one: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06_07/ - The full webrev that cleaned up a few problems for windows and solaris and now passes the submit repo: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.07/ And again, my sincere apologies that it took me SO long to get back to this but I had to work through the random submit repo failures and some of them took time for me to debug (thanks Serguei for your help :-)), Jc On Tue, Jan 22, 2019 at 6:03 PM Alex Menkov wrote: > +1 > > --alex > > On 01/22/2019 10:29, JC Beyler wrote: > > Thanks Paul! > > > > Anybody else for the review for version 6? > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > > > > > Thanks, > > Jc > > > > On Tue, Jan 22, 2019 at 6:10 AM Hohensee, Paul > > wrote: > > > > Lgtm :) > > > > Paul > > > > ?On 1/14/19, 7:46 AM, "hotspot-dev on behalf of JC Beyler" > > > on behalf of > > jcbeyler at google.com > wrote: > > > > Hi all, > > > > Friendly ping on this one, I know that it has been a long > > process with back > > and forths, to which I apologize... > > > > But is there any way I could get a final LGTM for version 6? > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > > > Thanks! > > Jc > > > > On Tue, Jan 8, 2019 at 10:05 AM JC Beyler > > wrote: > > > > > Happy new year all! > > > > > > Could I get a final LGTM for version 6? > > > > > > Webrev: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > > > > > Thanks! > > > Jc > > > > > > On Mon, Dec 17, 2018 at 8:43 AM JC Beyler > > > wrote: > > > > > >> Hi all, > > >> > > >> I don't believe I got actual LGTM for this version: > > >> > > >> > > >> Webrev: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ > > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > >> > > >> > > >> It removed the namespaces and uses explicit static instead :) > > >> > > >> Thanks! > > >> Jc > > >> > > >> On Wed, Dec 12, 2018 at 8:06 PM JC Beyler > > > wrote: > > >> > > >>> So did I Alexey but with David & Serguei preferring static, > > it seems > > >>> more reasonable to go down their route :-) > > >>> > > >>> So here is the latest webrev with static instead of an > > anonymous > > >>> namespace: > > >>> > > >>> Webrev: > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.06/ > > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>> > > >>> Let me know what you think, can I get a webrev 06 review? > > >>> > > >>> Thanks! > > >>> Jc > > >>> > > >>> On Wed, Dec 12, 2018 at 3:10 PM Alex Menkov > > > > > >>> wrote: > > >>> > > >>>> Hm.. > > >>>> I considered unnamed namespaces "C++ style" (and static > > globals as "C > > >>>> style"). > > >>>> Static globals were deprecated in C++ (but some time ago > the > > >>>> deprecation > > >>>> was reverted). > > >>>> > > >>>> --alex > > >>>> > > >>>> On 12/12/2018 13:55, serguei.spitsyn at oracle.com > > wrote: > > >>>> > Agreed. > > >>>> > > > >>>> > Thanks, > > >>>> > Serguei > > >>>> > > > >>>> > > > >>>> > On 12/12/18 13:52, David Holmes wrote: > > >>>> >> FWIW I think namespaces are overkill in all of this > > test code and > > >>>> just > > >>>> >> obfuscates things - the declaration is easily missed. A > > static > > >>>> >> variable in a .cpp is clearly a global variable to the > > file. > > >>>> >> > > >>>> >> Cheers, > > >>>> >> David > > >>>> >> > > >>>> >> > > >>>> >> > > >>>> >> On 13/12/2018 5:37 am, serguei.spitsyn at oracle.com > > wrote: > > >>>> >>> Hi Jc, > > >>>> >>> > > >>>> >>> > > >>>> >>> On 12/11/18 21:16, JC Beyler wrote: > > >>>> >>>> Hi all, > > >>>> >>>> > > >>>> >>>> Here is the new webrev with the TEST.groups change. > > Serguei, let > > >>>> me > > >>>> >>>> know if I convinced you with the static vs anonymous > > namespaces or > > >>>> >>>> if you'd still rather have a "static" for now :-) > > >>>> >>> > > >>>> >>> > > >>>> >>> What do you think about this post? : > > >>>> >>> > > >>>> > > > https://stackoverflow.com/questions/11623451/static-vs-non-static-variables-in-namespace > > >>>> >>> > > >>>> >>> > > >>>> >>>> > > >>>> >>>> Webrev: > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.05/ > > >>>> >>>> > > > > >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>> >>> > > >>>> >>> The update looks fine. > > >>>> >>> > > >>>> >>> Thanks, > > >>>> >>> Serguei > > >>>> >>> > > >>>> >>> > > >>>> >>> Thanks, > > >>>> >>> Serguei > > >>>> >>> > > >>>> >>>> > > >>>> >>>> Thanks again for the reviews! > > >>>> >>>> Jc > > >>>> >>>> > > >>>> >>>> On Mon, Dec 10, 2018 at 3:10 PM JC Beyler > > > > >>>> >>>> > >> wrote: > > >>>> >>>> > > >>>> >>>> Hi Serguei, > > >>>> >>>> > > >>>> >>>> Yes basically it is equivalent :) I can put them > > in but they > > >>>> are > > >>>> >>>> not required. The norm actually wanted to > > deprecate it but then > > >>>> >>>> remembered that C compatibility would require the > > static > > >>>> key-word > > >>>> >>>> for this case [1] > > >>>> >>>> > > >>>> >>>> So, really, they are not required here and will > > amount to the > > >>>> same > > >>>> >>>> thing: only that file can refer to them and you > > cannot get to > > >>>> them > > >>>> >>>> without a globally available method to return a > > pointer to them > > >>>> >>>> (ie same as a static variable in C). > > >>>> >>>> > > >>>> >>>> I can put static if it makes it easier to see > > but, by being in > > >>>> an > > >>>> >>>> anonymous namespace they are only available for > > the file's > > >>>> >>>> translation unit. For example: > > >>>> >>>> > > >>>> >>>> $ cat main.cpp > > >>>> >>>> > > >>>> >>>> int totally_global; > > >>>> >>>> static int explictly_static; > > >>>> >>>> > > >>>> >>>> namespace { > > >>>> >>>> int implicitly_static; > > >>>> >>>> } > > >>>> >>>> > > >>>> >>>> void foo(); > > >>>> >>>> int main() { > > >>>> >>>> foo(); > > >>>> >>>> } > > >>>> >>>> > > >>>> >>>> $ g++ -O3 main.cpp -c > > >>>> >>>> $ nm main.o > > >>>> >>>> U _GLOBAL_OFFSET_TABLE_ > > >>>> >>>> 0000000000000000 T main > > >>>> >>>> 0000000000000000 B totally_global > > >>>> >>>> U _Z3foov > > >>>> >>>> > > >>>> >>>> As you can see, the static and anonymous > > namespace variables > > >>>> are > > >>>> >>>> not in the file due to not being used. If you > > were to use them, > > >>>> >>>> you'd see them show up as something like: > > >>>> >>>> 0000000000000008 b _ZL17explicitly_static > > >>>> >>>> 0000000000000004 b > > _ZN12_GLOBAL__N_117implicitly_staticE > > >>>> >>>> > > >>>> >>>> Where again, it shows that it is mangling the > > names so that no > > >>>> >>>> external usage can happen without tinkering. > > >>>> >>>> > > >>>> >>>> Hopefully that helps :-), > > >>>> >>>> Jc > > >>>> >>>> > > >>>> >>>> [1] > > >>>> >>>> > > http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1012 > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> On Mon, Dec 10, 2018 at 2:04 PM > > serguei.spitsyn at oracle.com > > >>>> >>>> > > < > > >>>> serguei.spitsyn at oracle.com serguei.spitsyn at oracle.com> > > >>>> >>>> > >> wrote: > > >>>> >>>> > > >>>> >>>> Hi Jc, > > >>>> >>>> > > >>>> >>>> I had little experience with the C++ > namespaces. > > >>>> >>>> My understanding is that static in this > > context should mean > > >>>> >>>> internal linkage. > > >>>> >>>> > > >>>> >>>> Thanks, > > >>>> >>>> Serguei > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> On 12/10/18 13:57, JC Beyler wrote: > > >>>> >>>>> Hi Serguei, > > >>>> >>>>> > > >>>> >>>>> The variables and functions are in a > > anonymous namespace; > > >>>> my > > >>>> >>>>> understanding of C++ is that this is > > equivalent to > > >>>> putting it > > >>>> >>>>> as static.Hence, I didn't add them there. > > Does that make > > >>>> >>>>> sense? > > >>>> >>>>> > > >>>> >>>>> Thanks! > > >>>> >>>>> Jc > > >>>> >>>>> > > >>>> >>>>> On Mon, Dec 10, 2018 at 1:33 PM > > >>>> serguei.spitsyn at oracle.com serguei.spitsyn at oracle.com> > > >>>> >>>>> > > > > >>>> >>>>> > > > >>>> >>>>> > >> wrote: > > >>>> >>>>> > > >>>> >>>>> Hi Jc, > > >>>> >>>>> > > >>>> >>>>> It looks good in general. > > >>>> >>>>> One question though. > > >>>> >>>>> > > >>>> >>>>> > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a_04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > > >>>> >>>>> > > >>>> >>>>> > > >>>> >>>>> I wonder if the variables and functions > > have to be > > >>>> static. > > >>>> >>>>> > > >>>> >>>>> Thanks, > > >>>> >>>>> Serguei > > >>>> >>>>> > > >>>> >>>>> > > >>>> >>>>> On 12/5/18 11:36, JC Beyler wrote: > > >>>> >>>>>> Hi all, > > >>>> >>>>>> > > >>>> >>>>>> My apologies to having to come back for > > another > > >>>> review > > >>>> >>>>>> for this change: I ran into a snag when > > trying to > > >>>> pull > > >>>> >>>>>> the latest changes compared to the base > > I was working > > >>>> >>>>>> on. I basically forgot that there was > > an issue with > > >>>> >>>>>> snprintf and that I had solved it via > > JDK-8213622. > > >>>> >>>>>> > > >>>> >>>>>> Could I have a new review of this > webrev: > > >>>> >>>>>> Webrev: > > >>>> >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/ > > >>>> >>>>>> > > > > >>>> >>>>>> Bug: > > >>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>> >>>>>> Incremental from the port of webrev.03 > > that got > > >>>> LGTMs: > > >>>> >>>>>> > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04/ > > >>>> >>>>>> > > > > >>>> >>>>>> > > >>>> >>>>>> A few comments on this because it took > > me a while to > > >>>> get > > >>>> >>>>>> things in a state I thought was good: > > >>>> >>>>>> - I had to implement an itoa method, > > do we have > > >>>> >>>>>> something like that in the test base > > (remember that > > >>>> >>>>>> JDK-8213622 could not use sprintf due > > to being in the > > >>>> >>>>>> test code)? > > >>>> >>>>>> > > >>>> >>>>>> - The differences here compared to > > the one you all > > >>>> >>>>>> reviewed are: > > >>>> >>>>>> - I found that adding to the > > strlen/memcpy > > >>>> error > > >>>> >>>>>> prone and thought that I would try to > > make it less > > >>>> so. > > >>>> >>>>>> If you want to compare, I extended the > > strlen/memcpy > > >>>> >>>>>> with the new format to show you if you > > prefer [1] > > >>>> >>>>>> - Note that the diff > > between the "old > > >>>> >>>>>> extended way from [1]" to the webrev.04 > > can be found > > >>>> >>>>>> in [2] > > >>>> >>>>>> > > >>>> >>>>>> - I added a test to test the > > exception wrapper > > >>>> in > > >>>> >>>>>> tests :); I'm not sure it is deemed > > useful or not but > > >>>> >>>>>> helped me assure myself that I was not > > doing things > > >>>> >>>>>> wrong; you can find the base test file > > here [3]; > > >>>> should > > >>>> >>>>>> we have this or not? (I know that > > normally we don't > > >>>> add > > >>>> >>>>>> tests to vmTestbase but thought this > > might be an > > >>>> >>>>>> exception) > > >>>> >>>>>> > > >>>> >>>>>> Thanks for your help and my apologies > > for the snag, > > >>>> >>>>>> Jc > > >>>> >>>>>> > > >>>> >>>>>> [1]: > > >>>> >>>>>> > > >>>> > > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > > >>>> >>>>>> > > >>>> >>>>>> < > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.03a/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > > > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> [2]: > > >>>> >>>>>> > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03a_04 > > >>>> >>>>>> > > > > >>>> >>>>>> [3] > > >>>> >>>>>> > > >>>> > > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > > >>>> >>>>>> > > >>>> >>>>>> < > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.04/test/hotspot/jtreg/vmTestbase/nsk/share/ExceptionCheckingJniEnv/exceptionjni001/exceptionjni001.cpp.html > > > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > >>>> >>>>>> On Mon, Dec 3, 2018 at 11:29 PM David > > Holmes > > >>>> >>>>>> > > > >>>> >>>>>> > >> wrote: > > >>>> >>>>>> > > >>>> >>>>>> Looks fine to me. > > >>>> >>>>>> > > >>>> >>>>>> Thanks, > > >>>> >>>>>> David > > >>>> >>>>>> > > >>>> >>>>>> On 4/12/2018 4:04 pm, JC Beyler > wrote: > > >>>> >>>>>> > Hi both, > > >>>> >>>>>> > > > >>>> >>>>>> > Thanks for the reviews! Since > > Serguei did not > > >>>> >>>>>> insist on get_basename, I > > >>>> >>>>>> > went for get_dirname since the > > method is a > > >>>> local > > >>>> >>>>>> static method and won't > > >>>> >>>>>> > have its name start spreading, I > > think it's ok > > >>>> too. > > >>>> >>>>>> > > > >>>> >>>>>> > For the naming of the local > > variable, the idea > > >>>> >>>>>> initially was to use the > > >>>> >>>>>> > same name as the local variable > > for JNIEnv > > >>>> already > > >>>> >>>>>> used to reduce the > > >>>> >>>>>> > code change. Since I'm now adding > > the line > > >>>> macro > > >>>> >>>>>> at the end anyway, this > > >>>> >>>>>> > does not matter anymore so I > > converged all > > >>>> local > > >>>> >>>>>> variables to "jni". > > >>>> >>>>>> > > > >>>> >>>>>> > So, without further ado, here is > > the new > > >>>> version: > > >>>> >>>>>> > Webrev: > > >>>> >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.03/ > > >>>> >>>>>> > > > > >>>> >>>>>> > Bug: > > >>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>> >>>>>> > > > >>>> >>>>>> > This passes the various tests > > changed by the > > >>>> >>>>>> webrev on my dev machine. > > >>>> >>>>>> > > > >>>> >>>>>> > Let me know what you think, > > >>>> >>>>>> > Jc > > >>>> >>>>>> > > > >>>> >>>>>> > On Mon, Dec 3, 2018 at 8:40 PM > > >>>> >>>>>> serguei.spitsyn at oracle.com > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > >> > > >>>> >>>>>> > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > >>> wrote: > > >>>> >>>>>> > > > >>>> >>>>>> > On 12/3/18 20:15, Chris > > Plummer wrote: > > >>>> >>>>>> > > Hi JC, > > >>>> >>>>>> > > > > >>>> >>>>>> > > Overall it looks good. A > > few naming nits > > >>>> >>>>>> thought: > > >>>> >>>>>> > > > > >>>> >>>>>> > > In bi01t001.cpp, why have > > you declared > > >>>> the > > >>>> >>>>>> > ExceptionCheckingJniEnvPtr > > >>>> >>>>>> > > using jni_env(jni). > > Elsewhere you use > > >>>> >>>>>> jni(jni_env) and rename the > > >>>> >>>>>> > > method argument passed in > > from jni to > > >>>> >>>>>> jni_env. > > >>>> >>>>>> > > > > >>>> >>>>>> > > Related to this, I also > > noticed in some > > >>>> >>>>>> files that already are using > > >>>> >>>>>> > > > > ExceptionCheckingJniEnvPtr, such as > > >>>> >>>>>> CharArrayCriticalLocker.cpp, you > > >>>> >>>>>> > > delcared it as > > env(jni_env). So that > > >>>> means > > >>>> >>>>>> there are 3 different > > >>>> >>>>>> > names > > >>>> >>>>>> > > you have used for the > > >>>> >>>>>> ExceptionCheckingJniEnvPtr local > > variable. > > >>>> >>>>>> > They > > >>>> >>>>>> > > should be consistent. > > >>>> >>>>>> > > > > >>>> >>>>>> > > Also, can you rename > > get_basename() to > > >>>> >>>>>> get_dirname()? I know Serguei > > >>>> >>>>>> > > suggested get_basename() a > > while back, > > >>>> but > > >>>> >>>>>> unless "basename" is > > >>>> >>>>>> > > commonly used for this > > purpose, I think > > >>>> >>>>>> "dirname" is more self > > >>>> >>>>>> > > explanatory. > > >>>> >>>>>> > > > >>>> >>>>>> > In general, I'm Okay with > > get_dirname(). > > >>>> >>>>>> > Just to mention dirname can > > be both short > > >>>> or > > >>>> >>>>>> full, so it is a little > > >>>> >>>>>> > confusing as well. > > >>>> >>>>>> > It is the reason why the > > get_basename() was > > >>>> >>>>>> suggested. > > >>>> >>>>>> > However, I do not insist on > > get_basename() > > >>>> nor > > >>>> >>>>>> get_full_dirname(). :) > > >>>> >>>>>> > > > >>>> >>>>>> > Thanks, > > >>>> >>>>>> > Serguei > > >>>> >>>>>> > > > >>>> >>>>>> > > > >>>> >>>>>> > > thanks, > > >>>> >>>>>> > > > > >>>> >>>>>> > > Chris > > >>>> >>>>>> > > > > >>>> >>>>>> > > On 12/2/18 10:29 PM, David > > Holmes wrote: > > >>>> >>>>>> > >> Hi Jc, > > >>>> >>>>>> > >> > > >>>> >>>>>> > >> I've been lurking on this > > one and have > > >>>> had > > >>>> >>>>>> a look through. I'm okay > > >>>> >>>>>> > >> with the FatalError > > approach for the > > >>>> tests > > >>>> >>>>>> - we don't expect > > >>>> >>>>>> > anything > > >>>> >>>>>> > >> to go wrong in a well > > written test in a > > >>>> >>>>>> correctly functioning VM. > > >>>> >>>>>> > >> > > >>>> >>>>>> > >> Thanks, > > >>>> >>>>>> > >> David > > >>>> >>>>>> > >> > > >>>> >>>>>> > >> > > >>>> >>>>>> > >> > > >>>> >>>>>> > >> On 3/12/2018 3:24 pm, JC > > Beyler wrote: > > >>>> >>>>>> > >>> Hi all, > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Would someone on the GC > > or runtime > > >>>> team > > >>>> >>>>>> be motivated to give > > >>>> >>>>>> > this a > > >>>> >>>>>> > >>> review? :) > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> It would be much > > appreciated! > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Webrev: > > >>>> >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > > >>>> >>>>>> > > > > >>>> >>>>>> > >>> Bug: > > >>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Thanks for your help, > > >>>> >>>>>> > >>> Jc > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> On Tue, Nov 27, 2018 at > > 4:36 PM JC > > >>>> Beyler > > >>>> >>>>>> > > > > >>>> > > > >>>> >>>>>> > > > > >>>> >>>>>> > >> > > >>>> >>>>>> > >>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > >>>> >>>>>> > >>>> wrote: > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Hi Chris, > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Yes I was waiting > > for another > > >>>> review > > >>>> >>>>>> since you had explicitly > > >>>> >>>>>> > >>> asked :) > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> And sounds good that > > when someone > > >>>> >>>>>> from GC or runtime gives a > > >>>> >>>>>> > >>> review, > > >>>> >>>>>> > >>> I'll wait for your > > full review on > > >>>> the > > >>>> >>>>>> webrev.02! > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Thanks again for > > your help, > > >>>> >>>>>> > >>> Jc > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> On Tue, Nov 27, 2018 > > at 12:48 PM > > >>>> >>>>>> Chris Plummer > > >>>> >>>>>> > >>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > >>>> >>>>>> > >> > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > >>>> >>>>>> > >>>> > > >>>> >>>>>> > wrote: > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Hi JC, > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> I think it would > > be good to > > >>>> get a > > >>>> >>>>>> review from the gc or > > >>>> >>>>>> > runtime > > >>>> >>>>>> > >>> teams, since > > this also affects > > >>>> >>>>>> their tests. > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Also, once we > > are settled on > > >>>> this > > >>>> >>>>>> FatalError approach, > > >>>> >>>>>> > I still > > >>>> >>>>>> > >>> need to give > > your webrev-02 a > > >>>> >>>>>> full review. I only > > >>>> >>>>>> > skimmed over > > >>>> >>>>>> > >>> parts of it (I > > did look at all > > >>>> >>>>>> the changes in webrevo-01). > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> thanks, > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Chris > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> On 11/27/18 8:58 > AM, > > >>>> >>>>>> serguei.spitsyn at oracle.com > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > >> > > >>>> >>>>>> > >>> > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > > > > >>>> >>>>>> > > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > >>> wrote: > > >>>> >>>>>> > >>>> Hi Jc, > > >>>> >>>>>> > >>>> > > >>>> >>>>>> > >>>> I've already > > reviewed this > > >>>> too. > > >>>> >>>>>> > >>>> > > >>>> >>>>>> > >>>> Thanks, > > >>>> >>>>>> > >>>> Serguei > > >>>> >>>>>> > >>>> > > >>>> >>>>>> > >>>> > > >>>> >>>>>> > >>>> On 11/27/18 > > 06:56, JC Beyler > > >>>> >>>>>> wrote: > > >>>> >>>>>> > >>>>> Thanks Chris, > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> Anybody else motivated > > to look at > > >>>> this > > >>>> >>>>>> and review it? :) > > >>>> >>>>>> > >>>>> Jc > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> On Mon, Nov > > 26, 2018 at > > >>>> 1:26 PM > > >>>> >>>>>> Chris Plummer > > >>>> >>>>>> > >>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >> > > >>>> >>>>>> > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>> > > >>>> >>>>>> > >>>>> wrote: > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> Hi JC, > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> I'm ok with the > > FatalError approach, > > >>>> >>>>>> but would > > >>>> >>>>>> > like to > > >>>> >>>>>> > >>>>> hear opinions from > > others also. > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> thanks, > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> Chris > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> On 11/21/18 8:19 AM, > > JC Beyler > > >>>> wrote: > > >>>> >>>>>> > >>>>>> Hi Chris, > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> Thanks > > for taking the > > >>>> time > > >>>> >>>>>> to look at it and yes you > > >>>> >>>>>> > >>>>>> have > > raised exactly why > > >>>> >>>>>> the webrev is between two > > >>>> >>>>>> > >>>>>> worlds: > > in cases where > > >>>> a > > >>>> >>>>>> fatal error on failure is > > >>>> >>>>>> > >>>>>> wanted, > > should we > > >>>> simplify > > >>>> >>>>>> the code to remove > > >>>> >>>>>> > the return > > >>>> >>>>>> > >>>>>> tests > > since we do them > > >>>> >>>>>> internally? Now that I've > > >>>> >>>>>> > looked > > >>>> >>>>>> > >>>>>> around > > for non-fatal > > >>>> >>>>>> cases, I think the answer > > >>>> >>>>>> > is yes, > > >>>> >>>>>> > >>>>>> it > > simplifies the code > > >>>> >>>>>> while maintaining the checks. > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> I looked > > a bit and it > > >>>> >>>>>> seems that I can't find > > >>>> >>>>>> > easily a > > >>>> >>>>>> > >>>>>> case > > where the test > > >>>> >>>>>> accepts a JNI failure to > > >>>> >>>>>> > then move > > >>>> >>>>>> > >>>>>> on. > > Therefore, perhaps, > > >>>> >>>>>> for now, the fail with a > > >>>> >>>>>> > Fatal > > >>>> >>>>>> > >>>>>> is enough > > and we can > > >>>> work > > >>>> >>>>>> on the tests to clean > > >>>> >>>>>> > them up? > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> That > > means that this is > > >>>> >>>>>> the new webrev with only > > >>>> >>>>>> > Fatal > > >>>> >>>>>> > >>>>>> and > > cleans up the > > >>>> tests so > > >>>> >>>>>> that it is no longer in > > >>>> >>>>>> > >>>>>> between > > two worlds: > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> Webrev: > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>>>> Bug: > > >>>> >>>>>> > > > >>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> (This > > passes testing > > >>>> on my > > >>>> >>>>>> dev machine for all the > > >>>> >>>>>> > >>>>>> modified > > tests) > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> with the > > example you > > >>>> >>>>>> provided, it now looks like: > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > >>>> >>>>>> > > >>>> >>>>>> < > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> < > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> Where it > > does, to me at > > >>>> >>>>>> least, seem cleaner and less > > >>>> >>>>>> > >>>>>> "noisy". > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> Let me > > know what you > > >>>> think, > > >>>> >>>>>> > >>>>>> Jc > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> On Tue, > > Nov 20, 2018 at > > >>>> >>>>>> 9:33 PM Chris Plummer > > >>>> >>>>>> > >>>>>> < > > >>>> chris.plummer at oracle.com > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >> > > >>>> >>>>>> > >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>> wrote: > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> Hi JC, > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> Sorry > > about the > > >>>> delay. > > >>>> >>>>>> I had to go back an > > >>>> >>>>>> > look at > > >>>> >>>>>> > >>>>>> the > > initial 8210842 > > >>>> >>>>>> webrev and RFR thread to see > > >>>> >>>>>> > >>>>>> what > > this was > > >>>> >>>>>> initially all about. > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> In > > general the > > >>>> changes > > >>>> >>>>>> look good. > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> I > > don't have a good > > >>>> >>>>>> answer to your > > >>>> >>>>>> > >>>>>> FatalError/NonFatalError > > question. It > > >>>> makes > > >>>> >>>>>> > the code > > >>>> >>>>>> > >>>>>> a lot > > cleaner to > > >>>> use > > >>>> >>>>>> FatalError, but then it > > >>>> >>>>>> > is a > > >>>> >>>>>> > >>>>>> > > behavior change, > > >>>> and > > >>>> >>>>>> you also need to deal with > > >>>> >>>>>> > >>>>>> tests > > that > > >>>> >>>>>> intentionally induce errors (do > > >>>> >>>>>> > you have > > >>>> >>>>>> > >>>>>> an > > example of > > >>>> that). > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> In > > any case, right > > >>>> now > > >>>> >>>>>> your webrev seems to be > > >>>> >>>>>> > >>>>>> > > between two worlds. > > >>>> >>>>>> You are producing > > >>>> >>>>>> > FatalError, > > >>>> >>>>>> > >>>>>> but > > still checking > > >>>> >>>>>> results. Here's a good > > >>>> >>>>>> > example: > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > >>>> >>>>>> > > >>>> >>>>>> < > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> < > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> I'm > > not sure if > > >>>> this > > >>>> >>>>>> is just a temporary > > >>>> >>>>>> > state until > > >>>> >>>>>> > >>>>>> it > > was decided > > >>>> which > > >>>> >>>>>> approach to take. > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > thanks, > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> Chris > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> On > > 11/20/18 2:14 > > >>>> PM, > > >>>> >>>>>> JC Beyler wrote: > > >>>> >>>>>> > >>>>>>> Hi > all, > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > Chris thought it > > >>>> made > > >>>> >>>>>> sense to have more > > >>>> >>>>>> > eyes on > > >>>> >>>>>> > >>>>>>> this > > change than > > >>>> just > > >>>> >>>>>> serviceability as it will > > >>>> >>>>>> > >>>>>>> > > modify to tests > > >>>> that > > >>>> >>>>>> are not only > > >>>> >>>>>> > serviceability > > >>>> >>>>>> > >>>>>>> > > tests so I've > > >>>> moved > > >>>> >>>>>> this to conversation > > >>>> >>>>>> > here :) > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> For > > convenience, > > >>>> I've > > >>>> >>>>>> copy-pasted the > > >>>> >>>>>> > initial RFR: > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > Could I have a > > >>>> review > > >>>> >>>>>> for the extension and > > >>>> >>>>>> > usage > > >>>> >>>>>> > >>>>>>> of > the > > >>>> >>>>>> ExceptionJniWrapper. This adds > > lines and > > >>>> >>>>>> > >>>>>>> > > filenames to the > > >>>> end > > >>>> >>>>>> of the wrapper JNI > > >>>> >>>>>> > methods, > > >>>> >>>>>> > >>>>>>> adds > > tracing, and > > >>>> >>>>>> throws an error if need > > >>>> >>>>>> > be. I've > > >>>> >>>>>> > >>>>>>> > > ported the gc/lock > > >>>> >>>>>> files to use the new > > >>>> >>>>>> > >>>>>>> > > TRACE_JNI_CALL > > >>>> add-on > > >>>> >>>>>> and I've ported a few > > >>>> >>>>>> > of the > > >>>> >>>>>> > >>>>>>> > > tests that were > > >>>> >>>>>> already changed for the > > >>>> >>>>>> > assignment > > >>>> >>>>>> > >>>>>>> > > webrev for > > >>>> >>>>>> JDK-8212884. > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > Webrev: > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>>>>> Bug: > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> For > > illustration, > > >>>> if > > >>>> >>>>>> I force an error to the > > >>>> >>>>>> > >>>>>>> > > AP04/ap04t03 test > > >>>> and > > >>>> >>>>>> set the verbosity on, > > >>>> >>>>>> > I get > > >>>> >>>>>> > >>>>>>> > > something like: > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> >> > > Calling JNI > > >>>> method > > >>>> >>>>>> FindClass from > > >>>> >>>>>> > >>>>>>> ap04t003.cpp:343 > > >>>> >>>>>> > >>>>>>> >> > > Calling with > > >>>> these > > >>>> >>>>>> parameter(s): > > >>>> >>>>>> > >>>>>>> java/lang/Threadd > > >>>> >>>>>> > >>>>>>> Wait > > for thread > > >>>> to > > >>>> >>>>>> finish > > >>>> >>>>>> > >>>>>>> << > > Called JNI > > >>>> method > > >>>> >>>>>> FindClass from > > >>>> >>>>>> > >>>>>>> ap04t003.cpp:343 > > >>>> >>>>>> > >>>>>>> > > Exception in > > >>>> thread > > >>>> >>>>>> "Thread-0" > > >>>> >>>>>> > >>>>>>> > > java.lang.NoClassDefFoundError: > > >>>> >>>>>> > java/lang/Threadd > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>> >>>>>> > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > Method) > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > Caused by: > > >>>> >>>>>> java.lang.ClassNotFoundException: > > >>>> >>>>>> > >>>>>>> > > java.lang.Threadd > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > > >>>> >>>>>> > >>>>>>> > > ... 3 more > > >>>> >>>>>> > >>>>>>> > > FATAL ERROR in > > >>>> native > > >>>> >>>>>> method: JNI method > > >>>> >>>>>> > FindClass > > >>>> >>>>>> > >>>>>>> : > > internal error > > >>>> from > > >>>> >>>>>> ap04t003.cpp:343 > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>> >>>>>> > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > Method) > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > at > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > >>>> Questions/comments I > > >>>> >>>>>> have about this are: > > >>>> >>>>>> > >>>>>>> - > > Do we want to > > >>>> >>>>>> force fatal errors when a JNI > > >>>> >>>>>> > >>>>>>> call > > fails in > > >>>> >>>>>> general? Most of these tests > > >>>> >>>>>> > do the > > >>>> >>>>>> > >>>>>>> > > right thing and > > >>>> test > > >>>> >>>>>> the return of the JNI > > >>>> >>>>>> > calls, > > >>>> >>>>>> > >>>>>>> for > > example: > > >>>> >>>>>> > >>>>>>> > > thrClass = > > >>>> >>>>>> > > jni->FindClass("java/lang/Threadd", > > >>>> >>>>>> > >>>>>>> > > TRACE_JNI_CALL); > > >>>> >>>>>> > >>>>>>> > > if (thrClass > > >>>> == > > >>>> >>>>>> NULL) { > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> but > > now the > > >>>> wrapper > > >>>> >>>>>> actually would do a > > >>>> >>>>>> > fatal if > > >>>> >>>>>> > >>>>>>> the > > FindClass call > > >>>> >>>>>> would return a nullptr, > > >>>> >>>>>> > so we > > >>>> >>>>>> > >>>>>>> > > could remove that > > >>>> >>>>>> test altogether. What do you > > >>>> >>>>>> > >>>>>>> think? > > >>>> >>>>>> > >>>>>>> > > - I prefer to > > >>>> >>>>>> leave them as the tests then > > >>>> >>>>>> > >>>>>>> > > become closer to > > >>>> what > > >>>> >>>>>> real users would have in > > >>>> >>>>>> > >>>>>>> > > their code and is > > >>>> the > > >>>> >>>>>> "recommended" way of > > >>>> >>>>>> > doing it > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> - > The > > >>>> alternative > > >>>> >>>>>> is to use the > > >>>> >>>>>> > NonFatalError I > > >>>> >>>>>> > >>>>>>> > > added which then > > >>>> just > > >>>> >>>>>> prints out that something > > >>>> >>>>>> > >>>>>>> went > > wrong, > > >>>> letting > > >>>> >>>>>> the test continue. Question > > >>>> >>>>>> > >>>>>>> will > > be what > > >>>> should > > >>>> >>>>>> be the default? The > > >>>> >>>>>> > fatal or > > >>>> >>>>>> > >>>>>>> the > > non-fatal > > >>>> error > > >>>> >>>>>> handling? > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> On a > > different > > >>>> >>>>>> subject: > > >>>> >>>>>> > >>>>>>> - > > On the new > > >>>> tests, > > >>>> >>>>>> I've removed the > > >>>> >>>>>> > >>>>>>> > > NSK_JNI_VERIFY > > >>>> since > > >>>> >>>>>> the JNI wrapper > > >>>> >>>>>> > handles the > > >>>> >>>>>> > >>>>>>> > > tracing and the > > >>>> >>>>>> verify in almost the same > > >>>> >>>>>> > way; only > > >>>> >>>>>> > >>>>>>> > > difference I can > > >>>> >>>>>> really tell is that the > > >>>> >>>>>> > complain > > >>>> >>>>>> > >>>>>>> > > method from NSK > > >>>> has a > > >>>> >>>>>> max complain before > > >>>> >>>>>> > stopping > > >>>> >>>>>> > >>>>>>> to > > "complain"; I > > >>>> have > > >>>> >>>>>> not added that part > > >>>> >>>>>> > of the > > >>>> >>>>>> > >>>>>>> code > > in this > > >>>> webrev > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> Once > > we decide on > > >>>> >>>>>> these, I can continue on the > > >>>> >>>>>> > >>>>>>> > > files from > > >>>> >>>>>> JDK-8212884 and then do both the > > >>>> >>>>>> > >>>>>>> > > assignment in an > > >>>> if > > >>>> >>>>>> extraction followed-by this > > >>>> >>>>>> > >>>>>>> type > > of webrev in > > >>>> an > > >>>> >>>>>> easier fashion. > > >>>> >>>>>> > Depending on > > >>>> >>>>>> > >>>>>>> > > decisions here, > > >>>> >>>>>> NSK*VERIFY can be deprecated as > > >>>> >>>>>> > >>>>>>> well > > as we go > > >>>> forward. > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > Thanks! > > >>>> >>>>>> > >>>>>>> Jc > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> On > > Mon, Nov 19, > > >>>> 2018 > > >>>> >>>>>> at 11:34 AM Chris Plummer > > >>>> >>>>>> > >>>>>>> < > > >>>> chris.plummer at oracle.com > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >> > > >>>> >>>>>> > >>>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>> wrote: > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > On 11/19/18 > > >>>> 10:07 > > >>>> >>>>>> AM, JC Beyler wrote: > > >>>> >>>>>> > >>>>>>>> > > Hi all, > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > @David/Chris: > > >>>> >>>>>> should I then push this > > >>>> >>>>>> > RFR to > > >>>> >>>>>> > >>>>>>>> > > the hotspot > > >>>> >>>>>> mailing or the runtime > > >>>> >>>>>> > one? For > > >>>> >>>>>> > >>>>>>>> > > what it's > > >>>> worth, > > >>>> >>>>>> a lot of the tests > > >>>> >>>>>> > under the > > >>>> >>>>>> > >>>>>>>> > > vmTestbase > > >>>> are > > >>>> >>>>>> jvmti so the review also > > >>>> >>>>>> > >>>>>>>> > > affects > > >>>> >>>>>> serviceability; it just turns > > >>>> >>>>>> > out I > > >>>> >>>>>> > >>>>>>>> > > started with > > >>>> the > > >>>> >>>>>> GC originally and > > >>>> >>>>>> > then hit > > >>>> >>>>>> > >>>>>>>> > > some other > > >>>> tests > > >>>> >>>>>> I had touched via the > > >>>> >>>>>> > >>>>>>>> > > assignment > > >>>> >>>>>> extraction. > > >>>> >>>>>> > >>>>>>> > > I think > > >>>> hotspot > > >>>> >>>>>> would be best. > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > Chris > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > @Serguei: > > >>>> Done > > >>>> >>>>>> for the method > > >>>> >>>>>> > renaming, for > > >>>> >>>>>> > >>>>>>>> > > the indent, > > >>>> are > > >>>> >>>>>> you talking about > > >>>> >>>>>> > going from > > >>>> >>>>>> > >>>>>>>> > > the 8-indent > > >>>> to > > >>>> >>>>>> 4-indent? If so, would > > >>>> >>>>>> > it not > > >>>> >>>>>> > >>>>>>>> > > just be > > >>>> better > > >>>> >>>>>> to do a new JBS bug and > > >>>> >>>>>> > do the > > >>>> >>>>>> > >>>>>>>> > > whole files > > >>>> in > > >>>> >>>>>> one go? I ask because > > >>>> >>>>>> > >>>>>>>> > > otherwise, it > > >>>> >>>>>> will look a bit weird to > > >>>> >>>>>> > have > > >>>> >>>>>> > >>>>>>>> > > parts of the > > >>>> >>>>>> file as 8-indent and others > > >>>> >>>>>> > >>>>>>>> 4-indent? > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > Thanks for > > >>>> >>>>>> looking at it! > > >>>> >>>>>> > >>>>>>>> > Jc > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > On Mon, Nov > > >>>> 19, > > >>>> >>>>>> 2018 at 1:25 AM > > >>>> >>>>>> > >>>>>>>> > > serguei.spitsyn at oracle.com > > >>>> >>>>>> > > > > >>>> >>>>>> > > > >>>> >>>>>> > >> > > >>>> >>>>>> > >>>>>>>> > >>>> serguei.spitsyn at oracle.com serguei.spitsyn at oracle.com> > > >>>> >>>>>> > > > > >>>> >>>>>> > > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>>>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > >> > > >>>> >>>>>> > >>>>>>>> > >>>> serguei.spitsyn at oracle.com serguei.spitsyn at oracle.com> > > >>>> >>>>>> > > > > >>>> >>>>>> > > > serguei.spitsyn at oracle.com> > > >>>> >>>>>> > >>>> wrote: > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > Hi Jc, > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > We have > > >>>> to > > >>>> >>>>>> start this review > > >>>> >>>>>> > anyway. :) > > >>>> >>>>>> > >>>>>>>> > > It looks > > >>>> >>>>>> good to me in general. > > >>>> >>>>>> > >>>>>>>> > > Thank you > > >>>> >>>>>> for your consistency in this > > >>>> >>>>>> > >>>>>>>> > > >>>> refactoring! > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > Some > > >>>> minor > > >>>> >>>>>> comments. > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > +static > > >>>> >>>>>> const char* > > >>>> >>>>>> > remove_folders(const > > >>>> >>>>>> > >>>>>>>> > > char* > > >>>> >>>>>> fullname) { I'd suggest to > > >>>> >>>>>> > rename > > >>>> >>>>>> > >>>>>>>> > > the > > >>>> function > > >>>> >>>>>> name to something > > >>>> >>>>>> > traditional > > >>>> >>>>>> > >>>>>>>> > > like > > >>>> >>>>>> get_basename. Otherwise, it > > >>>> >>>>>> > sounds > > >>>> >>>>>> > >>>>>>>> > > like this > > >>>> >>>>>> function has to really > > >>>> >>>>>> > remove > > >>>> >>>>>> > >>>>>>>> > > folders. > > >>>> :) > > >>>> >>>>>> Also, all *Locker.cpp have > > >>>> >>>>>> > >>>>>>>> > > wrong > > >>>> indent > > >>>> >>>>>> in the bodies of if > > >>>> >>>>>> > and while > > >>>> >>>>>> > >>>>>>>> > > >>>> statements. > > >>>> >>>>>> Could this be fixed > > >>>> >>>>>> > with the > > >>>> >>>>>> > >>>>>>>> > > >>>> refactoring? > > >>>> >>>>>> I did not look on how > > >>>> >>>>>> > this > > >>>> >>>>>> > >>>>>>>> > > impacts > > >>>> the > > >>>> >>>>>> tests other than > > >>>> >>>>>> > >>>>>>>> > > serviceability. > > >>>> Thanks, > > >>>> >>>>>> Serguei > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > On > > >>>> 11/16/18 > > >>>> >>>>>> 19:43, JC Beyler wrote: > > >>>> >>>>>> > >>>>>>>>> > > Hi all, > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Anybody > > >>>> >>>>>> motivated to review this? :) > > >>>> >>>>>> > >>>>>>>>> > Jc > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > On Wed, Nov > > >>>> 7, > > >>>> >>>>>> 2018 at 9:53 PM JC > > >>>> >>>>>> > Beyler > > >>>> >>>>>> > >>>>>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >> > > >>>> >>>>>> > >>>>>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>> wrote: > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Hi all, > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Could I > > >>>> have > > >>>> >>>>>> a review for the > > >>>> >>>>>> > >>>>>>>>> > > extension > > >>>> >>>>>> and usage of the > > >>>> >>>>>> > >>>>>>>>> ExceptionJniWrapper. > This > > >>>> >>>>>> > adds lines > > >>>> >>>>>> > >>>>>>>>> > > and > > >>>> >>>>>> filenames to the end of the > > >>>> >>>>>> > >>>>>>>>> > > wrapper > > >>>> JNI > > >>>> >>>>>> methods, adds > > >>>> >>>>>> > tracing, > > >>>> >>>>>> > >>>>>>>>> > > and > > >>>> throws > > >>>> >>>>>> an error if need > > >>>> >>>>>> > be. I've > > >>>> >>>>>> > >>>>>>>>> > > ported > > >>>> the > > >>>> >>>>>> gc/lock files to > > >>>> >>>>>> > use the > > >>>> >>>>>> > >>>>>>>>> > > new > > >>>> >>>>>> TRACE_JNI_CALL add-on and > > >>>> >>>>>> > I've > > >>>> >>>>>> > >>>>>>>>> > > ported a > > >>>> few > > >>>> >>>>>> of the tests > > >>>> >>>>>> > that were > > >>>> >>>>>> > >>>>>>>>> > > already > > >>>> >>>>>> changed for the > > >>>> >>>>>> > assignment > > >>>> >>>>>> > >>>>>>>>> > > webrev > > >>>> for > > >>>> >>>>>> JDK-8212884. > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Webrev: > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > > >>>> >>>>>> > >>>>>>>>> > > Bug: > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213501 > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > For > > >>>> >>>>>> illustration, if I force > > >>>> >>>>>> > an error > > >>>> >>>>>> > >>>>>>>>> > > to the > > >>>> >>>>>> AP04/ap04t03 test and > > >>>> >>>>>> > set the > > >>>> >>>>>> > >>>>>>>>> > > verbosity > > >>>> >>>>>> on, I get something > > >>>> >>>>>> > like: > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > >> > > >>>> Calling > > >>>> >>>>>> JNI method > > >>>> >>>>>> > FindClass from > > >>>> >>>>>> > >>>>>>>>> ap04t003.cpp:343 > > >>>> >>>>>> > >>>>>>>>> > > >> > > >>>> Calling > > >>>> >>>>>> with these > > >>>> >>>>>> > parameter(s): > > >>>> >>>>>> > >>>>>>>>> java/lang/Threadd > > >>>> >>>>>> > >>>>>>>>> > > Wait for > > >>>> >>>>>> thread to finish > > >>>> >>>>>> > >>>>>>>>> > > << Called > > >>>> >>>>>> JNI method > > >>>> >>>>>> > FindClass from > > >>>> >>>>>> > >>>>>>>>> ap04t003.cpp:343 > > >>>> >>>>>> > >>>>>>>>> > > >>>> Exception in > > >>>> >>>>>> thread "Thread-0" > > >>>> >>>>>> > >>>>>>>>> > > java.lang.NoClassDefFoundError: > > >>>> >>>>>> > >>>>>>>>> java/lang/Threadd > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>> >>>>>> > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Method) > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Caused > > >>>> by: > > >>>> >>>>>> > >>>>>>>>> > > java.lang.ClassNotFoundException: > > >>>> >>>>>> > >>>>>>>>> java.lang.Threadd > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > > >>>> >>>>>> > >>>>>>>>> > > ... 3 > > >>>> more > > >>>> >>>>>> > >>>>>>>>> > > FATAL > > >>>> ERROR > > >>>> >>>>>> in native method: JNI > > >>>> >>>>>> > >>>>>>>>> > > method > > >>>> >>>>>> FindClass : internal error > > >>>> >>>>>> > >>>>>>>>> > > from > > >>>> >>>>>> ap04t003.cpp:343 > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > > >>>> >>>>>> > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Method) > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > > >>>> > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > at > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> > > > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > >>>> >>>>>> > > >>>> >>>>>> > > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> Questions/comments I > > have about > > >>>> >>>>>> > >>>>>>>>> this are: > > >>>> >>>>>> > >>>>>>>>> > > - Do we > > >>>> >>>>>> want to force fatal > > >>>> >>>>>> > errors > > >>>> >>>>>> > >>>>>>>>> > > when a > > >>>> JNI > > >>>> >>>>>> call fails in general? > > >>>> >>>>>> > >>>>>>>>> > > Most of > > >>>> >>>>>> these tests do the right > > >>>> >>>>>> > >>>>>>>>> > > thing and > > >>>> >>>>>> test the return of > > >>>> >>>>>> > the JNI > > >>>> >>>>>> > >>>>>>>>> > > calls, > > >>>> for > > >>>> >>>>>> example: > > >>>> >>>>>> > >>>>>>>>> > > thrClass > > >>>> = > > >>>> >>>>>> > >>>>>>>>> > > jni->FindClass("java/lang/Threadd", > > >>>> >>>>>> > >>>>>>>>> TRACE_JNI_CALL); > > >>>> >>>>>> > >>>>>>>>> > > if > > >>>> >>>>>> (thrClass == NULL) { > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > but now > > >>>> the > > >>>> >>>>>> wrapper actually > > >>>> >>>>>> > would do > > >>>> >>>>>> > >>>>>>>>> > > a fatal > > >>>> if > > >>>> >>>>>> the FindClass call > > >>>> >>>>>> > would > > >>>> >>>>>> > >>>>>>>>> > > return a > > >>>> >>>>>> nullptr, so we could > > >>>> >>>>>> > remove > > >>>> >>>>>> > >>>>>>>>> > > that test > > >>>> >>>>>> altogether. What do > > >>>> >>>>>> > you > > >>>> >>>>>> > >>>>>>>>> think? > > >>>> >>>>>> > >>>>>>>>> > > - I > > >>>> >>>>>> prefer to leave them > > >>>> >>>>>> > as the > > >>>> >>>>>> > >>>>>>>>> > > tests > > >>>> then > > >>>> >>>>>> become closer to > > >>>> >>>>>> > what real > > >>>> >>>>>> > >>>>>>>>> > > users > > >>>> would > > >>>> >>>>>> have in their > > >>>> >>>>>> > code and is > > >>>> >>>>>> > >>>>>>>>> > > the > > >>>> >>>>>> "recommended" way of doing it > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > - The > > >>>> >>>>>> alternative is to > > >>>> >>>>>> > use the > > >>>> >>>>>> > >>>>>>>>> > > NonFatalError I > > >>>> added > > >>>> >>>>>> which > > >>>> >>>>>> > then just > > >>>> >>>>>> > >>>>>>>>> > > prints > > >>>> out > > >>>> >>>>>> that something > > >>>> >>>>>> > went wrong, > > >>>> >>>>>> > >>>>>>>>> > > letting > > >>>> the > > >>>> >>>>>> test continue. > > >>>> >>>>>> > Question > > >>>> >>>>>> > >>>>>>>>> > > will be > > >>>> what > > >>>> >>>>>> should be the > > >>>> >>>>>> > default? > > >>>> >>>>>> > >>>>>>>>> > > The > > >>>> fatal or > > >>>> >>>>>> the non-fatal error > > >>>> >>>>>> > >>>>>>>>> > > handling? > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > On a > > >>>> >>>>>> different subject: > > >>>> >>>>>> > >>>>>>>>> > > - On > > >>>> the > > >>>> >>>>>> new tests, I've > > >>>> >>>>>> > removed > > >>>> >>>>>> > >>>>>>>>> > > the > > >>>> >>>>>> NSK_JNI_VERIFY since the JNI > > >>>> >>>>>> > >>>>>>>>> > > wrapper > > >>>> >>>>>> handles the tracing > > >>>> >>>>>> > and the > > >>>> >>>>>> > >>>>>>>>> > > verify in > > >>>> >>>>>> almost the same > > >>>> >>>>>> > way; only > > >>>> >>>>>> > >>>>>>>>> > > >>>> difference I > > >>>> >>>>>> can really tell > > >>>> >>>>>> > is that > > >>>> >>>>>> > >>>>>>>>> > > the > > >>>> complain > > >>>> >>>>>> method from NSK > > >>>> >>>>>> > has a > > >>>> >>>>>> > >>>>>>>>> > > max > > >>>> complain > > >>>> >>>>>> before stopping to > > >>>> >>>>>> > >>>>>>>>> > > >>>> "complain"; > > >>>> >>>>>> I have not added that > > >>>> >>>>>> > >>>>>>>>> > > part of > > >>>> the > > >>>> >>>>>> code in this webrev > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Once we > > >>>> >>>>>> decide on these, I can > > >>>> >>>>>> > >>>>>>>>> > > continue > > >>>> on > > >>>> >>>>>> the files from > > >>>> >>>>>> > >>>>>>>>> > > >>>> JDK-8212884 > > >>>> >>>>>> and then do both the > > >>>> >>>>>> > >>>>>>>>> > > >>>> assignment > > >>>> >>>>>> in an if extraction > > >>>> >>>>>> > >>>>>>>>> > > >>>> followed-by > > >>>> >>>>>> this type of > > >>>> >>>>>> > webrev in an > > >>>> >>>>>> > >>>>>>>>> > > easier > > >>>> >>>>>> fashion. Depending on > > >>>> >>>>>> > >>>>>>>>> > > decisions > > >>>> >>>>>> here, NSK*VERIFY can be > > >>>> >>>>>> > >>>>>>>>> > > >>>> deprecated > > >>>> >>>>>> as well as we go > > >>>> >>>>>> > forward. > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > Thank you > > >>>> >>>>>> for the > > >>>> >>>>>> > reviews/comments :) > > >>>> >>>>>> > >>>>>>>>> > > Jc > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > > >>>> >>>>>> > >>>>>>>>> > -- > > >>>> >>>>>> > >>>>>>>>> > > Thanks, > > >>>> >>>>>> > >>>>>>>>> > Jc > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > > >>>> >>>>>> > >>>>>>>> > -- > > >>>> >>>>>> > >>>>>>>> > > Thanks, > > >>>> >>>>>> > >>>>>>>> > Jc > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> > > >>>> >>>>>> > >>>>>>> -- > > >>>> >>>>>> > >>>>>>> > Thanks, > > >>>> >>>>>> > >>>>>>> Jc > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> > > >>>> >>>>>> > >>>>>> -- > > >>>> >>>>>> > >>>>>> Thanks, > > >>>> >>>>>> > >>>>>> Jc > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> > > >>>> >>>>>> > >>>>> -- > > >>>> >>>>>> > >>>>> Thanks, > > >>>> >>>>>> > >>>>> Jc > > >>>> >>>>>> > >>>> > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> -- > > >>>> >>>>>> > >>> Thanks, > > >>>> >>>>>> > >>> Jc > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> -- > > >>>> >>>>>> > >>> > > >>>> >>>>>> > >>> Thanks, > > >>>> >>>>>> > >>> Jc > > >>>> >>>>>> > > > > >>>> >>>>>> > > > > >>>> >>>>>> > > > >>>> >>>>>> > > > >>>> >>>>>> > > > >>>> >>>>>> > -- > > >>>> >>>>>> > > > >>>> >>>>>> > Thanks, > > >>>> >>>>>> > Jc > > >>>> >>>>>> > > >>>> >>>>>> > > >>>> >>>>>> > > >>>> >>>>>> -- > > >>>> >>>>>> Thanks, > > >>>> >>>>>> Jc > > >>>> >>>>> > > >>>> >>>>> > > >>>> >>>>> > > >>>> >>>>> -- > > >>>> >>>>> Thanks, > > >>>> >>>>> Jc > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> -- > > >>>> >>>> Thanks, > > >>>> >>>> Jc > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> -- > > >>>> >>>> > > >>>> >>>> Thanks, > > >>>> >>>> Jc > > >>>> >>> > > >>>> > > > >>>> > > >>> > > >>> > > >>> -- > > >>> > > >>> Thanks, > > >>> Jc > > >>> > > >> > > >> > > >> -- > > >> > > >> Thanks, > > >> Jc > > >> > > > > > > > > > -- > > > > > > Thanks, > > > Jc > > > > > > > > > -- > > > > Thanks, > > Jc > > > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc From thomas.stuefe at gmail.com Tue Mar 26 19:39:30 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 26 Mar 2019 20:39:30 +0100 Subject: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? In-Reply-To: References: Message-ID: Hi Matthias, On Tue, Mar 26, 2019 at 4:17 PM Baesken, Matthias wrote: > > Hello, it seems that the fopen calls in > CompilerOracle::append_comment_to_file and > CompilerOracle::append_exclude_to_file do not have a corresponding fclose. > Or do I miss something ? > > No, you are right, this is a bug. However, I think those functions are nowhere used, so maybe they could be removed completely. Cheers, Thomas We could also add a "true" for the second argument of the fileStream > constructor calls ( fileStream(FILE* file, bool need_close = false) ) > to get the closing. > > > coding : > ------------ > > jdk/src/hotspot/share/compiler/compilerOracle.cpp > > > 741void CompilerOracle::append_comment_to_file(const char* message) { > 742 assert(has_command_file(), "command file must be specified"); > 743 fileStream stream(fopen(cc_file(), "at")); > 744 stream.print("# "); > 745 for (int index = 0; message[index] != '\0'; index++) { > 746 stream.put(message[index]); > 747 if (message[index] == '\n') stream.print("# "); > 748 } > 749 stream.cr(); > 750} > 751 > 752void CompilerOracle::append_exclude_to_file(const methodHandle& method) > { > 753 assert(has_command_file(), "command file must be specified"); > 754 fileStream stream(fopen(cc_file(), "at")); > 755 stream.print("exclude "); > 756 method->method_holder()->name()->print_symbol_on(&stream); > 757 stream.print("."); > 758 method->name()->print_symbol_on(&stream); > 759 method->signature()->print_symbol_on(&stream); > 760 stream.cr(); > 761 stream.cr(); > 762} > > Best regards, Matthias > From christoph.langer at sap.com Tue Mar 26 22:17:23 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 26 Mar 2019 22:17:23 +0000 Subject: [RFR]: [S390] Fix JFR profiling In-Reply-To: References: Message-ID: +1. Please update the copyright years when pushing. > -----Original Message----- > From: hotspot-dev On Behalf Of > Doerr, Martin > Sent: Dienstag, 26. M?rz 2019 16:45 > To: Haug, Gunter ; hotspot-dev at openjdk.java.net > Subject: RE: [RFR]: [S390] Fix JFR profiling > > Hi Gunter, > > looks good and trivial. I can push it tomorrow if there are no objections. > > Thanks for fixing. > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of > Haug, Gunter > Sent: Dienstag, 26. M?rz 2019 16:31 > To: hotspot-dev at openjdk.java.net > Subject: [RFR]: [S390] Fix JFR profiling > > Hi all, > > can I please have reviews and a sponsor for the following tiny fix which only > affects one s390 platform file: > > https://bugs.openjdk.java.net/browse/JDK-8221479 > http://cr.openjdk.java.net/~ghaug/webrevs/8221479/ > > It detects addresses which are totally off and would have been dereferenced > otherwise. This caused very rare crashes on s390. > > > Thanks, > Gunter > > From coleen.phillimore at oracle.com Tue Mar 26 23:03:51 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 26 Mar 2019 19:03:51 -0400 Subject: RFR (S) 8074355: make MutexLocker smarter about non-JavaThreads In-Reply-To: <80323289-ab58-6f81-d56c-4a6421d29374@oracle.com> References: <80323289-ab58-6f81-d56c-4a6421d29374@oracle.com> Message-ID: Kim and I had a discussion about this, so I'm withdrawing this change. Coleen On 3/22/19 12:32 PM, coleen.phillimore at oracle.com wrote: > Summary: Use safepoint_check_always/safepoint_check_never instead of > safepoint_check_sometimes for locks that are taken by JavaThreads and > non-JavaThreads > > This is a small cleanup to avoid having safepoint_check_sometimes. > Threads_lock is the only one remaining.? 'Sometimes' was used for some > locks because they were taken out by GC or other threads and > JavaThreads, where the other threads weren't going to check for the > safepoint anyway since they don't participate in the safepoint > protocol.? There could be further cleanups here.? We could make the > safepoint checking related to the rank of the lock in the future, or > the safepoint checking could be a property of the lock so we don't > have to pass in Mutex::_no_safepoint_check_flag (except > Threads_lock).? I think these could be further enhancements after this. > > Tested with hs-tier1-6, runThese jck with G1, CMS, Serial and ParallelGC. > > 8074355: make MutexLocker smarter about non-JavaThreads > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8074355.01/webrev > > Thanks, > Coleen From kim.barrett at oracle.com Wed Mar 27 01:18:28 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 26 Mar 2019 21:18:28 -0400 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: <6fcffd19-a198-9ee5-46a5-dcbf3c3ed2a9@redhat.com> References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> <43C7702A-C046-4E9A-9773-889E7741FBF1@oracle.com> <7571856a-5809-ab9d-c2be-5b6855be5c3c@redhat.com> <6fcffd19-a198-9ee5-46a5-dcbf3c3ed2a9@redhat.com> Message-ID: <204D8F46-0366-4764-8F1A-030CE2A560D7@oracle.com> > On Mar 26, 2019, at 6:31 AM, Roman Kennke wrote: >> OK. So does that count as a "review"? And I still need a second. > > No, not yet. > > I've tested it against my failing stress test (together with your njt_lock patch) and it seems to fix it. > > I'm also testing it against Shenandoah test suite, and so far looks good. OK, thanks for doing the additional testing. > I don't quite understand the resetting logic though. It first increments the global counter. If it turns 0, it resets all thread tokens to 0, using the claim_oops_do/cmpxchg protocol, but that is now expecting 0 too? Why not simply assign 0 instead? Maybe that code just isn't obvious enough?? There isn't a setter for the per-thread token, only the claim function. But note that we're using the non-parallel variant (the first argument is false), so it effectively is just a simple assignment to the new claim value (when not already that value), with no cmpxchg involved. > Also, the resetting logic warrants a test, it is so rare, it will otherwise not get executed, but if it fails, it would be catastrophic. I was hoping the resetting code was simple enough to not require a test. I will see if I can come up with a sensible gtest. From kim.barrett at oracle.com Wed Mar 27 01:53:49 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 26 Mar 2019 21:53:49 -0400 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: <44663bc09a529c415470ce609edc2e1f59f9b97b.camel@oracle.com> References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> <44663bc09a529c415470ce609edc2e1f59f9b97b.camel@oracle.com> Message-ID: <31513282-B0BA-4C50-894D-5508FD53F27B@oracle.com> > On Mar 26, 2019, at 6:12 AM, Thomas Schatzl wrote: > > Hi, > > On Thu, 2019-03-21 at 18:25 -0400, Kim Barrett wrote: >> Please review this change to the thread claiming protocol for >> parallel iteration over threads. Previously, the protocol used a 1bit >> claim counter plus zero as a "never claimed" marker. The new protocol >> uses a uintx counter, while retaining zero as a "never claimed" >> marker. >> >> > [...] >> We address this by expanding the claim counter to a uintx-sized >> counter. There is still a theoretical possibility of collision, which >> > > [...] >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8221102 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8221102/open.00/ >> >> Testing: >> mach5 tier1-5. >> > > looks good. One minor style nit: I would prefer that in thread.cpp: > Threads::change_hread_claim_token():4521 the thread_claim_token would > be set to a particular value (i.e. 1) directly instead of incremented > from its current original value. This makes that assignment context- > free. But I am fine with keeping it as is. > > Thanks, > Thomas I?ll make that change. Thanks for the review. From yumin.qi at gmail.com Wed Mar 27 03:35:44 2019 From: yumin.qi at gmail.com (yumin qi) Date: Tue, 26 Mar 2019 20:35:44 -0700 Subject: RFC: JWarmup precompile java hot methods at application startup In-Reply-To: References: <1a9e9d8c-9a80-4cf7-9510-7de1cc9cbc4e.kuaiwei.kw@alibaba-inc.com> Message-ID: Hi, Tobias We will guard the flag and test cases on Linux platform only. BTW, it passed submit repo tests. Thanks Yumin On Tue, Mar 26, 2019 at 12:53 AM Tobias Hartmann wrote: > > On 22.03.19 13:15, Kuai Wei wrote: > > jwarmup is designed to work on linux x86-64 only. So we have not tested > on other platform. > > Okay, in this case it should not be possible to enable the feature on any > other platform (if the > corresponding flag is set, a "... not supported on this platform" warning > should be issued and the > flag should be disabled, see examples in arguments.cpp). > > Also, the corresponding tests should only be executed on supported > platforms. > > Best regards, > Tobias > From fujie at loongson.cn Wed Mar 27 03:47:35 2019 From: fujie at loongson.cn (Jie Fu) Date: Wed, 27 Mar 2019 11:47:35 +0800 Subject: [Request for help] Questions about make test TEST="micro" failures Message-ID: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> Hi all, Following benchmarks failed when I do 'make test TEST="micro"'. ---------------------------------------------- org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testVolatile org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testMutable org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testVolatile org.openjdk.bench.java.lang.invoke.CallSiteStable.testConstant org.openjdk.bench.java.lang.invoke.CallSiteStable.testMutable org.openjdk.bench.java.lang.invoke.CallSiteStable.testVolatile ---------------------------------------------- These failures can also be reproduced by a single run of each benchmark, e.g. ? make test TEST="micro:org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable" Questions: why there are so many shouldNotCallThis() in CallSiteSetTarget.java/CallSiteSetTargetSelf.java/CallSiteStable.java? ?????????? How can I run these benchmarks properly? Thanks a lot. Best regards, Jie From david.holmes at oracle.com Wed Mar 27 04:12:55 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Mar 2019 14:12:55 +1000 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> Message-ID: <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> Hi Jie, These aren't hotspot tests. Maybe ask on core-libs-dev? Also I thought "make test ..." was no longer supported? David On 27/03/2019 1:47 pm, Jie Fu wrote: > Hi all, > > Following benchmarks failed when I do 'make test TEST="micro"'. > ---------------------------------------------- > org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable > org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testVolatile > org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testMutable > org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testVolatile > org.openjdk.bench.java.lang.invoke.CallSiteStable.testConstant > org.openjdk.bench.java.lang.invoke.CallSiteStable.testMutable > org.openjdk.bench.java.lang.invoke.CallSiteStable.testVolatile > ---------------------------------------------- > > These failures can also be reproduced by a single run of each benchmark, > e.g. > ? make test > TEST="micro:org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable" > > > Questions: why there are so many shouldNotCallThis() in > CallSiteSetTarget.java/CallSiteSetTargetSelf.java/CallSiteStable.java? > ?????????? How can I run these benchmarks properly? > > Thanks a lot. > > > Best regards, > Jie > > From kim.barrett at oracle.com Wed Mar 27 05:09:02 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 27 Mar 2019 01:09:02 -0400 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> Message-ID: > On Mar 27, 2019, at 12:12 AM, David Holmes wrote: > Also I thought "make test ..." was no longer supported? Not according to docs/testing.md. ?make test? used to invoke an old system for running tests, and ?make run-test? was the new way, but that was changed a few months ago: JDK-8210958: Rename "make run-test" to "make test? From david.holmes at oracle.com Wed Mar 27 06:41:32 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Mar 2019 16:41:32 +1000 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> Message-ID: <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> On 27/03/2019 3:09 pm, Kim Barrett wrote: >> On Mar 27, 2019, at 12:12 AM, David Holmes wrote: >> Also I thought "make test ..." was no longer supported? > > Not according to docs/testing.md. ?make test? used to invoke an old system for running > tests, and ?make run-test? was the new way, but that was changed a few months ago: > JDK-8210958: Rename "make run-test" to "make test? Ah yes confused myself. I sponsored the change that updated the docs just the other day. Personally I never user make to run tests (at least not directly). Thanks, David From fujie at loongson.cn Wed Mar 27 06:56:34 2019 From: fujie at loongson.cn (Jie Fu) Date: Wed, 27 Mar 2019 14:56:34 +0800 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> Message-ID: <35d3331a-258a-3d75-0bd9-09c0072ebdd9@loongson.cn> Hi David, > Personally I never user make to run tests (at least not directly). How did you usually run the tests in the repository? Is there a more convenient or efficient way? Thanks a lot. Best regards, Jie From rkennke at redhat.com Wed Mar 27 08:37:05 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 27 Mar 2019 09:37:05 +0100 Subject: RFR: 8221102: Allow GC threads to participate in threads claiming protocol In-Reply-To: <204D8F46-0366-4764-8F1A-030CE2A560D7@oracle.com> References: <38977DC6-957B-400A-8A19-1EA6977F4A2F@oracle.com> <43C7702A-C046-4E9A-9773-889E7741FBF1@oracle.com> <7571856a-5809-ab9d-c2be-5b6855be5c3c@redhat.com> <6fcffd19-a198-9ee5-46a5-dcbf3c3ed2a9@redhat.com> <204D8F46-0366-4764-8F1A-030CE2A560D7@oracle.com> Message-ID: >>> OK. So does that count as a "review"? And I still need a second. >> >> No, not yet. >> >> I've tested it against my failing stress test (together with your njt_lock patch) and it seems to fix it. >> >> I'm also testing it against Shenandoah test suite, and so far looks good. > > OK, thanks for doing the additional testing. > >> I don't quite understand the resetting logic though. It first increments the global counter. If it turns 0, it resets all thread tokens to 0, using the claim_oops_do/cmpxchg protocol, but that is now expecting 0 too? Why not simply assign 0 instead? Maybe that code just isn't obvious enough?? > > There isn't a setter for the per-thread token, only the claim > function. But note that we're using the non-parallel variant (the > first argument is false), so it effectively is just a simple > assignment to the new claim value (when not already that value), with > no cmpxchg involved. Ah, ok. Well, I don't like methods that behave differently on a boolean flag argument for a reason. >> Also, the resetting logic warrants a test, it is so rare, it will otherwise not get executed, but if it fails, it would be catastrophic. > > I was hoping the resetting code was simple enough to not require a > test. I will see if I can come up with a sensible gtest. I guess the 'design' of the claiming protocol is just making almost everything not-obvious. I think for this reason alone I shall go ahead and work on limiting/eliminating its usage outside of thread.cpp anyway. If it's too hard to come up with a gtest, then go ahead with your proposed change. Roman From matthias.baesken at sap.com Wed Mar 27 08:41:39 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 27 Mar 2019 08:41:39 +0000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] Message-ID: Hello, please review the following change . Recent Linux kernels provide in the /proc filesystem so called "steal" information, see the the proc man page : .. "there is an eighth column, steal - stolen time, which is the time spent in other operating systems when running in a virtualized environment" The info was useful in our internal VM, when looking into performance issues (especially in virtualized environments), so it should be added to the hs_err file . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8221535 http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ Best Regards, Matthias From martin.doerr at sap.com Wed Mar 27 09:09:10 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 27 Mar 2019 09:09:10 +0000 Subject: [RFR]: [S390] Fix JFR profiling In-Reply-To: References: Message-ID: Pushed with updated copyright. Best regards, Martin -----Original Message----- From: Langer, Christoph Sent: Dienstag, 26. M?rz 2019 23:17 To: Doerr, Martin ; Haug, Gunter ; hotspot-dev at openjdk.java.net Subject: RE: [RFR]: [S390] Fix JFR profiling +1. Please update the copyright years when pushing. > -----Original Message----- > From: hotspot-dev On Behalf Of > Doerr, Martin > Sent: Dienstag, 26. M?rz 2019 16:45 > To: Haug, Gunter ; hotspot-dev at openjdk.java.net > Subject: RE: [RFR]: [S390] Fix JFR profiling > > Hi Gunter, > > looks good and trivial. I can push it tomorrow if there are no objections. > > Thanks for fixing. > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of > Haug, Gunter > Sent: Dienstag, 26. M?rz 2019 16:31 > To: hotspot-dev at openjdk.java.net > Subject: [RFR]: [S390] Fix JFR profiling > > Hi all, > > can I please have reviews and a sponsor for the following tiny fix which only > affects one s390 platform file: > > https://bugs.openjdk.java.net/browse/JDK-8221479 > http://cr.openjdk.java.net/~ghaug/webrevs/8221479/ > > It detects addresses which are totally off and would have been dereferenced > otherwise. This caused very rare crashes on s390. > > > Thanks, > Gunter > > From david.holmes at oracle.com Wed Mar 27 09:22:40 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Mar 2019 19:22:40 +1000 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <35d3331a-258a-3d75-0bd9-09c0072ebdd9@loongson.cn> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> <35d3331a-258a-3d75-0bd9-09c0072ebdd9@loongson.cn> Message-ID: Hi Jie, On 27/03/2019 4:56 pm, Jie Fu wrote: > Hi David, > >> Personally I never user make to run tests (at least not directly). > How did you usually run the tests in the repository? I cd to the test directory and use jtreg directly. > Is there a more convenient or efficient way? I find it more effective as it removes a layer of software from the testing process. But it's mainly what you're used to. Cheers, David > Thanks a lot. > > Best regards, > Jie > From robbin.ehn at oracle.com Wed Mar 27 09:23:12 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 27 Mar 2019 10:23:12 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> Message-ID: Thanks Martin! /Robbin On 3/26/19 2:45 PM, Doerr, Martin wrote: > Hi Robbin, > > the rebased version looks good to me. > > I'm only missing barrier implementations for aarch64 and SPARC. > I'd expect isb on aarch64 and at least a compiler barrier on SPARC. But I'm not an expert for these platforms. > > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 26. M?rz 2019 13:51 > To: hotspot-dev at openjdk.java.net; Doerr, Martin ; David Holmes ; Erik Osterlund ; Reingruber, Richard ; Andrew Haley > Subject: Re: RFR(m): 8220351: Cross-modifying code > > Hi all, please review. > > This is v4, which is just an rebase on-top of "RFR: 8221207: Redo JDK-8218446 - > SuspendAtExit hangs". The commit simplified S/R code a bit, making this patch > simpler. > > Full: > http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ > Inc (not complete since v3 did not apply proper, one of the reject wasn't > needed): > http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ > > To summarize, before 8221207 we had 4 safe states: > *Off threads_list (_thread_new) > *Blocked (_thread_blocked) > *In native (_thread_in_native) > *Suspend (*ANY*) > *After handshake (X) > > After 8221207 we have: > *Off threads_list (_thread_new) (far from cross-mod code) > *Blocked (_thread_blocked) > *In native (_thread_in_native) > *After handshake (X) > > -We have one instruction barrier for when going from _thread_new. > -We have four instruction barrier for when going from blocked: > - ThreadBlockInVM > - ThreadBlockInVMWithDeadlockCheck > - The manual blocked transition for suspended blocked case. > - Blocked in SS:block() uses the barrier in block_if_requested_slow. > -We let threads doing their own handshake hit the barrier in > block_if_requested_slow. > -We let threads in native stay armed so they hit the barrier in > block_if_requested_slow. > > Not disarming threads in native works because, false positives already happens > today and are handled. Both handshakes and safepoints disarms in correct order, > so a JavaThread can notice that is should be armed, but with false positives. > Before first check of handshake/safepoint the JavaThread is in a unsafe state. > After that check a new handshake/safepoint cannot start/progress since the > JavaThread is unsafe. Thus re-arming and continue to next polling site is okay. > > Patch only adds barrier for cross-modifying code that happens during a > handshake/safepoint and implements this barrier for those platforms which have a > simple implementaion for accomplish it. sparc/arm32/aarch64 needs additional > work. > > The deprecated global poll path does not contain proper barriers, and will be > obsoleted in JDK 14. > > Thanks, Robbin > > On 2019-03-08 16:24, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it can start >> executing updated code. E.g. an oop in the instruction stream can have been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is far away to >> guarantee that the thread executing the updated instruction stream sees the >> modification. >> >> What far away is and how far an update instruction stream is from a safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From thomas.stuefe at gmail.com Wed Mar 27 09:27:50 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 27 Mar 2019 10:27:50 +0100 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: Hi Matthias, this is a lot of coding for just one counter... Also os_perf_linux.cpp does something very similar. However, that code does not get the steal ticks you want, does not seem to care about kernel differences and could also be improved. Proposal: Why do you not tweak your new API to be a general purpose API to read all ticks from /proc/stat. That would be way more useful. Later, if we feel like it, could change os_perf_linux.cpp to use your function. We did something similar for the statistical history in the SapMachine, see https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linux/stathist_linux.cpp . So I would suggest to have a function reading proc stat and returning all values in a struct. That struct should have provisions for "value not found" since which values you get depends in kernel version. -------- Example: struct value_t { bool valid; uint64_t v; } struct cpu_values_t { value_t user; value_t nice; value_t system; value_t idle; value_t iowait; value_t steal; value_t guest; value_t guest_nice; }; void parse_proc_stat_cpu_line(const char* line, cpu_values_t* out) { // Note: existence of some of these values depends on kernel version out->user.valid = out->nice.valid = out->system.valid = out->idle.valid = out->iowait.valid = out->steal.valid = out->guest.valid = out->guest_nice.valid = false; int user, nice, system, idle, iowait, irq, softirq, steal, guest, guest_nice; int num = ::sscanf(line, "cpu %d %d %d %d %d %d %d %d %d %d", &user, &nice, &system, &idle, &iowait, &irq, &softirq, &steal, &guest, &guest_nice); if (num >= 4) { out->user.v = user; out->user.valid = true; out->nice.v = nice; out->nice.valid = true; out->system.v = system; out->system.valid = true; out->idle.v = idle; out->idle.valid = true; if (num >= 5) { // iowait (5) (since Linux 2.5.41) out->iowait = iowait; out->iowait.valid = true; if (num >= 8) { // steal (8) (since Linux 2.6.11) out->steal = steal; out->steal.valid = true; if (num >= 9) { // guest (9) (since Linux 2.6.24) out->guest = guest; out->guest.valid = true; if (num >= 10) { // guest (9) (since Linux 2.6.33) out->guest_nice = guest_nice; out->guest_nice.valid = true; } } } } Cheers Thomas On Wed, Mar 27, 2019 at 9:42 AM Baesken, Matthias wrote: > Hello, please review the following change . > > Recent Linux kernels provide in the /proc filesystem so called "steal" > information, see the the proc man page : > > .. > "there is an eighth column, steal - stolen time, which is the time spent > in other operating systems when running in a virtualized environment" > > The info was useful in our internal VM, when looking into performance > issues (especially in virtualized environments), so it should be added to > the hs_err file . > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8221535 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ > > > Best Regards, Matthias > > From david.holmes at oracle.com Wed Mar 27 09:28:30 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Mar 2019 19:28:30 +1000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: Hi Matthias, We already have code of this form in os_perf_linux.cpp. Can you update that for steal-ticks and reuse it? Thanks, David On 27/03/2019 6:41 pm, Baesken, Matthias wrote: > Hello, please review the following change . > > Recent Linux kernels provide in the /proc filesystem so called "steal" information, see the the proc man page : > > .. > "there is an eighth column, steal - stolen time, which is the time spent in other operating systems when running in a virtualized environment" > > The info was useful in our internal VM, when looking into performance issues (especially in virtualized environments), so it should be added to the hs_err file . > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8221535 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ > > > Best Regards, Matthias > From matthias.baesken at sap.com Wed Mar 27 09:29:44 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 27 Mar 2019 09:29:44 +0000 Subject: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? Message-ID: Hi Thomas, good point . I removed the 2 functions . Btw. I checked them (using nm ) in the product libjvm.so and they showed up in the binary . So it might help also to get the binary a little bit smaller ? Please review ! Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8221541 http://cr.openjdk.java.net/~mbaesken/webrevs/8221541.0/ Thanks, Matthias From: Thomas St?fe Sent: Dienstag, 26. M?rz 2019 20:40 To: Baesken, Matthias Cc: hotspot-dev at openjdk.java.net Subject: Re: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? Hi Matthias, On Tue, Mar 26, 2019 at 4:17 PM Baesken, Matthias > wrote: Hello, it seems that the fopen calls in CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file do not have a corresponding fclose. Or do I miss something ? No, you are right, this is a bug. However, I think those functions are nowhere used, so maybe they could be removed completely. Cheers, Thomas We could also add a "true" for the second argument of the fileStream constructor calls ( fileStream(FILE* file, bool need_close = false) ) to get the closing. coding : ------------ jdk/src/hotspot/share/compiler/compilerOracle.cpp 741void CompilerOracle::append_comment_to_file(const char* message) { 742 assert(has_command_file(), "command file must be specified"); 743 fileStream stream(fopen(cc_file(), "at")); 744 stream.print("# "); 745 for (int index = 0; message[index] != '\0'; index++) { 746 stream.put(message[index]); 747 if (message[index] == '\n') stream.print("# "); 748 } 749 stream.cr(); 750} 751 752void CompilerOracle::append_exclude_to_file(const methodHandle& method) { 753 assert(has_command_file(), "command file must be specified"); 754 fileStream stream(fopen(cc_file(), "at")); 755 stream.print("exclude "); 756 method->method_holder()->name()->print_symbol_on(&stream); 757 stream.print("."); 758 method->name()->print_symbol_on(&stream); 759 method->signature()->print_symbol_on(&stream); 760 stream.cr(); 761 stream.cr(); 762} Best regards, Matthias From adinn at redhat.com Wed Mar 27 09:39:52 2019 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 27 Mar 2019 09:39:52 +0000 Subject: 8219993: AArch64: Compiled CI stubs are unsafely modified In-Reply-To: <2fa5fcb0-3bcc-f490-4948-405e23c10e40@redhat.com> References: <3abc4b04-a6d4-62f7-4cd9-36a89fe020e4@redhat.com> <2fa5fcb0-3bcc-f490-4948-405e23c10e40@redhat.com> Message-ID: On 26/03/2019 18:26, Andrew Haley wrote: > Here's a simpler patch. All I do this time is insert an ISB between the > call and the load instructions so that we don't have a stale instruction > buffer. > > I also took the opportunity to unify some code which was duplicated in > C1 and compiledIC_aarch64. > > Andrew Dinn, please approve this one. :-) > > http://cr.openjdk.java.net/~aph/8219993.2/ That looks ok modulo one detail I don't really understand which, I think, applies to both old and new code. The stub target address gets updated by NativeMovConstReg::set_data which calls pd_patch_instruction. When that is called in some patching thread how is it guaranteed that some other reading/executing thread does not observe partial writes of the movz+movk+movk sequence? I understand that the patching thread will flush the icache after completing all 3 writes and that threads executing the stub will do an isb to update their view of the code. What stops the icache getting flushed in mid-write for some other reason (e.g. prefetch of instructions in the patching thread) allowing the reader -- or indeed another thread competing to set the stub data -- to see an invalid address? I suppose the same question applies for the write of the constant loaded into rscratch1. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From fujie at loongson.cn Wed Mar 27 09:41:48 2019 From: fujie at loongson.cn (Jie Fu) Date: Wed, 27 Mar 2019 17:41:48 +0800 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> <35d3331a-258a-3d75-0bd9-09c0072ebdd9@loongson.cn> Message-ID: <1a1834f7-3922-1233-5c45-5de83d269241@loongson.cn> I got it. Thanks David. On 2019/3/27 ??5:22, David Holmes wrote: > Hi Jie, > > On 27/03/2019 4:56 pm, Jie Fu wrote: >> Hi David, >> >>> Personally I never user make to run tests (at least not directly). >> How did you usually run the tests in the repository? > > I cd to the test directory and use jtreg directly. > >> Is there a more convenient or efficient way? > > I find it more effective as it removes a layer of software from the > testing process. But it's mainly what you're used to. > > Cheers, > David > >> Thanks a lot. >> >> Best regards, >> Jie >> From david.holmes at oracle.com Wed Mar 27 09:50:14 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Mar 2019 19:50:14 +1000 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> <35d3331a-258a-3d75-0bd9-09c0072ebdd9@loongson.cn> Message-ID: <70d3f07c-855b-f356-d2cc-3061de40e572@oracle.com> It's been pointed out to me that these "micro" "tests" are outside of jtreg and can only be run via make. Cheers, David On 27/03/2019 7:22 pm, David Holmes wrote: > Hi Jie, > > On 27/03/2019 4:56 pm, Jie Fu wrote: >> Hi David, >> >>> Personally I never user make to run tests (at least not directly). >> How did you usually run the tests in the repository? > > I cd to the test directory and use jtreg directly. > >> Is there a more convenient or efficient way? > > I find it more effective as it removes a layer of software from the > testing process. But it's mainly what you're used to. > > Cheers, > David > >> Thanks a lot. >> >> Best regards, >> Jie >> From aph at redhat.com Wed Mar 27 09:56:23 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 27 Mar 2019 09:56:23 +0000 Subject: 8219993: AArch64: Compiled CI stubs are unsafely modified In-Reply-To: References: <3abc4b04-a6d4-62f7-4cd9-36a89fe020e4@redhat.com> <2fa5fcb0-3bcc-f490-4948-405e23c10e40@redhat.com> Message-ID: <72275f80-303c-36c6-0a96-08977f808d29@redhat.com> On 3/27/19 9:39 AM, Andrew Dinn wrote: > On 26/03/2019 18:26, Andrew Haley wrote: >> Here's a simpler patch. All I do this time is insert an ISB between the >> call and the load instructions so that we don't have a stale instruction >> buffer. >> >> I also took the opportunity to unify some code which was duplicated in >> C1 and compiledIC_aarch64. >> >> Andrew Dinn, please approve this one. :-) >> >> http://cr.openjdk.java.net/~aph/8219993.2/ > That looks ok modulo one detail I don't really understand which, I > think, applies to both old and new code. > > The stub target address gets updated by NativeMovConstReg::set_data > which calls pd_patch_instruction. When that is called in some patching > thread how is it guaranteed that some other reading/executing thread > does not observe partial writes of the movz+movk+movk sequence? It's only written once, when the c2i stub is first used. After that it is only rewritten at a safepoint, and we assert this. It's written synchronously by the first thread that encounters it, and there is a lock to prevent races. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From david.holmes at oracle.com Wed Mar 27 09:58:57 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Mar 2019 19:58:57 +1000 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> Message-ID: <003054ae-76a9-ef83-82a8-35efbbc7a30d@oracle.com> Hi Robbin, On 26/03/2019 10:51 pm, Robbin Ehn wrote: > Hi all, please review. > > This is v4, which is just an rebase on-top of "RFR: 8221207: Redo > JDK-8218446 - > SuspendAtExit hangs". The commit simplified S/R code a bit, making this > patch simpler. That's good :) > Full: > http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ > Inc (not complete since v3 did not apply proper, one of the reject wasn't > needed): > http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ > > To summarize, before 8221207 we had 4 safe states: > *Off threads_list (_thread_new) > *Blocked????????? (_thread_blocked) > *In native??????? (_thread_in_native) > *Suspend????????? (*ANY*) > *After handshake? (X) > > After 8221207 we have: > *Off threads_list (_thread_new) (far from cross-mod code) > *Blocked????????? (_thread_blocked) > *In native??????? (_thread_in_native) > *After handshake? (X) > > -We have one instruction barrier for when going from _thread_new. > -We have four instruction barrier for when going from blocked: > ????- ThreadBlockInVM > ????- ThreadBlockInVMWithDeadlockCheck > ????- The manual blocked transition for suspended blocked case. > ????- Blocked in SS:block() uses the barrier in block_if_requested_slow. I finally realized how this translates into covering all the return-to-Java paths, without having as many unnecessary calls to cross_modify_barrier. If we return to Java after a safepoint (or action that could have installed modified code) we need the barrier. If a safepoint has occurred then we must have become safepoint-safe at some point, which means we either were in _thread_blocked (by one means or another) or _thread_in_native. So we guard the transitions out of those states. > -We let threads doing their own handshake hit the barrier in > block_if_requested_slow. > -We let threads in native stay armed so they hit the barrier in > block_if_requested_slow. > > Not disarming threads in native works because, false positives > already happens today and are handled. Both handshakes and safepoints > disarms in correct order, so a JavaThread can notice that is should > be armed, but with false positives. Before first check of > handshake/safepoint the JavaThread is in a unsafe state. After that > check a new handshake/safepoint cannot start/progress since the > JavaThread is unsafe. Thus re-arming and continue to next polling > site is okay. I'll take your word for it. I can't claim sufficient knowledge of the handshake mechanism to truly comment. > Patch only adds barrier for cross-modifying code that happens during > a handshake/safepoint and implements this barrier for those > platforms which have a simple implementaion for accomplish it. > sparc/arm32/aarch64 needs additional work. I suggest creating subtasks for those platforms so this additional work isn't forgotten. > The deprecated global poll path does not contain proper barriers, and > will be obsoleted in JDK 14. Ok. A couple of minor typos: src/hotspot/share/runtime/safepointMechanism.inline.hpp + // JavaThread will disarm it self ... s/it self/itself/ --- src/hotspot/share/runtime/thread.cpp + // from here is probably far enought, ... s/enought/enough/ + // Since we are not using a regular thread-state tranisition helper here, s/tranisition/transition/ Thanks, David ----- > Thanks, Robbin > > On 2019-03-08 16:24, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it >> can start >> executing updated code. E.g. an oop in the instruction stream can have >> been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is >> far away to >> guarantee that the thread executing the updated instruction stream >> sees the >> modification. >> >> What far away is and how far an update instruction stream is from a >> safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier >> must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a >> safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra >> serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to >> have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the >> thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From claes.redestad at oracle.com Wed Mar 27 10:03:03 2019 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 27 Mar 2019 11:03:03 +0100 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> Message-ID: <2111ddfd-3171-5351-b3f4-4fd167599073@oracle.com> Hi Jie, generally I'd not recommend running all microbenchmarks in one go, i.e, qualify micro:. Mostly from an operational point of view as running them all (with default settings) will take days. I'll have a look at these CallSite micros. I suspect we don't run them regularly and they might have atrophied. /Claes On 2019-03-27 04:47, Jie Fu wrote: > Hi all, > > Following benchmarks failed when I do 'make test TEST="micro"'. > ---------------------------------------------- > org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable > org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testVolatile > org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testMutable > org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testVolatile > org.openjdk.bench.java.lang.invoke.CallSiteStable.testConstant > org.openjdk.bench.java.lang.invoke.CallSiteStable.testMutable > org.openjdk.bench.java.lang.invoke.CallSiteStable.testVolatile > ---------------------------------------------- > > These failures can also be reproduced by a single run of each benchmark, > e.g. > ? make test > TEST="micro:org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable" > > > Questions: why there are so many shouldNotCallThis() in > CallSiteSetTarget.java/CallSiteSetTargetSelf.java/CallSiteStable.java? > ?????????? How can I run these benchmarks properly? > > Thanks a lot. > > > Best regards, > Jie > > From matthias.baesken at sap.com Wed Mar 27 10:04:10 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 27 Mar 2019 10:04:10 +0000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: Hi Thomas, interesting idea . David ? would you be fine with what Thomas proposes ? Best regards, Matthias From: Thomas St?fe Sent: Mittwoch, 27. M?rz 2019 10:28 To: Baesken, Matthias Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR: 8221535: add steal tick related information to hs_error file [linux] Hi Matthias, this is a lot of coding for just one counter... Also os_perf_linux.cpp does something very similar. However, that code does not get the steal ticks you want, does not seem to care about kernel differences and could also be improved. Proposal: Why do you not tweak your new API to be a general purpose API to read all ticks from /proc/stat. That would be way more useful. Later, if we feel like it, could change os_perf_linux.cpp to use your function. We did something similar for the statistical history in the SapMachine, see https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linux/stathist_linux.cpp . So I would suggest to have a function reading proc stat and returning all values in a struct. That struct should have provisions for "value not found" since which values you get depends in kernel version. -------- Example: struct value_t { bool valid; uint64_t v; } struct cpu_values_t { value_t user; value_t nice; value_t system; value_t idle; value_t iowait; value_t steal; value_t guest; value_t guest_nice; }; void parse_proc_stat_cpu_line(const char* line, cpu_values_t* out) { // Note: existence of some of these values depends on kernel version out->user.valid = out->nice.valid = out->system.valid = out->idle.valid = out->iowait.valid = out->steal.valid = out->guest.valid = out->guest_nice.valid = false; int user, nice, system, idle, iowait, irq, softirq, steal, guest, guest_nice; int num = ::sscanf(line, "cpu %d %d %d %d %d %d %d %d %d %d", &user, &nice, &system, &idle, &iowait, &irq, &softirq, &steal, &guest, &guest_nice); if (num >= 4) { out->user.v = user; out->user.valid = true; out->nice.v = nice; out->nice.valid = true; out->system.v = system; out->system.valid = true; out->idle.v = idle; out->idle.valid = true; if (num >= 5) { // iowait (5) (since Linux 2.5.41) out->iowait = iowait; out->iowait.valid = true; if (num >= 8) { // steal (8) (since Linux 2.6.11) out->steal = steal; out->steal.valid = true; if (num >= 9) { // guest (9) (since Linux 2.6.24) out->guest = guest; out->guest.valid = true; if (num >= 10) { // guest (9) (since Linux 2.6.33) out->guest_nice = guest_nice; out->guest_nice.valid = true; } } } } Cheers Thomas On Wed, Mar 27, 2019 at 9:42 AM Baesken, Matthias > wrote: Hello, please review the following change . Recent Linux kernels provide in the /proc filesystem so called "steal" information, see the the proc man page : .. "there is an eighth column, steal - stolen time, which is the time spent in other operating systems when running in a virtualized environment" The info was useful in our internal VM, when looking into performance issues (especially in virtualized environments), so it should be added to the hs_err file . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8221535 http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ Best Regards, Matthias From adinn at redhat.com Wed Mar 27 10:04:53 2019 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 27 Mar 2019 10:04:53 +0000 Subject: 8219993: AArch64: Compiled CI stubs are unsafely modified In-Reply-To: <72275f80-303c-36c6-0a96-08977f808d29@redhat.com> References: <3abc4b04-a6d4-62f7-4cd9-36a89fe020e4@redhat.com> <2fa5fcb0-3bcc-f490-4948-405e23c10e40@redhat.com> <72275f80-303c-36c6-0a96-08977f808d29@redhat.com> Message-ID: On 27/03/2019 09:56, Andrew Haley wrote: > On 3/27/19 9:39 AM, Andrew Dinn wrote: >> On 26/03/2019 18:26, Andrew Haley wrote: >>> Here's a simpler patch. All I do this time is insert an ISB between the >>> call and the load instructions so that we don't have a stale instruction >>> buffer. >>> >>> I also took the opportunity to unify some code which was duplicated in >>> C1 and compiledIC_aarch64. >>> >>> Andrew Dinn, please approve this one. :-) >>> >>> http://cr.openjdk.java.net/~aph/8219993.2/ >> That looks ok modulo one detail I don't really understand which, I >> think, applies to both old and new code. >> >> The stub target address gets updated by NativeMovConstReg::set_data >> which calls pd_patch_instruction. When that is called in some patching >> thread how is it guaranteed that some other reading/executing thread >> does not observe partial writes of the movz+movk+movk sequence? > > It's only written once, when the c2i stub is first used. After that it > is only rewritten at a safepoint, and we assert this. It's written > synchronously by the first thread that encounters it, and there is a > lock to prevent races. Ok, looks good then. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Wed Mar 27 10:12:12 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Mar 2019 10:12:12 +0000 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <70d3f07c-855b-f356-d2cc-3061de40e572@oracle.com> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> <35d3331a-258a-3d75-0bd9-09c0072ebdd9@loongson.cn> <70d3f07c-855b-f356-d2cc-3061de40e572@oracle.com> Message-ID: On 27/03/2019 09:50, David Holmes wrote: > It's been pointed out to me that these "micro" "tests" are outside of > jtreg and can only be run via make. Once the build-microbenchmark target is built then you should have images/test/micro/benchmarks.jar in your build output directory. That's an executable JAR that includes the JMH so you can execute that directly and specify whatever parameters you need. -Alan From fujie at loongson.cn Wed Mar 27 10:19:02 2019 From: fujie at loongson.cn (Jie Fu) Date: Wed, 27 Mar 2019 18:19:02 +0800 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <2111ddfd-3171-5351-b3f4-4fd167599073@oracle.com> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <2111ddfd-3171-5351-b3f4-4fd167599073@oracle.com> Message-ID: <9fc0b1ec-4c5d-659e-ce76-35a8fe13ad4f@loongson.cn> Thanks Claes. On 2019/3/27 ??6:03, Claes Redestad wrote: > Hi Jie, > > generally I'd not recommend running all microbenchmarks in one go, i.e, > qualify micro:. Mostly from an > operational point of view as running them all (with default settings) > will take days. > > I'll have a look at these CallSite micros. I suspect we don't run them > regularly and they might have atrophied. > > /Claes > > On 2019-03-27 04:47, Jie Fu wrote: >> Hi all, >> >> Following benchmarks failed when I do 'make test TEST="micro"'. >> ---------------------------------------------- >> org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable >> org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testVolatile >> org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testMutable >> org.openjdk.bench.java.lang.invoke.CallSiteSetTargetSelf.testVolatile >> org.openjdk.bench.java.lang.invoke.CallSiteStable.testConstant >> org.openjdk.bench.java.lang.invoke.CallSiteStable.testMutable >> org.openjdk.bench.java.lang.invoke.CallSiteStable.testVolatile >> ---------------------------------------------- >> >> These failures can also be reproduced by a single run of each >> benchmark, e.g. >> ?? make test >> TEST="micro:org.openjdk.bench.java.lang.invoke.CallSiteSetTarget.testMutable" >> >> >> Questions: why there are so many shouldNotCallThis() in >> CallSiteSetTarget.java/CallSiteSetTargetSelf.java/CallSiteStable.java? >> ??????????? How can I run these benchmarks properly? >> >> Thanks a lot. >> >> >> Best regards, >> Jie >> >> From david.holmes at oracle.com Wed Mar 27 10:22:01 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Mar 2019 20:22:01 +1000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: On 27/03/2019 8:04 pm, Baesken, Matthias wrote: > Hi Thomas, interesting idea . David ? would you be fine with? what > Thomas proposes ? Isn't adding the missing fields to the existing os_perf code basically the same thing? I don't see any point in duplicating existing code like this. David ----- > Best regards, Matthias > > *From:*Thomas St?fe > *Sent:* Mittwoch, 27. M?rz 2019 10:28 > *To:* Baesken, Matthias > *Cc:* hotspot-dev at openjdk.java.net > *Subject:* Re: RFR: 8221535: add steal tick related information to > hs_error file [linux] > > Hi Matthias, > > this is a lot of coding for just one counter... > > Also os_perf_linux.cpp does something very similar. However, that code > does not get the steal ticks you want, does not seem to care about > kernel differences and could also be improved. > > Proposal: Why do you not tweak your new API to be a general purpose API > to read all ticks from /proc/stat. That would be way more useful. Later, > if we feel like it, could change os_perf_linux.cpp to use your function. > > We did something similar for the statistical history in the SapMachine, see > > https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linux/stathist_linux.cpp > > . So I would suggest to have a function reading proc stat and returning > all values in a struct. That struct should have provisions for "value > not found" since which values you get depends in kernel version. > > -------- > > Example: > > struct value_t { bool valid; uint64_t v; } > > struct cpu_values_t { > ?value_t user; > ?value_t nice; > ?value_t system; > ?value_t idle; > ?value_t iowait; > ?value_t steal; > ?value_t guest; > ?value_t guest_nice; > }; > > void parse_proc_stat_cpu_line(const char* line, cpu_values_t* out) { > ? // Note: existence of some of these values depends on kernel version > ? out->user.valid = out->nice.valid?= out->system.valid?= > out->idle.valid?= out->iowait.valid?= out->steal.valid?= > out->guest.valid?= out->guest_nice.valid?= false; > ? int user, nice, system, idle, iowait, irq, softirq, steal, guest, > guest_nice; > ? int num = ::sscanf(line, "cpu %d %d %d %d %d %d %d %d %d %d", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &user, &nice, &system, &idle, &iowait, > &irq, &softirq, &steal, &guest, &guest_nice); > ? if (num >= 4) { > ? ? out->user.v = user; out->user.valid = true; > ? ? out->nice.v?= nice; out->nice.valid = true; > ? ? out->system.v?= system; out->system.valid = true; > ? ? out->idle.v?= idle; out->idle.valid = true; > ? ? if (num >= 5) { // iowait (5) (since Linux 2.5.41) > ? ? ? out->iowait = iowait; out->iowait.valid = true; > ? ? ? if (num >= 8) { // steal (8) (since Linux 2.6.11) > ? ? ? ? out->steal = steal; out->steal.valid = true; > ? ? ? ? if (num >= 9) { // guest (9) (since Linux 2.6.24) > ? ? ? ? ? out->guest = guest; out->guest.valid = true; > ? ? ? ? ? if (num >= 10) { // guest (9) (since Linux 2.6.33) > ? ? ? ? ? ? out->guest_nice = guest_nice; out->guest_nice.valid = true; > ? ? ? ? ? } > ? ? ? } > ? ?} > } > > Cheers Thomas > > On Wed, Mar 27, 2019 at 9:42 AM Baesken, Matthias > > wrote: > > Hello, please review? the? following change . > > Recent Linux kernels provide in the /proc filesystem? ?so called > "steal" information, see the the proc man page : > > ? .. > "there is an eighth column, steal - stolen time, which is the time > spent in other operating systems when running in a virtualized > environment" > > The info was useful? in our? internal VM,? when looking into > performance issues? (especially in virtualized environments), so it > should be added to the hs_err file . > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8221535 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ > > > Best Regards, Matthias > From claes.redestad at oracle.com Wed Mar 27 10:25:28 2019 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 27 Mar 2019 11:25:28 +0100 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <2111ddfd-3171-5351-b3f4-4fd167599073@oracle.com> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <2111ddfd-3171-5351-b3f4-4fd167599073@oracle.com> Message-ID: <6bcf04fc-e94e-c883-81b5-a10f88a662bf@oracle.com> On 2019-03-27 11:03, Claes Redestad wrote: > I'll have a look at these CallSite micros. I suspect we don't run them > regularly and they might have atrophied. Seems these micros depend on a utility (Indify) that couldn't be ported over from jmh-jdk-microbenchmarks to run properly, and I missed these benchmarks were depending on that utility and failing with AssertionErrors in JEP 230 - mainly since make test logs this as a pass. I've filed https://bugs.openjdk.java.net/browse/JDK-8221543 to deal with this. /Claes From thomas.stuefe at gmail.com Wed Mar 27 10:46:59 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 27 Mar 2019 11:46:59 +0100 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: On Wed, Mar 27, 2019 at 11:22 AM David Holmes wrote: > On 27/03/2019 8:04 pm, Baesken, Matthias wrote: > > Hi Thomas, interesting idea . David ? would you be fine with what > > Thomas proposes ? > > Isn't adding the missing fields to the existing os_perf code basically > the same thing? I don't see any point in duplicating existing code like > this. > We are on one page here. But os_perf_linux.cpp, get_total_ticks() (which I assume you mean by pre-existing code) has a number of issues I can see: - It does not allow for values not found since the kernel is too old. See http://man7.org/linux/man-pages/man5/proc.5.html All values in the /proc/stat cpu line beyond "idle" (the fourth) are depending on kernel version. Matthias wants to read steal ticks, which have been introduced with 2.6.11. A clean API would recognize this and return all it can return instead of nothing (which is what I believe now happens). Of course, the API should be able to signal "is_valid" to the user. For a way to do this, see my proposal, resp. what we did in SapMachine: https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linux/stathist_linux.cpp#L106 - I believe also this means the calculated "totals" can be wrong as it is now: pticks->total = userTicks + niceTicks + systemTicks + idleTicks + iowTicks + irqTicks + sirqTicks; Pretty sure that on newer kernels, where you have numbers for "steal" and "guest", those ticks are substracted by the kernel from "user" or "system" and must be added manually to a total. I also would like to get all values, not just the two or three ticks, and for that, a structure would be better as sketched out. It also should then be removed from os_perf_linux.cpp to the "os::Linux" namespace and os_linux.cpp (since parsing /proc/stat is a useful thing and it would be good to consolidate that coding at a central place). Bottomline, I would have a central, low-level, clean and capable and platform-specific solution which can be reused without much fuss in Matthias code as well as in os_perf (I think the JFR uses this too). I really do not want code duplication, at least not long term. Cheers, Thomas You > > David > ----- > > > Best regards, Matthias > > > > *From:*Thomas St?fe > > *Sent:* Mittwoch, 27. M?rz 2019 10:28 > > *To:* Baesken, Matthias > > *Cc:* hotspot-dev at openjdk.java.net > > *Subject:* Re: RFR: 8221535: add steal tick related information to > > hs_error file [linux] > > > > Hi Matthias, > > > > this is a lot of coding for just one counter... > > > > Also os_perf_linux.cpp does something very similar. However, that code > > does not get the steal ticks you want, does not seem to care about > > kernel differences and could also be improved. > > > > Proposal: Why do you not tweak your new API to be a general purpose API > > to read all ticks from /proc/stat. That would be way more useful. Later, > > if we feel like it, could change os_perf_linux.cpp to use your function. > > > > We did something similar for the statistical history in the SapMachine, > see > > > > > https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linux/stathist_linux.cpp > > > > . So I would suggest to have a function reading proc stat and returning > > all values in a struct. That struct should have provisions for "value > > not found" since which values you get depends in kernel version. > > > > -------- > > > > Example: > > > > struct value_t { bool valid; uint64_t v; } > > > > struct cpu_values_t { > > value_t user; > > value_t nice; > > value_t system; > > value_t idle; > > value_t iowait; > > value_t steal; > > value_t guest; > > value_t guest_nice; > > }; > > > > void parse_proc_stat_cpu_line(const char* line, cpu_values_t* out) { > > // Note: existence of some of these values depends on kernel version > > out->user.valid = out->nice.valid = out->system.valid = > > out->idle.valid = out->iowait.valid = out->steal.valid = > > out->guest.valid = out->guest_nice.valid = false; > > int user, nice, system, idle, iowait, irq, softirq, steal, guest, > > guest_nice; > > int num = ::sscanf(line, "cpu %d %d %d %d %d %d %d %d %d %d", > > &user, &nice, &system, &idle, &iowait, > > &irq, &softirq, &steal, &guest, &guest_nice); > > if (num >= 4) { > > out->user.v = user; out->user.valid = true; > > out->nice.v = nice; out->nice.valid = true; > > out->system.v = system; out->system.valid = true; > > out->idle.v = idle; out->idle.valid = true; > > if (num >= 5) { // iowait (5) (since Linux 2.5.41) > > out->iowait = iowait; out->iowait.valid = true; > > if (num >= 8) { // steal (8) (since Linux 2.6.11) > > out->steal = steal; out->steal.valid = true; > > if (num >= 9) { // guest (9) (since Linux 2.6.24) > > out->guest = guest; out->guest.valid = true; > > if (num >= 10) { // guest (9) (since Linux 2.6.33) > > out->guest_nice = guest_nice; out->guest_nice.valid = true; > > } > > } > > } > > } > > > > Cheers Thomas > > > > On Wed, Mar 27, 2019 at 9:42 AM Baesken, Matthias > > > wrote: > > > > Hello, please review the following change . > > > > Recent Linux kernels provide in the /proc filesystem so called > > "steal" information, see the the proc man page : > > > > .. > > "there is an eighth column, steal - stolen time, which is the time > > spent in other operating systems when running in a virtualized > > environment" > > > > The info was useful in our internal VM, when looking into > > performance issues (especially in virtualized environments), so it > > should be added to the hs_err file . > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8221535 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ > > > > > > Best Regards, Matthias > > > From erik.osterlund at oracle.com Wed Mar 27 11:04:02 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 27 Mar 2019 12:04:02 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> Message-ID: <8e8443e4-1d6b-e61d-ac6d-d77fc09bd53b@oracle.com> Hi Robbin, In runtime/safepointMechanism.cpp: 110 OrderAccess::storeload(); This should be OrderAccess::fence() as it must also act as a loadload() between the local poll and the global poll; if they reorder, we could miss a safepoint request that we disarmed, and forget to arm again, causing potentially causing a deadlock in the whole system. The only thing interleaving the two now is a storeload and a release (comprising loadstore and storestore), in other words no loadload. Naturally, all known storeload fences are also loadload fences, so it will work anyway, but it is nevertheless good for the semantics to be explicit about it. Otherwise, it looks good. Don't need another webrev. Thanks for sorting this out. Thanks, /Erik On 2019-03-26 13:51, Robbin Ehn wrote: > Hi all, please review. > > This is v4, which is just an rebase on-top of "RFR: 8221207: Redo > JDK-8218446 - > SuspendAtExit hangs". The commit simplified S/R code a bit, making this > patch > simpler. > > Full: > http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ > Inc (not complete since v3 did not apply proper, one of the reject wasn't > needed): > http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ > > To summarize, before 8221207 we had 4 safe states: > *Off threads_list (_thread_new) > *Blocked????????? (_thread_blocked) > *In native??????? (_thread_in_native) > *Suspend????????? (*ANY*) > *After handshake? (X) > > After 8221207 we have: > *Off threads_list (_thread_new) (far from cross-mod code) > *Blocked????????? (_thread_blocked) > *In native??????? (_thread_in_native) > *After handshake? (X) > > -We have one instruction barrier for when going from _thread_new. > -We have four instruction barrier for when going from blocked: > ????- ThreadBlockInVM > ????- ThreadBlockInVMWithDeadlockCheck > ????- The manual blocked transition for suspended blocked case. > ????- Blocked in SS:block() uses the barrier in block_if_requested_slow. > -We let threads doing their own handshake hit the barrier in > block_if_requested_slow. > -We let threads in native stay armed so they hit the barrier in > block_if_requested_slow. > > Not disarming threads in native works because, false positives already > happens > today and are handled. Both handshakes and safepoints disarms in correct > order, > so a JavaThread can notice that is should be armed, but with false > positives. > Before first check of handshake/safepoint the JavaThread is in a unsafe > state. > After that check a new handshake/safepoint cannot start/progress since the > JavaThread is unsafe. Thus re-arming and continue to next polling site > is okay. > > Patch only adds barrier for cross-modifying code that happens during a > handshake/safepoint and implements this barrier for those platforms > which have a > simple implementaion for accomplish it. sparc/arm32/aarch64 needs > additional > work. > > The deprecated global poll path does not contain proper barriers, and > will be > obsoleted in JDK 14. > > Thanks, Robbin > > On 2019-03-08 16:24, Robbin Ehn wrote: >> Hi all, please review. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220351 >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >> >> After a JavaThread have been in a safepoint/(handshake) safe state it >> can start >> executing updated code. E.g. an oop in the instruction stream can have >> been >> updated. >> >> Most hardware's require a barrier or that the code cross modified is >> far away to >> guarantee that the thread executing the updated instruction stream >> sees the >> modification. >> >> What far away is and how far an update instruction stream is from a >> safepoint >> safe state is not clear. >> >> To be compliant with those architectures an instruction stream barrier >> must be >> added when leaving the safepoint safe state. >> >> There may be crashes today due to this missing barrier. >> A new CPU with deeper pipeline or changes to the VM which moves a >> safepoint safe >> state closer to a nmethod can increase changes of a crash. >> >> A few benchmarks will see a performance regression with the extra >> serializing, >> such as Octane-Crypto -5% (x86). >> >> With a much more elaborate fix, such as changing the local poll to >> have more >> than two states (armed/disarmed), it would be possible to only emit such >> instruction stream barrier when the poll have been armed while the >> thread was >> safepoint safe. >> >> Passes t1-3 on our platforms, and test built what I can. >> >> Thanks, Robbin From robbin.ehn at oracle.com Wed Mar 27 12:02:34 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 27 Mar 2019 13:02:34 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <8e8443e4-1d6b-e61d-ac6d-d77fc09bd53b@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> <8e8443e4-1d6b-e61d-ac6d-d77fc09bd53b@oracle.com> Message-ID: Hi Erik, On 3/27/19 12:04 PM, Erik ?sterlund wrote: > Hi Robbin, > > In runtime/safepointMechanism.cpp: > 110???? OrderAccess::storeload(); > > This should be OrderAccess::fence() as it must also act as a loadload() between > the local poll and the global poll; if they reorder, we could miss a safepoint > request that we disarmed, and forget to arm again, causing potentially causing a > deadlock in the whole system. The only thing interleaving the two now is a > storeload and a release (comprising loadstore and storestore), in other words no > loadload. Naturally, all known storeload fences are also loadload fences, so it > will work anyway, but it is nevertheless good for the semantics to be explicit > about it. A question, in this case we have: A: load poll loadstore|storestore (release) B: store poll storeload C: load state So B is ordered after A and C is ordered after B. Which I thought implied that C also was ordered after A ? That is the reason for having a storeload and not a fence. > > Otherwise, it looks good. Don't need another webrev. Thanks for sorting this out. Thanks! /Robbin > > Thanks, > /Erik > > On 2019-03-26 13:51, Robbin Ehn wrote: >> Hi all, please review. >> >> This is v4, which is just an rebase on-top of "RFR: 8221207: Redo JDK-8218446 - >> SuspendAtExit hangs". The commit simplified S/R code a bit, making this patch >> simpler. >> >> Full: >> http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ >> Inc (not complete since v3 did not apply proper, one of the reject wasn't >> needed): >> http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ >> >> To summarize, before 8221207 we had 4 safe states: >> *Off threads_list (_thread_new) >> *Blocked????????? (_thread_blocked) >> *In native??????? (_thread_in_native) >> *Suspend????????? (*ANY*) >> *After handshake? (X) >> >> After 8221207 we have: >> *Off threads_list (_thread_new) (far from cross-mod code) >> *Blocked????????? (_thread_blocked) >> *In native??????? (_thread_in_native) >> *After handshake? (X) >> >> -We have one instruction barrier for when going from _thread_new. >> -We have four instruction barrier for when going from blocked: >> ?????- ThreadBlockInVM >> ?????- ThreadBlockInVMWithDeadlockCheck >> ?????- The manual blocked transition for suspended blocked case. >> ?????- Blocked in SS:block() uses the barrier in block_if_requested_slow. >> -We let threads doing their own handshake hit the barrier in >> block_if_requested_slow. >> -We let threads in native stay armed so they hit the barrier in >> block_if_requested_slow. >> >> Not disarming threads in native works because, false positives already happens >> today and are handled. Both handshakes and safepoints disarms in correct order, >> so a JavaThread can notice that is should be armed, but with false positives. >> Before first check of handshake/safepoint the JavaThread is in a unsafe state. >> After that check a new handshake/safepoint cannot start/progress since the >> JavaThread is unsafe. Thus re-arming and continue to next polling site is okay. >> >> Patch only adds barrier for cross-modifying code that happens during a >> handshake/safepoint and implements this barrier for those platforms which have a >> simple implementaion for accomplish it. sparc/arm32/aarch64 needs additional >> work. >> >> The deprecated global poll path does not contain proper barriers, and will be >> obsoleted in JDK 14. >> >> Thanks, Robbin >> >> On 2019-03-08 16:24, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> After a JavaThread have been in a safepoint/(handshake) safe state it can start >>> executing updated code. E.g. an oop in the instruction stream can have been >>> updated. >>> >>> Most hardware's require a barrier or that the code cross modified is far away to >>> guarantee that the thread executing the updated instruction stream sees the >>> modification. >>> >>> What far away is and how far an update instruction stream is from a safepoint >>> safe state is not clear. >>> >>> To be compliant with those architectures an instruction stream barrier must be >>> added when leaving the safepoint safe state. >>> >>> There may be crashes today due to this missing barrier. >>> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >>> state closer to a nmethod can increase changes of a crash. >>> >>> A few benchmarks will see a performance regression with the extra serializing, >>> such as Octane-Crypto -5% (x86). >>> >>> With a much more elaborate fix, such as changing the local poll to have more >>> than two states (armed/disarmed), it would be possible to only emit such >>> instruction stream barrier when the poll have been armed while the thread was >>> safepoint safe. >>> >>> Passes t1-3 on our platforms, and test built what I can. >>> >>> Thanks, Robbin From coleen.phillimore at oracle.com Wed Mar 27 13:09:42 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 27 Mar 2019 09:09:42 -0400 Subject: RFR (M) 8221183: Avoid code cache walk in MetadataOnStackMark Message-ID: Summary: Note nmethods with "old" Methods in them in table to walk instead. See RFE for more details.? Tested with RedefineClasses tests with/without -Xcomp, mach5 tier1-5. open webrev at http://cr.openjdk.java.net/~coleenp/2019/8221183.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8221183 Thanks, Coleen From robbin.ehn at oracle.com Wed Mar 27 13:16:08 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 27 Mar 2019 14:16:08 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <003054ae-76a9-ef83-82a8-35efbbc7a30d@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> <003054ae-76a9-ef83-82a8-35efbbc7a30d@oracle.com> Message-ID: Hi David, > > I finally realized how this translates into covering all the return-to-Java > paths, without having as many unnecessary calls to cross_modify_barrier. If we > return to Java after a safepoint (or action that could have installed modified > code) we need the barrier. If a safepoint has occurred then we must have become > safepoint-safe at some point, which means we either were in _thread_blocked (by > one means or another) or _thread_in_native. So we guard the transitions out of > those states. > Yes, great! > > I suggest creating subtasks for those platforms so this additional work isn't > forgotten. Fixed. > >> The deprecated global poll path does not contain proper barriers, and will be >> obsoleted in JDK 14. > > Ok. > > A couple of minor typos: > > src/hotspot/share/runtime/safepointMechanism.inline.hpp > > +???? // JavaThread will disarm it self? ... > > s/it self/itself/ > > --- > src/hotspot/share/runtime/thread.cpp > > +?? // from here is probably far enought, ... > > s/enought/enough/ > > +?? // Since we are not using a regular thread-state tranisition helper here, > > s/tranisition/transition/ Fixed, thanks! /Robbin > > Thanks, > David > ----- > >> Thanks, Robbin >> >> On 2019-03-08 16:24, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>> Changeset: >>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>> >>> After a JavaThread have been in a safepoint/(handshake) safe state it can start >>> executing updated code. E.g. an oop in the instruction stream can have been >>> updated. >>> >>> Most hardware's require a barrier or that the code cross modified is far away to >>> guarantee that the thread executing the updated instruction stream sees the >>> modification. >>> >>> What far away is and how far an update instruction stream is from a safepoint >>> safe state is not clear. >>> >>> To be compliant with those architectures an instruction stream barrier must be >>> added when leaving the safepoint safe state. >>> >>> There may be crashes today due to this missing barrier. >>> A new CPU with deeper pipeline or changes to the VM which moves a safepoint safe >>> state closer to a nmethod can increase changes of a crash. >>> >>> A few benchmarks will see a performance regression with the extra serializing, >>> such as Octane-Crypto -5% (x86). >>> >>> With a much more elaborate fix, such as changing the local poll to have more >>> than two states (armed/disarmed), it would be possible to only emit such >>> instruction stream barrier when the poll have been armed while the thread was >>> safepoint safe. >>> >>> Passes t1-3 on our platforms, and test built what I can. >>> >>> Thanks, Robbin From thomas.schatzl at oracle.com Wed Mar 27 13:57:16 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 27 Mar 2019 14:57:16 +0100 Subject: RFC: Draft JEP: Improved Metaspace allocator In-Reply-To: References: Message-ID: On Wed, 2019-03-20 at 20:19 +0100, Thomas St?fe wrote: > Dear all, > > I would like to hear what you think about the following proposal: > > https://bugs.openjdk.java.net/browse/JDK-8221173 > > Do you consider this worthwhile? Yes, definitely! At the beginning of this month we were internally looking at a metaspace stress application, and it took some time to nail the problem on metaspace fragmentation. Improving fragmentation, the visibility of such problems (fragmentation statistics) and potential options to e.g. favor compactness vs. performance. > I debated with myself whether a JEP is the correct umbrella for it. > In the past I did larger changes like this without JEP but in this > case I feel better with a confirmed and sponsored JEP behind me - > from a certain size on it just feels more appropriate. > > As usual, I am not sure where to put Metaspace-related proposals to > catch all interested people, so I start with hs-dev. Please feel free > to redirect if needed. You are right that a JEP is the only way to get guaranteed visibility to people. JEPs are mostly the only source for third party "what's new in JDK XX" write-ups. So if you want to reach a wide (non-hotspot-dev) audience, JEPs are basically the only way. I could kind of see that this might be the case. Thanks, Thomas From erik.osterlund at oracle.com Wed Mar 27 14:04:53 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 27 Mar 2019 15:04:53 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> <8e8443e4-1d6b-e61d-ac6d-d77fc09bd53b@oracle.com> Message-ID: <0969b368-9914-6b8e-12ea-c901e97db0da@oracle.com> Hi Robbin, On 2019-03-27 13:02, Robbin Ehn wrote: > Hi Erik, > > On 3/27/19 12:04 PM, Erik ?sterlund wrote: >> Hi Robbin, >> >> In runtime/safepointMechanism.cpp: >> 110???? OrderAccess::storeload(); >> >> This should be OrderAccess::fence() as it must also act as a >> loadload() between the local poll and the global poll; if they >> reorder, we could miss a safepoint request that we disarmed, and >> forget to arm again, causing potentially causing a deadlock in the >> whole system. The only thing interleaving the two now is a storeload >> and a release (comprising loadstore and storestore), in other words no >> loadload. Naturally, all known storeload fences are also loadload >> fences, so it will work anyway, but it is nevertheless good for the >> semantics to be explicit about it. > > A question, in this case we have: > A: load poll > loadstore|storestore (release) > B: store poll > storeload > C: load state > > So B is ordered after A and C is ordered after B. > Which I thought implied that C also was ordered after A ? > > That is the reason for having a storeload and not a fence. We have not defined transitivity between chains of {load|store}{load|store} style fenced access pairs. Our current spec only defines access class pairs, and how they interact. In fact, in general, if you have N threads running sequences of loads and stores, and interleaved all access pairs with the corresponding {load|store}{load|store} fence for the immediately neighbouring accesses, you would not end up with a total ordering of all accesses in the system (in which you could reason about transitive happens-before ordering properties). However, in practice, given our current bindings, your code here *will* be correctly ordered on all of our platforms, as your loads only race with stores of one other thread, and storeload implies loadload in our bindings. But it's not really defined in our memory model. I noticed though that the local poll seemingly unconditionally has acquire semantics nowadays. A bit of a mismatch I suppose given that the VM-thread store-pair it races with is interleaved by storestore(). But anyway, that acquire should imply the loadload you need and I thought was missing. So I guess we are good. Perhaps the mixing of {load|store}{load|store}-style fences racing with acquire/release style fences ought to be cleaned up a bit. Some other day maybe. ;) Looks good. Thanks, /Erik >> >> Otherwise, it looks good. Don't need another webrev. Thanks for >> sorting this out. > > Thanks! > > /Robbin > >> >> Thanks, >> /Erik >> >> On 2019-03-26 13:51, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> This is v4, which is just an rebase on-top of "RFR: 8221207: Redo >>> JDK-8218446 - >>> SuspendAtExit hangs". The commit simplified S/R code a bit, making >>> this patch >>> simpler. >>> >>> Full: >>> http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ >>> Inc (not complete since v3 did not apply proper, one of the reject >>> wasn't >>> needed): >>> http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ >>> >>> To summarize, before 8221207 we had 4 safe states: >>> *Off threads_list (_thread_new) >>> *Blocked????????? (_thread_blocked) >>> *In native??????? (_thread_in_native) >>> *Suspend????????? (*ANY*) >>> *After handshake? (X) >>> >>> After 8221207 we have: >>> *Off threads_list (_thread_new) (far from cross-mod code) >>> *Blocked????????? (_thread_blocked) >>> *In native??????? (_thread_in_native) >>> *After handshake? (X) >>> >>> -We have one instruction barrier for when going from _thread_new. >>> -We have four instruction barrier for when going from blocked: >>> ?????- ThreadBlockInVM >>> ?????- ThreadBlockInVMWithDeadlockCheck >>> ?????- The manual blocked transition for suspended blocked case. >>> ?????- Blocked in SS:block() uses the barrier in >>> block_if_requested_slow. >>> -We let threads doing their own handshake hit the barrier in >>> block_if_requested_slow. >>> -We let threads in native stay armed so they hit the barrier in >>> block_if_requested_slow. >>> >>> Not disarming threads in native works because, false positives >>> already happens >>> today and are handled. Both handshakes and safepoints disarms in >>> correct order, >>> so a JavaThread can notice that is should be armed, but with false >>> positives. >>> Before first check of handshake/safepoint the JavaThread is in a >>> unsafe state. >>> After that check a new handshake/safepoint cannot start/progress >>> since the >>> JavaThread is unsafe. Thus re-arming and continue to next polling >>> site is okay. >>> >>> Patch only adds barrier for cross-modifying code that happens during a >>> handshake/safepoint and implements this barrier for those platforms >>> which have a >>> simple implementaion for accomplish it. sparc/arm32/aarch64 needs >>> additional >>> work. >>> >>> The deprecated global poll path does not contain proper barriers, and >>> will be >>> obsoleted in JDK 14. >>> >>> Thanks, Robbin >>> >>> On 2019-03-08 16:24, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>> Changeset: >>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>> >>>> After a JavaThread have been in a safepoint/(handshake) safe state >>>> it can start >>>> executing updated code. E.g. an oop in the instruction stream can >>>> have been >>>> updated. >>>> >>>> Most hardware's require a barrier or that the code cross modified is >>>> far away to >>>> guarantee that the thread executing the updated instruction stream >>>> sees the >>>> modification. >>>> >>>> What far away is and how far an update instruction stream is from a >>>> safepoint >>>> safe state is not clear. >>>> >>>> To be compliant with those architectures an instruction stream >>>> barrier must be >>>> added when leaving the safepoint safe state. >>>> >>>> There may be crashes today due to this missing barrier. >>>> A new CPU with deeper pipeline or changes to the VM which moves a >>>> safepoint safe >>>> state closer to a nmethod can increase changes of a crash. >>>> >>>> A few benchmarks will see a performance regression with the extra >>>> serializing, >>>> such as Octane-Crypto -5% (x86). >>>> >>>> With a much more elaborate fix, such as changing the local poll to >>>> have more >>>> than two states (armed/disarmed), it would be possible to only emit >>>> such >>>> instruction stream barrier when the poll have been armed while the >>>> thread was >>>> safepoint safe. >>>> >>>> Passes t1-3 on our platforms, and test built what I can. >>>> >>>> Thanks, Robbin From robbin.ehn at oracle.com Wed Mar 27 14:45:27 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 27 Mar 2019 15:45:27 +0100 Subject: RFR(m): 8220351: Cross-modifying code In-Reply-To: <0969b368-9914-6b8e-12ea-c901e97db0da@oracle.com> References: <4021e6fe-a2e7-7e66-bd54-1ea9d80863ae@oracle.com> <610b37ca-da47-991b-1e7d-aba2ecf6acc9@oracle.com> <8e8443e4-1d6b-e61d-ac6d-d77fc09bd53b@oracle.com> <0969b368-9914-6b8e-12ea-c901e97db0da@oracle.com> Message-ID: Hi Erik, > I noticed though that the local poll seemingly unconditionally has acquire > semantics nowadays. A bit of a mismatch I suppose given that the VM-thread > store-pair it races with is interleaved by storestore(). But anyway, that > acquire should imply the loadload you need and I thought was missing. So I guess > we are good. Perhaps the mixing of {load|store}{load|store}-style fences racing > with acquire/release style fences ought to be cleaned up a bit. Some other day > maybe. ;) Yes! Gah, I also missed the acquire on get_polling_page, not so intuitive. > > Looks good. Thanks, Robbin > > Thanks, > /Erik > >>> >>> Otherwise, it looks good. Don't need another webrev. Thanks for sorting this >>> out. >> >> Thanks! >> >> /Robbin >> >>> >>> Thanks, >>> /Erik >>> >>> On 2019-03-26 13:51, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> This is v4, which is just an rebase on-top of "RFR: 8221207: Redo JDK-8218446 - >>>> SuspendAtExit hangs". The commit simplified S/R code a bit, making this patch >>>> simpler. >>>> >>>> Full: >>>> http://cr.openjdk.java.net/~rehn/8220351/v4/webrev/ >>>> Inc (not complete since v3 did not apply proper, one of the reject wasn't >>>> needed): >>>> http://cr.openjdk.java.net/~rehn/8220351/v4/inc/webrev/ >>>> >>>> To summarize, before 8221207 we had 4 safe states: >>>> *Off threads_list (_thread_new) >>>> *Blocked????????? (_thread_blocked) >>>> *In native??????? (_thread_in_native) >>>> *Suspend????????? (*ANY*) >>>> *After handshake? (X) >>>> >>>> After 8221207 we have: >>>> *Off threads_list (_thread_new) (far from cross-mod code) >>>> *Blocked????????? (_thread_blocked) >>>> *In native??????? (_thread_in_native) >>>> *After handshake? (X) >>>> >>>> -We have one instruction barrier for when going from _thread_new. >>>> -We have four instruction barrier for when going from blocked: >>>> ?????- ThreadBlockInVM >>>> ?????- ThreadBlockInVMWithDeadlockCheck >>>> ?????- The manual blocked transition for suspended blocked case. >>>> ?????- Blocked in SS:block() uses the barrier in block_if_requested_slow. >>>> -We let threads doing their own handshake hit the barrier in >>>> block_if_requested_slow. >>>> -We let threads in native stay armed so they hit the barrier in >>>> block_if_requested_slow. >>>> >>>> Not disarming threads in native works because, false positives already happens >>>> today and are handled. Both handshakes and safepoints disarms in correct order, >>>> so a JavaThread can notice that is should be armed, but with false positives. >>>> Before first check of handshake/safepoint the JavaThread is in a unsafe state. >>>> After that check a new handshake/safepoint cannot start/progress since the >>>> JavaThread is unsafe. Thus re-arming and continue to next polling site is okay. >>>> >>>> Patch only adds barrier for cross-modifying code that happens during a >>>> handshake/safepoint and implements this barrier for those platforms which >>>> have a >>>> simple implementaion for accomplish it. sparc/arm32/aarch64 needs additional >>>> work. >>>> >>>> The deprecated global poll path does not contain proper barriers, and will be >>>> obsoleted in JDK 14. >>>> >>>> Thanks, Robbin >>>> >>>> On 2019-03-08 16:24, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8220351 >>>>> Changeset: >>>>> http://cr.openjdk.java.net/~rehn/8220351/webrev/ >>>>> >>>>> After a JavaThread have been in a safepoint/(handshake) safe state it can >>>>> start >>>>> executing updated code. E.g. an oop in the instruction stream can have been >>>>> updated. >>>>> >>>>> Most hardware's require a barrier or that the code cross modified is far >>>>> away to >>>>> guarantee that the thread executing the updated instruction stream sees the >>>>> modification. >>>>> >>>>> What far away is and how far an update instruction stream is from a safepoint >>>>> safe state is not clear. >>>>> >>>>> To be compliant with those architectures an instruction stream barrier must be >>>>> added when leaving the safepoint safe state. >>>>> >>>>> There may be crashes today due to this missing barrier. >>>>> A new CPU with deeper pipeline or changes to the VM which moves a safepoint >>>>> safe >>>>> state closer to a nmethod can increase changes of a crash. >>>>> >>>>> A few benchmarks will see a performance regression with the extra serializing, >>>>> such as Octane-Crypto -5% (x86). >>>>> >>>>> With a much more elaborate fix, such as changing the local poll to have more >>>>> than two states (armed/disarmed), it would be possible to only emit such >>>>> instruction stream barrier when the poll have been armed while the thread was >>>>> safepoint safe. >>>>> >>>>> Passes t1-3 on our platforms, and test built what I can. >>>>> >>>>> Thanks, Robbin From matthias.baesken at sap.com Wed Mar 27 15:10:52 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 27 Mar 2019 15:10:52 +0000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: * All values in the /proc/stat cpu line beyond "idle" (the fourth) are depending on kernel version. Hello, I think we have to worry only about ?steal? (kernel 2.6.11) and maybe (if we want to add this ) ?guest? (kernel 2.6.24). The 3 others beyond idle came with 2.6 or even earlier : ?In Linux 2.6 this line includes three additional columns: iowait - time waiting for I/O to complete (since 2.5.41); irq - time servicing interrupts (since 2.6.0-test4); softirq - time servicing softirqs (since 2.6.0-test4).? Even the SUSE Linux 9 and Redhat 5 machines I checked have such ?new? (2.6+) kernels . Most likely we cannot run for some time on such old systems . Best regards, Matthias From: Thomas St?fe Sent: Mittwoch, 27. M?rz 2019 11:47 To: David Holmes Cc: Baesken, Matthias ; hotspot-dev at openjdk.java.net Subject: Re: RFR: 8221535: add steal tick related information to hs_error file [linux] On Wed, Mar 27, 2019 at 11:22 AM David Holmes > wrote: On 27/03/2019 8:04 pm, Baesken, Matthias wrote: > Hi Thomas, interesting idea . David ? would you be fine with what > Thomas proposes ? Isn't adding the missing fields to the existing os_perf code basically the same thing? I don't see any point in duplicating existing code like this. We are on one page here. But os_perf_linux.cpp, get_total_ticks() (which I assume you mean by pre-existing code) has a number of issues I can see: - It does not allow for values not found since the kernel is too old. See http://man7.org/linux/man-pages/man5/proc.5.html All values in the /proc/stat cpu line beyond "idle" (the fourth) are depending on kernel version. Matthias wants to read steal ticks, which have been introduced with 2.6.11. A clean API would recognize this and return all it can return instead of nothing (which is what I believe now happens). Of course, the API should be able to signal "is_valid" to the user. For a way to do this, see my proposal, resp. what we did in SapMachine: https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linux/stathist_linux.cpp#L106 - I believe also this means the calculated "totals" can be wrong as it is now: pticks->total = userTicks + niceTicks + systemTicks + idleTicks + iowTicks + irqTicks + sirqTicks; Pretty sure that on newer kernels, where you have numbers for "steal" and "guest", those ticks are substracted by the kernel from "user" or "system" and must be added manually to a total. I also would like to get all values, not just the two or three ticks, and for that, a structure would be better as sketched out. It also should then be removed from os_perf_linux.cpp to the "os::Linux" namespace and os_linux.cpp (since parsing /proc/stat is a useful thing and it would be good to consolidate that coding at a central place). Bottomline, I would have a central, low-level, clean and capable and platform-specific solution which can be reused without much fuss in Matthias code as well as in os_perf (I think the JFR uses this too). I really do not want code duplication, at least not long term. Cheers, Thomas You David ----- > Best regards, Matthias > > *From:*Thomas St?fe > > *Sent:* Mittwoch, 27. M?rz 2019 10:28 > *To:* Baesken, Matthias > > *Cc:* hotspot-dev at openjdk.java.net > *Subject:* Re: RFR: 8221535: add steal tick related information to > hs_error file [linux] > > Hi Matthias, > > this is a lot of coding for just one counter... > > Also os_perf_linux.cpp does something very similar. However, that code > does not get the steal ticks you want, does not seem to care about > kernel differences and could also be improved. > > Proposal: Why do you not tweak your new API to be a general purpose API > to read all ticks from /proc/stat. That would be way more useful. Later, > if we feel like it, could change os_perf_linux.cpp to use your function. > > We did something similar for the statistical history in the SapMachine, see > > https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linux/stathist_linux.cpp > > . So I would suggest to have a function reading proc stat and returning > all values in a struct. That struct should have provisions for "value > not found" since which values you get depends in kernel version. > > -------- > > Example: > > struct value_t { bool valid; uint64_t v; } > > struct cpu_values_t { > value_t user; > value_t nice; > value_t system; > value_t idle; > value_t iowait; > value_t steal; > value_t guest; > value_t guest_nice; > }; > > void parse_proc_stat_cpu_line(const char* line, cpu_values_t* out) { > // Note: existence of some of these values depends on kernel version > out->user.valid = out->nice.valid = out->system.valid = > out->idle.valid = out->iowait.valid = out->steal.valid = > out->guest.valid = out->guest_nice.valid = false; > int user, nice, system, idle, iowait, irq, softirq, steal, guest, > guest_nice; > int num = ::sscanf(line, "cpu %d %d %d %d %d %d %d %d %d %d", > &user, &nice, &system, &idle, &iowait, > &irq, &softirq, &steal, &guest, &guest_nice); > if (num >= 4) { > out->user.v = user; out->user.valid = true; > out->nice.v = nice; out->nice.valid = true; > out->system.v = system; out->system.valid = true; > out->idle.v = idle; out->idle.valid = true; > if (num >= 5) { // iowait (5) (since Linux 2.5.41) > out->iowait = iowait; out->iowait.valid = true; > if (num >= 8) { // steal (8) (since Linux 2.6.11) > out->steal = steal; out->steal.valid = true; > if (num >= 9) { // guest (9) (since Linux 2.6.24) > out->guest = guest; out->guest.valid = true; > if (num >= 10) { // guest (9) (since Linux 2.6.33) > out->guest_nice = guest_nice; out->guest_nice.valid = true; > } > } > } > } > > Cheers Thomas > > On Wed, Mar 27, 2019 at 9:42 AM Baesken, Matthias > >> wrote: > > Hello, please review the following change . > > Recent Linux kernels provide in the /proc filesystem so called > "steal" information, see the the proc man page : > > .. > "there is an eighth column, steal - stolen time, which is the time > spent in other operating systems when running in a virtualized > environment" > > The info was useful in our internal VM, when looking into > performance issues (especially in virtualized environments), so it > should be added to the hs_err file . > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8221535 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ > > > Best Regards, Matthias > From thomas.stuefe at gmail.com Wed Mar 27 15:29:26 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 27 Mar 2019 16:29:26 +0100 Subject: RFC: Draft JEP: Improved Metaspace allocator In-Reply-To: References: Message-ID: Hi Thomas, On Wed, Mar 27, 2019 at 2:57 PM Thomas Schatzl wrote: > On Wed, 2019-03-20 at 20:19 +0100, Thomas St?fe wrote: > > Dear all, > > > > I would like to hear what you think about the following proposal: > > > > https://bugs.openjdk.java.net/browse/JDK-8221173 > > > > Do you consider this worthwhile? > > Yes, definitely! > Great :) > > At the beginning of this month we were internally looking at a > metaspace stress application, and it took some time to nail the problem > on metaspace fragmentation. Improving fragmentation, the visibility of > such problems (fragmentation statistics) and potential options to e.g. > favor compactness vs. performance. > Just to be sure, are you aware of the jcmd "VM.metaspace" command we (RedHat & SAP) added for jdk11? It can be used to analyze Metaspace in quite some detail. We did not make a big fuss over it, but it is there and quite useful. It may of course be missing functionality; if yes, please tell us. About your stress test, I'd be interested in a reproduction case if that is possible. I am always collecting pathological cases to play with. > > > I debated with myself whether a JEP is the correct umbrella for it. > > In the past I did larger changes like this without JEP but in this > > case I feel better with a confirmed and sponsored JEP behind me - > > from a certain size on it just feels more appropriate. > > > > As usual, I am not sure where to put Metaspace-related proposals to > > catch all interested people, so I start with hs-dev. Please feel free > > to redirect if needed. > > You are right that a JEP is the only way to get guaranteed visibility > to people. JEPs are mostly the only source for third party "what's new > in JDK XX" write-ups. > So if you want to reach a wide (non-hotspot-dev) audience, JEPs are > basically the only way. I could kind of see that this might be the > case. > For me it is more of a question of "I don't want to make a fuss" vs "I don't want to sneak large changes in behind everyone's back". In this case, I side with the latter. As additional advantage, a JEP gives a notion of progress to the outside world. And I have the feeling it activates more Reviewer muscle than large-non-JEP patches. Thanks, Thomas > Thanks, > Thomas > > > From thomas.schatzl at oracle.com Wed Mar 27 16:01:57 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 27 Mar 2019 17:01:57 +0100 Subject: RFC: Draft JEP: Improved Metaspace allocator In-Reply-To: References: Message-ID: <1c64745ad2425ff4d1125344e1076568d38401b6.camel@oracle.com> Hi, On Wed, 2019-03-27 at 16:29 +0100, Thomas St?fe wrote: > Hi Thomas, > > On Wed, Mar 27, 2019 at 2:57 PM Thomas Schatzl < > thomas.schatzl at oracle.com> wrote: > > On Wed, 2019-03-20 at 20:19 +0100, Thomas St?fe wrote: > > > Dear all, > > > > > > I would like to hear what you think about the following proposal: > > > > > > https://bugs.openjdk.java.net/browse/JDK-8221173 > > > > > > Do you consider this worthwhile? > > > > Yes, definitely! > > Great :) > > > At the beginning of this month we were internally looking at a > > metaspace stress application, and it took some time to nail the > > problem on metaspace fragmentation. Improving fragmentation, the > > visibility of such problems (fragmentation statistics) and > > potential options to e.g. favor compactness vs. performance. > > Just to be sure, are you aware of the jcmd "VM.metaspace" command we > (RedHat & SAP) added for jdk11? No, at least I did not. At least not at the time when it would have been very useful :( > It can be used to analyze Metaspace in quite some detail. We did not > make a big fuss over it, but it is there and quite useful. Which is obviously the problem. > It may of course be missing functionality; if yes, please tell us. I will forward this information. I think we may get some time to do some more runs and look over the output. > About your stress test, I'd be interested in a reproduction case if > that is possible. I am always collecting pathological cases to play > with. I will bring that up too. > > > I debated with myself whether a JEP is the correct umbrella for > > > it. In the past I did larger changes like this without JEP but in > > > this case I feel better with a confirmed and sponsored JEP > > > behind me - from a certain size on it just feels more > > > appropriate. > > > > > > As usual, I am not sure where to put Metaspace-related proposals > > > to catch all interested people, so I start with hs-dev. Please > > > feel free to redirect if needed. > > > > You are right that a JEP is the only way to get guaranteed > > visibility to people. JEPs are mostly the only source for third > > party "what's new in JDK XX" write-ups. > > So if you want to reach a wide (non-hotspot-dev) audience, JEPs are > > basically the only way. I could kind of see that this might be the > > case. > > For me it is more of a question of "I don't want to make a fuss" vs > "I don't want to sneak large changes in behind everyone's back". In > this case, I side with the latter. > > As additional advantage, a JEP gives a notion of progress to the > outside world. And I have the feeling it activates more Reviewer > muscle than large-non-JEP patches. :) Thanks, Thomas From vladimir.kozlov at oracle.com Wed Mar 27 16:35:38 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 27 Mar 2019 09:35:38 -0700 Subject: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? In-Reply-To: References: Message-ID: <58dcefc5-4bdf-9994-f0b6-484d627d5cfa@oracle.com> Changes looks good. It was part of debug functionality in C2 which was removed in jdk7u but adlc part was not: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/a5dd6e3ef9f3#l6.7 Thanks, Vladimir On 3/27/19 2:29 AM, Baesken, Matthias wrote: > Hi Thomas, good point . > I removed the 2 functions . > Btw. I checked them (using nm ) in the product libjvm.so and they showed up in the binary . > So it might help also to get the binary a little bit smaller ? > > > Please review ! > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8221541 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221541.0/ > > > Thanks, Matthias > > > From: Thomas St?fe > Sent: Dienstag, 26. M?rz 2019 20:40 > To: Baesken, Matthias > Cc: hotspot-dev at openjdk.java.net > Subject: Re: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? > > Hi Matthias, > > On Tue, Mar 26, 2019 at 4:17 PM Baesken, Matthias > wrote: > > Hello, it seems that the fopen calls in CompilerOracle::append_comment_to_file and > CompilerOracle::append_exclude_to_file do not have a corresponding fclose. > Or do I miss something ? > > No, you are right, this is a bug. However, I think those functions are nowhere used, so maybe they could be removed completely. > > Cheers, Thomas > > We could also add a "true" for the second argument of the fileStream constructor calls ( fileStream(FILE* file, bool need_close = false) ) > to get the closing. > > > coding : > ------------ > > jdk/src/hotspot/share/compiler/compilerOracle.cpp > > > 741void CompilerOracle::append_comment_to_file(const char* message) { > 742 assert(has_command_file(), "command file must be specified"); > 743 fileStream stream(fopen(cc_file(), "at")); > 744 stream.print("# "); > 745 for (int index = 0; message[index] != '\0'; index++) { > 746 stream.put(message[index]); > 747 if (message[index] == '\n') stream.print("# "); > 748 } > 749 stream.cr(); > 750} > 751 > 752void CompilerOracle::append_exclude_to_file(const methodHandle& method) { > 753 assert(has_command_file(), "command file must be specified"); > 754 fileStream stream(fopen(cc_file(), "at")); > 755 stream.print("exclude "); > 756 method->method_holder()->name()->print_symbol_on(&stream); > 757 stream.print("."); > 758 method->name()->print_symbol_on(&stream); > 759 method->signature()->print_symbol_on(&stream); > 760 stream.cr(); > 761 stream.cr(); > 762} > > Best regards, Matthias > From matthias.baesken at sap.com Wed Mar 27 16:50:22 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 27 Mar 2019 16:50:22 +0000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: Hello, here is a second webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.1/ I followed Davids idea to use the os_perf coding . For reusing I moved os::Linux::get_tick_information (former name get_total_ticks , but it is now not only total but also steal ) into os_linux . The CPUPerfTicks struct has been enhanced by steal and a boolean flag has_steal_ticks . 28 struct CPUPerfTicks { 29 uint64_t used; 30 uint64_t usedKernel; 31 uint64_t total; 32 uint64_t steal; 33 bool has_steal_ticks; 34 }; I found the "big API" idea a bit too much for just 2 use cases . Best regards , Matthias > -----Original Message----- > From: David Holmes > Sent: Mittwoch, 27. M?rz 2019 11:22 > To: Baesken, Matthias ; Thomas St?fe > > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8221535: add steal tick related information to hs_error file > [linux] > > On 27/03/2019 8:04 pm, Baesken, Matthias wrote: > > Hi Thomas, interesting idea . David ? would you be fine with? what > > Thomas proposes ? > > Isn't adding the missing fields to the existing os_perf code basically > the same thing? I don't see any point in duplicating existing code like > this. > > David > ----- > > > Best regards, Matthias > > > > *From:*Thomas St?fe > > *Sent:* Mittwoch, 27. M?rz 2019 10:28 > > *To:* Baesken, Matthias > > *Cc:* hotspot-dev at openjdk.java.net > > *Subject:* Re: RFR: 8221535: add steal tick related information to > > hs_error file [linux] > > > > Hi Matthias, > > > > this is a lot of coding for just one counter... > > > > Also os_perf_linux.cpp does something very similar. However, that code > > does not get the steal ticks you want, does not seem to care about > > kernel differences and could also be improved. > > > > Proposal: Why do you not tweak your new API to be a general purpose API > > to read all ticks from /proc/stat. That would be way more useful. Later, > > if we feel like it, could change os_perf_linux.cpp to use your function. > > > > We did something similar for the statistical history in the SapMachine, see > > > > > https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linu > x/stathist_linux.cpp > > > > . So I would suggest to have a function reading proc stat and returning > > all values in a struct. That struct should have provisions for "value > > not found" since which values you get depends in kernel version. > > > > -------- > > > > Example: > > > > struct value_t { bool valid; uint64_t v; } > > > > struct cpu_values_t { > > ?value_t user; > > ?value_t nice; > > ?value_t system; > > ?value_t idle; > > ?value_t iowait; > > ?value_t steal; > > ?value_t guest; > > ?value_t guest_nice; > > }; > > > > void parse_proc_stat_cpu_line(const char* line, cpu_values_t* out) { > > ? // Note: existence of some of these values depends on kernel version > > ? out->user.valid = out->nice.valid?= out->system.valid?= > > out->idle.valid?= out->iowait.valid?= out->steal.valid?= > > out->guest.valid?= out->guest_nice.valid?= false; > > ? int user, nice, system, idle, iowait, irq, softirq, steal, guest, > > guest_nice; > > ? int num = ::sscanf(line, "cpu %d %d %d %d %d %d %d %d %d %d", > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &user, &nice, &system, &idle, &iowait, > > &irq, &softirq, &steal, &guest, &guest_nice); > > ? if (num >= 4) { > > ? ? out->user.v = user; out->user.valid = true; > > ? ? out->nice.v?= nice; out->nice.valid = true; > > ? ? out->system.v?= system; out->system.valid = true; > > ? ? out->idle.v?= idle; out->idle.valid = true; > > ? ? if (num >= 5) { // iowait (5) (since Linux 2.5.41) > > ? ? ? out->iowait = iowait; out->iowait.valid = true; > > ? ? ? if (num >= 8) { // steal (8) (since Linux 2.6.11) > > ? ? ? ? out->steal = steal; out->steal.valid = true; > > ? ? ? ? if (num >= 9) { // guest (9) (since Linux 2.6.24) > > ? ? ? ? ? out->guest = guest; out->guest.valid = true; > > ? ? ? ? ? if (num >= 10) { // guest (9) (since Linux 2.6.33) > > ? ? ? ? ? ? out->guest_nice = guest_nice; out->guest_nice.valid = true; > > ? ? ? ? ? } > > ? ? ? } > > ? ?} > > } > > > > Cheers Thomas > > > > On Wed, Mar 27, 2019 at 9:42 AM Baesken, Matthias > > > > wrote: > > > > Hello, please review? the? following change . > > > > Recent Linux kernels provide in the /proc filesystem? ?so called > > "steal" information, see the the proc man page : > > > > ? .. > > "there is an eighth column, steal - stolen time, which is the time > > spent in other operating systems when running in a virtualized > > environment" > > > > The info was useful? in our? internal VM,? when looking into > > performance issues? (especially in virtualized environments), so it > > should be added to the hs_err file . > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8221535 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ > > > > > > Best Regards, Matthias > > From aph at redhat.com Wed Mar 27 16:57:03 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 27 Mar 2019 16:57:03 +0000 Subject: TestOverloadCompileQueues sometimes doesn't test anything Message-ID: This test sets -XX:-TieredCompilation -XX:CompileThreshold=2 -XX:CICompilerCount=1 then repeatedly loads classes and invokes their methods. However, in testing as often as not this test exits *before* any of the test classes have been compiled, so the test is completely ineffective. If I set CICompilerCount=8 I can get some unloading. This is with a debug build: I need that for assertions. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From charlie.hunt at oracle.com Wed Mar 27 18:56:59 2019 From: charlie.hunt at oracle.com (charlie hunt) Date: Wed, 27 Mar 2019 13:56:59 -0500 Subject: RFC: Draft JEP: Improved Metaspace allocator In-Reply-To: <1c64745ad2425ff4d1125344e1076568d38401b6.camel@oracle.com> References: <1c64745ad2425ff4d1125344e1076568d38401b6.camel@oracle.com> Message-ID: To add to Thomas Schatzl's comments ... The stress test mentioned is something we would not be able to share since it looks like there may be some redistribution restrictions with some of its components. A little while ago you posted a link to the following webrev: http://cr.openjdk.java.net/~stuefe/webrevs/autouncommit-metachunks/webrev.00/webrev/index.html A build with the above changes added to it and running the same stress test showed better behavior. The above said, the stress test is rather extreme. I also had not realized jcmd "VM.metaspace" existed. Thanks for the tip! hths, Charlie On 3/27/19 11:01 AM, Thomas Schatzl wrote: > Hi, > > On Wed, 2019-03-27 at 16:29 +0100, Thomas St?fe wrote: >> Hi Thomas, >> >> On Wed, Mar 27, 2019 at 2:57 PM Thomas Schatzl < >> thomas.schatzl at oracle.com> wrote: >>> On Wed, 2019-03-20 at 20:19 +0100, Thomas St?fe wrote: >>>> Dear all, >>>> >>>> I would like to hear what you think about the following proposal: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8221173 >>>> >>>> Do you consider this worthwhile? >>> Yes, definitely! >> Great :) >> >>> At the beginning of this month we were internally looking at a >>> metaspace stress application, and it took some time to nail the >>> problem on metaspace fragmentation. Improving fragmentation, the >>> visibility of such problems (fragmentation statistics) and >>> potential options to e.g. favor compactness vs. performance. >> Just to be sure, are you aware of the jcmd "VM.metaspace" command we >> (RedHat & SAP) added for jdk11? > No, at least I did not. At least not at the time when it would have > been very useful :( > >> It can be used to analyze Metaspace in quite some detail. We did not >> make a big fuss over it, but it is there and quite useful. > Which is obviously the problem. > >> It may of course be missing functionality; if yes, please tell us. > I will forward this information. I think we may get some time to do > some more runs and look over the output. > >> About your stress test, I'd be interested in a reproduction case if >> that is possible. I am always collecting pathological cases to play >> with. > I will bring that up too. > >>>> I debated with myself whether a JEP is the correct umbrella for >>>> it. In the past I did larger changes like this without JEP but in >>>> this case I feel better with a confirmed and sponsored JEP >>>> behind me - from a certain size on it just feels more >>>> appropriate. >>>> >>>> As usual, I am not sure where to put Metaspace-related proposals >>>> to catch all interested people, so I start with hs-dev. Please >>>> feel free to redirect if needed. >>> You are right that a JEP is the only way to get guaranteed >>> visibility to people. JEPs are mostly the only source for third >>> party "what's new in JDK XX" write-ups. >>> So if you want to reach a wide (non-hotspot-dev) audience, JEPs are >>> basically the only way. I could kind of see that this might be the >>> case. >> For me it is more of a question of "I don't want to make a fuss" vs >> "I don't want to sneak large changes in behind everyone's back". In >> this case, I side with the latter. >> >> As additional advantage, a JEP gives a notion of progress to the >> outside world. And I have the feeling it activates more Reviewer >> muscle than large-non-JEP patches. > :) > > Thanks, > Thomas > From thomas.stuefe at gmail.com Wed Mar 27 22:39:23 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 27 Mar 2019 23:39:23 +0100 Subject: RFC: Draft JEP: Improved Metaspace allocator In-Reply-To: References: <1c64745ad2425ff4d1125344e1076568d38401b6.camel@oracle.com> Message-ID: Hi Thomas, Charlie, I feel a bit bad because I should have done a better job documenting and publishing that command. The command was developed by Zhengyu and me 2017/18 to find all the nooks and crannies where memory got lost in Metaspace. But somehow I forgot to write any release notes for it. I may have to come up with some official documentation, at least some belated release note. In the meantime: I recently wrote a some small articles covering Metaspace in the hotspot VM, among others describing how VM.metaspace can be used to analyze metaspace waste([1], [2]) - maybe that helps plugging the documentation whole a bit for now. I plan to flesh out these articles more when I have time, so feedback is appreciated. [1] https://stuefe.de/posts/metaspace/analyze-metaspace-with-jcmd/ [2] https://stuefe.de/posts/metaspace/analyze-classloader-footprint-with-jcmd/ Cheers, Thomas On Wed, Mar 27, 2019 at 7:59 PM charlie hunt wrote: > To add to Thomas Schatzl's comments ... > > The stress test mentioned is something we would not be able to share > since it looks like there may be some redistribution restrictions with > some of its components. > > A little while ago you posted a link to the following webrev: > > http://cr.openjdk.java.net/~stuefe/webrevs/autouncommit-metachunks/webrev.00/webrev/index.html > > A build with the above changes added to it and running the same stress > test showed better behavior. > > The above said, the stress test is rather extreme. > > I also had not realized jcmd "VM.metaspace" existed. Thanks for the tip! > > hths, > > Charlie > > On 3/27/19 11:01 AM, Thomas Schatzl wrote: > > Hi, > > > > On Wed, 2019-03-27 at 16:29 +0100, Thomas St?fe wrote: > >> Hi Thomas, > >> > >> On Wed, Mar 27, 2019 at 2:57 PM Thomas Schatzl < > >> thomas.schatzl at oracle.com> wrote: > >>> On Wed, 2019-03-20 at 20:19 +0100, Thomas St?fe wrote: > >>>> Dear all, > >>>> > >>>> I would like to hear what you think about the following proposal: > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8221173 > >>>> > >>>> Do you consider this worthwhile? > >>> Yes, definitely! > >> Great :) > >> > >>> At the beginning of this month we were internally looking at a > >>> metaspace stress application, and it took some time to nail the > >>> problem on metaspace fragmentation. Improving fragmentation, the > >>> visibility of such problems (fragmentation statistics) and > >>> potential options to e.g. favor compactness vs. performance. > >> Just to be sure, are you aware of the jcmd "VM.metaspace" command we > >> (RedHat & SAP) added for jdk11? > > No, at least I did not. At least not at the time when it would have > > been very useful :( > > > >> It can be used to analyze Metaspace in quite some detail. We did not > >> make a big fuss over it, but it is there and quite useful. > > Which is obviously the problem. > > > >> It may of course be missing functionality; if yes, please tell us. > > I will forward this information. I think we may get some time to do > > some more runs and look over the output. > > > >> About your stress test, I'd be interested in a reproduction case if > >> that is possible. I am always collecting pathological cases to play > >> with. > > I will bring that up too. > > > >>>> I debated with myself whether a JEP is the correct umbrella for > >>>> it. In the past I did larger changes like this without JEP but in > >>>> this case I feel better with a confirmed and sponsored JEP > >>>> behind me - from a certain size on it just feels more > >>>> appropriate. > >>>> > >>>> As usual, I am not sure where to put Metaspace-related proposals > >>>> to catch all interested people, so I start with hs-dev. Please > >>>> feel free to redirect if needed. > >>> You are right that a JEP is the only way to get guaranteed > >>> visibility to people. JEPs are mostly the only source for third > >>> party "what's new in JDK XX" write-ups. > >>> So if you want to reach a wide (non-hotspot-dev) audience, JEPs are > >>> basically the only way. I could kind of see that this might be the > >>> case. > >> For me it is more of a question of "I don't want to make a fuss" vs > >> "I don't want to sneak large changes in behind everyone's back". In > >> this case, I side with the latter. > >> > >> As additional advantage, a JEP gives a notion of progress to the > >> outside world. And I have the feeling it activates more Reviewer > >> muscle than large-non-JEP patches. > > :) > > > > Thanks, > > Thomas > > > From fujie at loongson.cn Thu Mar 28 00:56:07 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 28 Mar 2019 08:56:07 +0800 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <57cf05cb-38bd-43b2-f7d5-bc80ee087c03@oracle.com> <501b392f-24e0-860a-fff8-b46e0d0c8b7d@oracle.com> <35d3331a-258a-3d75-0bd9-09c0072ebdd9@loongson.cn> <70d3f07c-855b-f356-d2cc-3061de40e572@oracle.com> Message-ID: Nice! Thanks Alan. On 2019/3/27 ??6:12, Alan Bateman wrote: > On 27/03/2019 09:50, David Holmes wrote: >> It's been pointed out to me that these "micro" "tests" are outside of >> jtreg and can only be run via make. > Once the build-microbenchmark target is built then you should have > images/test/micro/benchmarks.jar in your build output directory. > That's an executable JAR that includes the JMH so you can execute that > directly and specify whatever parameters you need. > > -Alan From fujie at loongson.cn Thu Mar 28 00:57:54 2019 From: fujie at loongson.cn (Jie Fu) Date: Thu, 28 Mar 2019 08:57:54 +0800 Subject: [Request for help] Questions about make test TEST="micro" failures In-Reply-To: <6bcf04fc-e94e-c883-81b5-a10f88a662bf@oracle.com> References: <7c74000a-e299-2296-fe60-2a919798dc2c@loongson.cn> <2111ddfd-3171-5351-b3f4-4fd167599073@oracle.com> <6bcf04fc-e94e-c883-81b5-a10f88a662bf@oracle.com> Message-ID: <21bf205c-5837-ea81-be95-ce40acf4196d@loongson.cn> Thanks Claes. On 2019/3/27 ??6:25, Claes Redestad wrote: > On 2019-03-27 11:03, Claes Redestad wrote: >> I'll have a look at these CallSite micros. I suspect we don't run them >> regularly and they might have atrophied. > > Seems these micros depend on a utility (Indify) that couldn't be ported > over from jmh-jdk-microbenchmarks to run properly, and I missed these > benchmarks were depending on that utility and failing with > AssertionErrors in JEP 230 - mainly since make test logs this as a pass. > > I've filed https://bugs.openjdk.java.net/browse/JDK-8221543 to deal > with this. > > /Claes From jingtian at loongson.cn Thu Mar 28 05:22:24 2019 From: jingtian at loongson.cn (Jing Tian) Date: Thu, 28 Mar 2019 13:22:24 +0800 Subject: RFC: Make test failed because of the locale LANG Message-ID: <5fd4d748-c9cf-3779-3a65-b7c7547ab9ce@loongson.cn> Hi, When I am doing the 'make test'.If the local LANG is set as 'zh_CN.UTF-8', Test cases will have a lot of error messages. ============================== Test summary ============================== ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL ERROR >> jtreg:test/hotspot/jtreg:tier1???????????????????? 1373 1371???? 2???? 0 << >> jtreg:test/jdk:tier1?????????????????????????????? 1867 1860???? 7???? 0 << >> jtreg:test/langtools:tier1???????????????????????? 3922 2470? 1450???? 2 << ?? jtreg:test/nashorn:tier1????????????????????????????? 0 0???? 0???? 0 ?? jtreg:test/jaxp:tier1???????????????????????????????? 0 0???? 0???? 0 >> jtreg:test/jdk:tier2?????????????????????????????? 3334 3305??? 29???? 0 << >> jtreg:test/langtools:tier2 11???? 9???? 2???? 0 << ?? jtreg:test/nashorn:tier2???????????????????????????? 35 35???? 0???? 0 >> jtreg:test/jaxp:tier2?????????????????????????????? 438 437???? 1???? 0 << >> jtreg:test/jdk:tier3?????????????????????????????? 1104 1068??? 36???? 0 << ?? jtreg:test/langtools:tier3??????????????????????????? 0 0???? 0???? 0 ?? jtreg:test/nashorn:tier3????????????????????????????? 0 0???? 0???? 0 ?? jtreg:test/jaxp:tier3???????????????????????????????? 0 0???? 0???? 0 ============================== On the same machine,when i set LANG=en_US.UTF-8. ============================== Test summary ============================== ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL ERROR >> jtreg:test/hotspot/jtreg:tier1???????????????????? 1388 1386???? 2???? 0 << >> jtreg:test/jdk:tier1?????????????????????????????? 1867 1843??? 19???? 5 << ?? jtreg:test/langtools:tier1???????????????????????? 3920 3920???? 0???? 0 ?? jtreg:test/nashorn:tier1????????????????????????????? 0 0???? 0???? 0 ?? jtreg:test/jaxp:tier1???????????????????????????????? 0 0???? 0???? 0 >> jtreg:test/jdk:tier2?????????????????????????????? 3328 3290??? 31???? 7 << ?? jtreg:test/langtools:tier2?????????????????????????? 11 11???? 0???? 0 ?? jtreg:test/nashorn:tier2???????????????????????????? 35 35???? 0???? 0 ?? jtreg:test/jaxp:tier2?????????????????????????????? 438 438???? 0???? 0 >> jtreg:test/jdk:tier3?????????????????????????????? 1104 1067??? 37???? 0 << ?? jtreg:test/langtools:tier3??????????????????????????? 0 0???? 0???? 0 ?? jtreg:test/nashorn:tier3????????????????????????????? 0 0???? 0???? 0 ?? jtreg:test/jaxp:tier3???????????????????????????????? 0 0???? 0???? 0 By comparison we can find, lots of(1000+) langtools tests will get fail, and other(about 30+, exclude some X11 and sanity that result problem) test cases will also fail because of local LANG. such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java, shouldContain("Error: Could not find or load main class " + CLASSNAME) When in the zh_CN.UTF-8 environment, because some of the output information is changed to Chinese by some properties file, the English cannot be matched, which will result in an fail. When I set? LANG=en_US/LC_ALL=C, this test will pass. I think there are some possible solutions. 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this modification is more violent because he will affect all test cases. 2.We modify each individual test?E.g test/hotspot/jtreg/compiler/c2/Test8062950.java diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java ?package compiler.c2; ?import jdk.test.lib.process.ProcessTools; +import java.util.Map; +import jdk.test.lib.process.OutputAnalyzer; ?public class Test8062950 { ???? private static final String CLASSNAME = "DoesNotExist"; ???? public static void main(String[] args) throws Exception { - ProcessTools.executeTestJvm("-Xcomp", - "-XX:-TieredCompilation", - "-XX:-UseOptoBiasInlining", - CLASSNAME) - .shouldHaveExitValue(1) -??????????????????? .shouldContain("Error: Could not find or load main class " + CLASSNAME) -??????????????????? .shouldNotContain("A fatal error has been detected") -??????????????????? .shouldNotContain("Internal Error") -??????????????????? .shouldNotContain("HotSpot Virtual Machine Error"); +??????? final ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(true, + "-Xcomp", + "-XX:-TieredCompilation", + "-XX:-UseOptoBiasInlining", + CLASSNAME); +??????? final Map env = pb.environment(); +??????? env.put("LC_ALL", "en_US.UTF-8"); +??????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); + output.shouldHaveExitValue(1); +??????? output.shouldContain("Error: Could not find or load main class " + CLASSNAME); +??????? output.shouldNotContain("A fatal error has been detected"); +??????? output.shouldNotContain("Internal Error"); +??????? output.shouldNotContain("HotSpot Virtual Machine Error"); } ?} But I don't know if this will change the test program too much, because the related problems are a lot in langtools tests. 3.And i find that there is a function can judge the locale ?if (!isEnglishLocale()) { // only english version return; ?} But in this case, I think in many test cases, we may not be able to return so directly, because some test cases may have other test purposes. So I don't know if you have any ideas or some suggestions to solve the problem that the output information and English do not match in this non-English environment. Cheers, Jing Tian From david.holmes at oracle.com Thu Mar 28 05:47:06 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 28 Mar 2019 15:47:06 +1000 Subject: RFC: Make test failed because of the locale LANG In-Reply-To: <5fd4d748-c9cf-3779-3a65-b7c7547ab9ce@loongson.cn> References: <5fd4d748-c9cf-3779-3a65-b7c7547ab9ce@loongson.cn> Message-ID: <396490c6-9917-2306-c288-2de5ec92bdc9@oracle.com> Hi Jing, On 28/03/2019 3:22 pm, Jing Tian wrote: > Hi, > > When I am doing the 'make test'.If the local LANG is set as > 'zh_CN.UTF-8', Test cases will have a lot of error messages. > > ============================== > Test summary > ============================== > ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL > ERROR > >> jtreg:test/hotspot/jtreg:tier1???????????????????? 1373 1371 2???? 0 << > >> jtreg:test/jdk:tier1?????????????????????????????? 1867 1860 7???? 0 << > >> jtreg:test/langtools:tier1???????????????????????? 3922 2470 > 1450???? 2 << > ?? jtreg:test/nashorn:tier1????????????????????????????? 0 0???? 0???? 0 > ?? jtreg:test/jaxp:tier1???????????????????????????????? 0 0???? 0???? 0 > >> jtreg:test/jdk:tier2?????????????????????????????? 3334 3305 29 > 0 << > >> jtreg:test/langtools:tier2 11???? 9???? 2???? 0 << > ?? jtreg:test/nashorn:tier2???????????????????????????? 35 35???? 0???? 0 > >> jtreg:test/jaxp:tier2?????????????????????????????? 438 437 1???? 0 << > >> jtreg:test/jdk:tier3?????????????????????????????? 1104 1068 36 > 0 << > ?? jtreg:test/langtools:tier3??????????????????????????? 0 0???? 0???? 0 > ?? jtreg:test/nashorn:tier3????????????????????????????? 0 0???? 0???? 0 > ?? jtreg:test/jaxp:tier3???????????????????????????????? 0 0???? 0???? 0 > ============================== Given most of these are not hotspot issues and given the number of failures, I think this is something that would need to be discussed much more broadly. So I've cc'd core-libs-dev and compiler-dev. I recall there was a very recent discussion regarding some tests failing for the same reason, but don't recall the outcome. David ----- > On the same machine,when i set LANG=en_US.UTF-8. > > ============================== > Test summary > ============================== > ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL > ERROR > >> jtreg:test/hotspot/jtreg:tier1???????????????????? 1388 1386 2???? 0 << > >> jtreg:test/jdk:tier1?????????????????????????????? 1867 1843 19 > 5 << > ?? jtreg:test/langtools:tier1???????????????????????? 3920 3920 0???? 0 > ?? jtreg:test/nashorn:tier1????????????????????????????? 0 0???? 0???? 0 > ?? jtreg:test/jaxp:tier1???????????????????????????????? 0 0???? 0???? 0 > >> jtreg:test/jdk:tier2?????????????????????????????? 3328 3290 31 > 7 << > ?? jtreg:test/langtools:tier2?????????????????????????? 11 11???? 0???? 0 > ?? jtreg:test/nashorn:tier2???????????????????????????? 35 35???? 0???? 0 > ?? jtreg:test/jaxp:tier2?????????????????????????????? 438 438???? 0???? 0 > >> jtreg:test/jdk:tier3?????????????????????????????? 1104 1067 37 > 0 << > ?? jtreg:test/langtools:tier3??????????????????????????? 0 0???? 0???? 0 > ?? jtreg:test/nashorn:tier3????????????????????????????? 0 0???? 0???? 0 > ?? jtreg:test/jaxp:tier3???????????????????????????????? 0 0???? 0???? 0 > > > By comparison we can find, lots of(1000+) langtools tests will get fail, > and other(about 30+, exclude some X11 and sanity that > > result problem) test cases will also fail because of local LANG. > > > such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java, > > shouldContain("Error: Could not find or load main class " + CLASSNAME) > > > When in the zh_CN.UTF-8 environment, because some of the output > information is changed to Chinese by some properties file, > > the English cannot be matched, which will result in an fail. > > When I set? LANG=en_US/LC_ALL=C, this test will pass. > > > I think there are some possible solutions. > > > 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this modification > is more violent because he will affect all test cases. > > > 2.We modify each individual test?E.g > test/hotspot/jtreg/compiler/c2/Test8062950.java > > diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java > ?package compiler.c2; > ?import jdk.test.lib.process.ProcessTools; > +import java.util.Map; > +import jdk.test.lib.process.OutputAnalyzer; > > ?public class Test8062950 { > ???? private static final String CLASSNAME = "DoesNotExist"; > ???? public static void main(String[] args) throws Exception { > - ProcessTools.executeTestJvm("-Xcomp", > - "-XX:-TieredCompilation", > - "-XX:-UseOptoBiasInlining", > - CLASSNAME) > - .shouldHaveExitValue(1) > -??????????????????? .shouldContain("Error: Could not find or load main > class " + CLASSNAME) > -??????????????????? .shouldNotContain("A fatal error has been detected") > -??????????????????? .shouldNotContain("Internal Error") > -??????????????????? .shouldNotContain("HotSpot Virtual Machine Error"); > +??????? final ProcessBuilder pb = > ProcessTools.createJavaProcessBuilder(true, > + "-Xcomp", > + "-XX:-TieredCompilation", > + "-XX:-UseOptoBiasInlining", > + CLASSNAME); > +??????? final Map env = pb.environment(); > +??????? env.put("LC_ALL", "en_US.UTF-8"); > +??????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); > + output.shouldHaveExitValue(1); > +??????? output.shouldContain("Error: Could not find or load main class > " + CLASSNAME); > +??????? output.shouldNotContain("A fatal error has been detected"); > +??????? output.shouldNotContain("Internal Error"); > +??????? output.shouldNotContain("HotSpot Virtual Machine Error"); > } > ?} > > But I don't know if this will change the test program too much, because > the related problems are a lot in langtools tests. > > > 3.And i find that there is a function can judge the locale > > ?if (!isEnglishLocale()) { // only english version > return; > ?} > > But in this case, I think in many test cases, we may not be able to > return so directly, because some test cases may have other test purposes. > > > So I don't know if you have any ideas or some suggestions to solve the > problem that the output information and English do not match in this > > non-English environment. > > > Cheers, > > Jing Tian > > > > From david.holmes at oracle.com Thu Mar 28 07:20:33 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 28 Mar 2019 17:20:33 +1000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> Message-ID: <30b2b3c5-38f7-8569-10cd-e781129b5da5@oracle.com> On 26/03/2019 6:32 pm, Baesken, Matthias wrote: > Hi David, > >> This is looking better though I'm struggling to keep all the dots >> connected in terms of the APIs - I can't quite see the pattern for what >> calls what and what overrides what. > > I just followed here what is done at other places of VM_Version / Abstract_VM_Version , see > for example : Sorry but I'm having trouble following all the levels of indirection. You have: VM_version::print_detected_virtualization which calls VM_Version::print_platform_virtualization_info which you have defined for s390 and PPC, but I don't see it for other architectures? Then you have os::Linux and os::Windows print_virtualization_info that then calls VM_version::print_detected_virtualization Are the other platforms handled by the switch on VM_Version::get_detected_virtualization() ? Why isn't that encapsulated in print_platform_virtualization_info for those platforms (which I think is only x86 at this stage?). I'm finding it extremely hard to see the clear shape of this API - sorry. David ----- > src/hotspot/share/runtime/vm_version.hpp > > 82 // ..... Platforms that > 83 // need to specialize this define VM_Version::early_initialize(). > 84 static void early_initialize() { } > 85 > 86 // Called to initialize VM variables needing initialization > 87 // after command line parsing. Platforms that need to specialize > 88 // this should define VM_Version::init_before_ergo(). > 89 static void init_before_ergo() {} > > >> 2. os_windows.cpp >> >> Why the indirection through os::win32::print_virtualization_info instead >> of just calling VM_Version::print_detected_virtualization directly? > > Yes that's a good point - I think I should change this . > > Best regards, Matthias > > > > >> -----Original Message----- >> From: David Holmes >> Sent: Mittwoch, 20. M?rz 2019 07:15 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Cc: Doerr, Martin >> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >> hs_error file on linux/windows x86_64 >> >> Hi Matthias, >> >> On 19/03/2019 11:53 pm, Baesken, Matthias wrote: >>> Hello, here is a new webrev : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ >>> >>> This moved the ppc/s390 virtualization related info output into the >> platform/cpu - specific coding . >> >> This is looking better though I'm struggling to keep all the dots >> connected in terms of the APIs - I can't quite see the pattern for what >> calls what and what overrides what. A couple of initial comments: >> >> 1. We already have a ton of cpuid related parsing in >> ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check >> virtualizations" and store the information for later use? >> >> 2. os_windows.cpp >> >> Why the indirection through os::win32::print_virtualization_info instead >> of just calling VM_Version::print_detected_virtualization directly? >> >> I'll keep trying to piece this together. :) >> >> Thanks, >> David >> ----- >> >>> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Montag, 11. M?rz 2019 08:36 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Cc: Doerr, Martin >>>> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >>>> hs_error file on linux/windows x86_64 >>>> >>>> On 7/03/2019 3:13 am, Baesken, Matthias wrote: >>>>> Hello , could I push the latest version of 8219241 ? >>>> >>>> Sorry based on your earlier email I thought you were looking at doing >>>> the restructuring. I was then on vacation most of last week. >>>> >>>>> A change that moves the coding more into platform/cpu - specific >> coding >>>> (if this is wanted) >>>>> could be done when bringing in the AIX virtualization related info in >>>> another patch which I plan to do . >>>> >>>> If that is imminent then okay - I don't like the code as it is. >>>> >>>> Thanks, >>>> David >>>> >>> From tobias.hartmann at oracle.com Thu Mar 28 07:54:20 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 28 Mar 2019 08:54:20 +0100 Subject: TestOverloadCompileQueues sometimes doesn't test anything In-Reply-To: References: Message-ID: Hi Andrew, On 27.03.19 17:57, Andrew Haley wrote: > This test sets -XX:-TieredCompilation -XX:CompileThreshold=2 > -XX:CICompilerCount=1 then repeatedly loads classes and invokes their > methods. However, in testing as often as not this test exits *before* > any of the test classes have been compiled, so the test is completely > ineffective. If I set CICompilerCount=8 I can get some unloading. > > This is with a debug build: I need that for assertions. I've added this test with my fix for 8163511 [1]. Its purpose is not to compile methods but to *try* to overload the compile queue with compilation requests. CICompilerCount is set to 1 to prevent the compiler threads to keep up with these requests. Since it's a regression test, the behavior you are observing is expected - it means that the fix works and unloaded methods are removed from the queue before it fills up. Otherwise, you would hit the "leaking compilation tasks" assert (at least with the configuration/machines I've tested on, the test might not always reproduce the problem). Here is a detailed explanation: https://mail.openjdk.java.net/pipermail/hotspot-dev/2019-February/037006.html Best regards, Tobias [1] https://bugs.openjdk.java.net/browse/JDK-8163511 From adinn at redhat.com Thu Mar 28 10:44:17 2019 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 28 Mar 2019 10:44:17 +0000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM Message-ID: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> Could I please have reviews for this patch which changes the initialization of four os/cpu-specific static final constants used by class Unsafe. The patch injects values during JVM startup (along similar lines to how String field COMPACT_STRINGS is initialized) rather than retrieving them via native method calls. This localizes the computation of the assigned values in one place, relocates the constants into a separate final, static-only Java class and avoids the need to maintain four separate native methods. A further motive for making this change is to pave the way for adding the writeback cache line size/address mask as Unsafe constants for use by the JEP proposed in JDK-8207851. JIRA: https://bugs.openjdk.java.net/browse/JDK-8221477 Webrev: http://cr.openjdk.java.net/~adinn/8221477/webrev.01 Testing: submit repo tests passed regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From thomas.stuefe at gmail.com Thu Mar 28 12:17:26 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 28 Mar 2019 13:17:26 +0100 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> Message-ID: Hi, this looks fine, nits only: UnsafeConstantsFixup::do_field() you could shrink that coding a bit by factoring out the checks, e.g.: static oop mirror_with_checks_from_field_descriptor(fieldDescriptor* fd) { oop mirror = fd->field_holder()->java_mirror(); assert(fd->field_holder() == SystemDictionary::UnsafeConstants_klass(), "Should be UnsafeConstants"); assert(mirror != NULL, "UnsafeConstants must have mirror already"); return oop; } Also, is there a reason that code from thread.cpp: jint address_size = sizeof(void*); jint page_size = os::vm_page_size(); jboolean is_big_endian = LITTLE_ENDIAN_ONLY(false) BIG_ENDIAN_ONLY(true); jint use_unaligned_access = UseUnalignedAccesses; could not just happen right in UnsafeConstantsFixup::do_field()? You would not have to pass all these arguments around, param list could be empty. src/hotspot/share/classfile/vmSymbols.hpp Alignment of trailing '\' broken src/java.base/share/classes/jdk/internal/misc/UnsafeConstants.java s/intitialize/initialize s/iff/if Cheers, Thomas On Thu, Mar 28, 2019 at 11:44 AM Andrew Dinn wrote: > Could I please have reviews for this patch which changes the > initialization of four os/cpu-specific static final constants used by > class Unsafe. The patch injects values during JVM startup (along similar > lines to how String field COMPACT_STRINGS is initialized) rather than > retrieving them via native method calls. This localizes the computation > of the assigned values in one place, relocates the constants into a > separate final, static-only Java class and avoids the need to maintain > four separate native methods. > > A further motive for making this change is to pave the way for adding > the writeback cache line size/address mask as Unsafe constants for use > by the JEP proposed in JDK-8207851. > > JIRA: https://bugs.openjdk.java.net/browse/JDK-8221477 > Webrev: http://cr.openjdk.java.net/~adinn/8221477/webrev.01 > > Testing: > submit repo tests passed > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > > > From adinn at redhat.com Thu Mar 28 14:41:12 2019 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 28 Mar 2019 14:41:12 +0000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> Message-ID: <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> On 28/03/2019 12:17, Thomas St?fe wrote: > this looks fine, nits only: Thank you for the review, Thomas. I'll post a follow up webrev to address the comments. Responses are inline. > UnsafeConstantsFixup::do_field() > > you could shrink that coding a bit by factoring out the checks, e.g.: > . . . Yes, that was egregious cut-and-paste coding. Actually, the idea is that all fields of the class are final static and all of them are expected to be injected (which is why the final else clause asserts). So, I could just move the asserts up to precede the if-elseif cascade and also include checks that the field is final and static. void do_field(fieldDescriptor* fd) { oop mirror = fd->field_holder()->java_mirror(); assert(mirror != NULL, "UnsafeConstants must have mirror already"); assert(fd->field_holder() == SystemDictionary::UnsafeConstants_klass(), "Should be UnsafeConstants"); assert(fd->is_final(), "fields of UnsafeConstants must be final"); assert(fd->is_static() "fields of UnsafeConstants must be static"); if (fd->name() == vmSymbols::address_size_name()) { mirror->int_field_put(fd->offset(), _address_size); } else if (fd->name() == vmSymbols::page_size_name()) { mirror->int_field_put(fd->offset(), _page_size); } else if (fd->name() == vmSymbols::big_endian_name()) { . . . } else { assert(false, "unexpected UnsafeConstants field"); } > Also, is there a reason that code from thread.cpp: > > ? ? jint address_size = sizeof(void*); > ? ? jint page_size = os::vm_page_size(); > ? ? jboolean is_big_endian = LITTLE_ENDIAN_ONLY(false) > BIG_ENDIAN_ONLY(true); > ? ? jint use_unaligned_access = UseUnalignedAccesses; > > could not just happen right in UnsafeConstantsFixup::do_field()? You > would not have to pass all these arguments around, param list could be > empty. No good reason. Your suggestion is better. > src/hotspot/share/classfile/vmSymbols.hpp > > Alignment of trailing '\' broken Oops! > src/java.base/share/classes/jdk/internal/misc/UnsafeConstants.java > > s/intitialize/initialize > s/iff/if The second of those was actually intended to be iff. This is a common abbreviation used by English/US mathematicians and logicians to write 'if and only if' (it is also sometimes written as <=>). Since you didn't recognize it I guess I really need to write it out in full. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From thomas.stuefe at gmail.com Thu Mar 28 15:22:19 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 28 Mar 2019 16:22:19 +0100 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> Message-ID: On Thu, Mar 28, 2019 at 3:41 PM Andrew Dinn wrote: > On 28/03/2019 12:17, Thomas St?fe wrote: > > this looks fine, nits only: > > Thank you for the review, Thomas. I'll post a follow up webrev to > address the comments. Responses are inline. > > > UnsafeConstantsFixup::do_field() > > > > you could shrink that coding a bit by factoring out the checks, e.g.: > > . . . > > Yes, that was egregious cut-and-paste coding. > > Actually, the idea is that all fields of the class are final static and > all of them are expected to be injected (which is why the final else > clause asserts). > > So, I could just move the asserts up to precede the if-elseif cascade > and also include checks that the field is final and static. > > void do_field(fieldDescriptor* fd) { > oop mirror = fd->field_holder()->java_mirror(); > assert(mirror != NULL, "UnsafeConstants must have mirror already"); > assert(fd->field_holder() == > SystemDictionary::UnsafeConstants_klass(), "Should be UnsafeConstants"); > assert(fd->is_final(), "fields of UnsafeConstants must be final"); > assert(fd->is_static() "fields of UnsafeConstants must be static"); > if (fd->name() == vmSymbols::address_size_name()) { > mirror->int_field_put(fd->offset(), _address_size); > } else if (fd->name() == vmSymbols::page_size_name()) { > mirror->int_field_put(fd->offset(), _page_size); > } else if (fd->name() == vmSymbols::big_endian_name()) { > . . . > } else { > assert(false, "unexpected UnsafeConstants field"); > } > > > Also, is there a reason that code from thread.cpp: > > > > jint address_size = sizeof(void*); > > jint page_size = os::vm_page_size(); > > jboolean is_big_endian = LITTLE_ENDIAN_ONLY(false) > > BIG_ENDIAN_ONLY(true); > > jint use_unaligned_access = UseUnalignedAccesses; > > > > could not just happen right in UnsafeConstantsFixup::do_field()? You > > would not have to pass all these arguments around, param list could be > > empty. > > No good reason. Your suggestion is better. > > > src/hotspot/share/classfile/vmSymbols.hpp > > > > Alignment of trailing '\' broken > > Oops! > > > src/java.base/share/classes/jdk/internal/misc/UnsafeConstants.java > > > > s/intitialize/initialize > > s/iff/if > The second of those was actually intended to be iff. This is a common > abbreviation used by English/US mathematicians and logicians to write > 'if and only if' (it is also sometimes written as <=>). Since you didn't > recognize it I guess I really need to write it out in full. > Oh, don't worry on my account. I am not a native speaker nor a mathematician. You could leave iff and add [sic] to make everyone curious and start googling "iff" :) ..Thomas > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From thomas.stuefe at gmail.com Thu Mar 28 15:32:22 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 28 Mar 2019 16:32:22 +0100 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> Message-ID: Btw congrats for finally getting JEP 352 to move on. That really took a while. Cheers, Thomas On Thu, Mar 28, 2019 at 4:22 PM Thomas St?fe wrote: > > > On Thu, Mar 28, 2019 at 3:41 PM Andrew Dinn wrote: > >> On 28/03/2019 12:17, Thomas St?fe wrote: >> > this looks fine, nits only: >> >> Thank you for the review, Thomas. I'll post a follow up webrev to >> address the comments. Responses are inline. >> >> > UnsafeConstantsFixup::do_field() >> > >> > you could shrink that coding a bit by factoring out the checks, e.g.: >> > . . . >> >> Yes, that was egregious cut-and-paste coding. >> >> Actually, the idea is that all fields of the class are final static and >> all of them are expected to be injected (which is why the final else >> clause asserts). >> >> So, I could just move the asserts up to precede the if-elseif cascade >> and also include checks that the field is final and static. >> >> void do_field(fieldDescriptor* fd) { >> oop mirror = fd->field_holder()->java_mirror(); >> assert(mirror != NULL, "UnsafeConstants must have mirror already"); >> assert(fd->field_holder() == >> SystemDictionary::UnsafeConstants_klass(), "Should be UnsafeConstants"); >> assert(fd->is_final(), "fields of UnsafeConstants must be final"); >> assert(fd->is_static() "fields of UnsafeConstants must be static"); >> if (fd->name() == vmSymbols::address_size_name()) { >> mirror->int_field_put(fd->offset(), _address_size); >> } else if (fd->name() == vmSymbols::page_size_name()) { >> mirror->int_field_put(fd->offset(), _page_size); >> } else if (fd->name() == vmSymbols::big_endian_name()) { >> . . . >> } else { >> assert(false, "unexpected UnsafeConstants field"); >> } >> >> > Also, is there a reason that code from thread.cpp: >> > >> > jint address_size = sizeof(void*); >> > jint page_size = os::vm_page_size(); >> > jboolean is_big_endian = LITTLE_ENDIAN_ONLY(false) >> > BIG_ENDIAN_ONLY(true); >> > jint use_unaligned_access = UseUnalignedAccesses; >> > >> > could not just happen right in UnsafeConstantsFixup::do_field()? You >> > would not have to pass all these arguments around, param list could be >> > empty. >> >> No good reason. Your suggestion is better. >> >> > src/hotspot/share/classfile/vmSymbols.hpp >> > >> > Alignment of trailing '\' broken >> >> Oops! >> >> > src/java.base/share/classes/jdk/internal/misc/UnsafeConstants.java >> > >> > s/intitialize/initialize >> > s/iff/if >> The second of those was actually intended to be iff. This is a common >> abbreviation used by English/US mathematicians and logicians to write >> 'if and only if' (it is also sometimes written as <=>). Since you didn't >> recognize it I guess I really need to write it out in full. >> > > Oh, don't worry on my account. I am not a native speaker nor a > mathematician. You could leave iff and add [sic] to make everyone curious > and start googling "iff" :) > > ..Thomas > > >> >> regards, >> >> >> Andrew Dinn >> ----------- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in England and Wales under Company Registration No. 03798903 >> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >> > From bob.vandette at oracle.com Thu Mar 28 15:37:53 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 28 Mar 2019 11:37:53 -0400 Subject: RFR: 8217338: [Containers] Improve systemd slice memory limit support In-Reply-To: <9a911da4b2a4a8eff42c7c2f1a19c06f6521da8d.camel@redhat.com> References: <4a1ae4e0e3efd68818bfe868349972aa878de60b.camel@redhat.com> <9a911da4b2a4a8eff42c7c2f1a19c06f6521da8d.camel@redhat.com> Message-ID: <226CB44C-7B3A-42E8-8385-4E85168DBA4E@oracle.com> Sorry for the delay. The update looks good. Bob. > On Mar 25, 2019, at 1:30 PM, Severin Gehwolf wrote: > > On Fri, 2019-03-22 at 14:25 -0400, Bob Vandette wrote: >> Could you maybe combine subsystem_file_contents with subsystem_file_line_contents >> by adding an additional argument? > > Done: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8217338/05/webrev/ > > Thanks, > Severin > From sgehwolf at redhat.com Thu Mar 28 15:59:22 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 28 Mar 2019 16:59:22 +0100 Subject: RFR: 8217338: [Containers] Improve systemd slice memory limit support In-Reply-To: <226CB44C-7B3A-42E8-8385-4E85168DBA4E@oracle.com> References: <4a1ae4e0e3efd68818bfe868349972aa878de60b.camel@redhat.com> <9a911da4b2a4a8eff42c7c2f1a19c06f6521da8d.camel@redhat.com> <226CB44C-7B3A-42E8-8385-4E85168DBA4E@oracle.com> Message-ID: <014ebe0ced3e94cd5563e19e87a337f686338248.camel@redhat.com> On Thu, 2019-03-28 at 11:37 -0400, Bob Vandette wrote: > Sorry for the delay. The update looks good. Thanks for the review, Bob. Any other Reviewer(s)? Thanks, Severin > Bob. > > > > On Mar 25, 2019, at 1:30 PM, Severin Gehwolf wrote: > > > > On Fri, 2019-03-22 at 14:25 -0400, Bob Vandette wrote: > > > Could you maybe combine subsystem_file_contents with subsystem_file_line_contents > > > by adding an additional argument? > > > > Done: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8217338/05/webrev/ > > > > Thanks, > > Severin > > From matthias.baesken at sap.com Thu Mar 28 16:12:39 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 28 Mar 2019 16:12:39 +0000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: <30b2b3c5-38f7-8569-10cd-e781129b5da5@oracle.com> References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> <30b2b3c5-38f7-8569-10cd-e781129b5da5@oracle.com> Message-ID: > > which calls > > VM_Version::print_platform_virtualization_info > > which you have defined for s390 and PPC, but I don't see it for other > architectures? Hi David , we have in class Abstract_VM_Version static void print_detected_virtualization(outputStream*); http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/src/hotspot/share/runtime/vm_version.hpp.frames.html ( same approach that is used for other Abstract_VM_Version methods like ..... Platforms that 83 // need to specialize this define VM_Version::early_initialize(). 84 static void early_initialize() { } ) > > Are the other platforms handled by the switch on > VM_Version::get_detected_virtualization() ? Why isn't that encapsulated > in print_platform_virtualization_info for those platforms (which I think > is only x86 at this stage?). > Yes (and true, it is x86 only because all the other platforms have _detected_virtualization = NoDetectedVirtualization; ). Best regards , Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 28. M?rz 2019 08:21 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Cc: Doerr, Martin > Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > hs_error file on linux/windows x86_64 > > On 26/03/2019 6:32 pm, Baesken, Matthias wrote: > > Hi David, > > > >> This is looking better though I'm struggling to keep all the dots > >> connected in terms of the APIs - I can't quite see the pattern for what > >> calls what and what overrides what. > > > > I just followed here what is done at other places of VM_Version / > Abstract_VM_Version , see > > for example : > > Sorry but I'm having trouble following all the levels of indirection. > You have: > > VM_version::print_detected_virtualization > > which calls > > VM_Version::print_platform_virtualization_info > > which you have defined for s390 and PPC, but I don't see it for other > architectures? > > Then you have os::Linux and os::Windows print_virtualization_info that > then calls VM_version::print_detected_virtualization > > Are the other platforms handled by the switch on > VM_Version::get_detected_virtualization() ? Why isn't that encapsulated > in print_platform_virtualization_info for those platforms (which I think > is only x86 at this stage?). > > I'm finding it extremely hard to see the clear shape of this API - sorry. > > David > ----- > > > src/hotspot/share/runtime/vm_version.hpp > > > > 82 // ..... Platforms that > > 83 // need to specialize this define VM_Version::early_initialize(). > > 84 static void early_initialize() { } > > 85 > > 86 // Called to initialize VM variables needing initialization > > 87 // after command line parsing. Platforms that need to specialize > > 88 // this should define VM_Version::init_before_ergo(). > > 89 static void init_before_ergo() {} > > > > > >> 2. os_windows.cpp > >> > >> Why the indirection through os::win32::print_virtualization_info instead > >> of just calling VM_Version::print_detected_virtualization directly? > > > > Yes that's a good point - I think I should change this . > > > > Best regards, Matthias > > > > > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Mittwoch, 20. M?rz 2019 07:15 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' > >> Cc: Doerr, Martin > >> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > >> hs_error file on linux/windows x86_64 > >> > >> Hi Matthias, > >> > >> On 19/03/2019 11:53 pm, Baesken, Matthias wrote: > >>> Hello, here is a new webrev : > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ > >>> > >>> This moved the ppc/s390 virtualization related info output into the > >> platform/cpu - specific coding . > >> > >> This is looking better though I'm struggling to keep all the dots > >> connected in terms of the APIs - I can't quite see the pattern for what > >> calls what and what overrides what. A couple of initial comments: > >> > >> 1. We already have a ton of cpuid related parsing in > >> ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check > >> virtualizations" and store the information for later use? > >> > >> 2. os_windows.cpp > >> > >> Why the indirection through os::win32::print_virtualization_info instead > >> of just calling VM_Version::print_detected_virtualization directly? > >> > >> I'll keep trying to piece this together. :) > >> > >> Thanks, > >> David > >> ----- > >> > >>> > >>> Best regards, Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Montag, 11. M?rz 2019 08:36 > >>>> To: Baesken, Matthias ; 'hotspot- > >>>> dev at openjdk.java.net' > >>>> Cc: Doerr, Martin > >>>> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the > >>>> hs_error file on linux/windows x86_64 > >>>> > >>>> On 7/03/2019 3:13 am, Baesken, Matthias wrote: > >>>>> Hello , could I push the latest version of 8219241 ? > >>>> > >>>> Sorry based on your earlier email I thought you were looking at doing > >>>> the restructuring. I was then on vacation most of last week. > >>>> > >>>>> A change that moves the coding more into platform/cpu - specific > >> coding > >>>> (if this is wanted) > >>>>> could be done when bringing in the AIX virtualization related info > in > >>>> another patch which I plan to do . > >>>> > >>>> If that is imminent then okay - I don't like the code as it is. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>> From aph at redhat.com Thu Mar 28 16:23:07 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 28 Mar 2019 16:23:07 +0000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> Message-ID: <1c95a02c-2660-097a-4e98-3f6d674f7a34@redhat.com> On 3/28/19 3:22 PM, Thomas St?fe wrote: > On Thu, Mar 28, 2019 at 3:41 PM Andrew Dinn wrote: > >> s/iff/if >> The second of those was actually intended to be iff. This is a common >> abbreviation used by English/US mathematicians and logicians to write >> 'if and only if' (it is also sometimes written as <=>). Since you didn't >> recognize it I guess I really need to write it out in full. > > Oh, don't worry on my account. I am not a native speaker nor a > mathematician. You could leave iff and add [sic] to make everyone curious > and start googling "iff" :) Dijkstra: The notation iff is used for "if and only if". A few years ago, while lecturing in Denmark, I used Fif instead, reasoning that since "if and only if" was a symmetric concept its notation should be symmetric also. Without knowing it, I had punned in Danish and the audience laughed, for fif in Danish means "a little trick". I resolved thereafter to use fif so I could tell my joke, but my colleagues talked me out of it. :-) -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Thu Mar 28 16:26:04 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 28 Mar 2019 16:26:04 +0000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: <1c95a02c-2660-097a-4e98-3f6d674f7a34@redhat.com> References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> <1c95a02c-2660-097a-4e98-3f6d674f7a34@redhat.com> Message-ID: <708cf75c-5e0c-13da-b6b3-c252bda8579a@redhat.com> On 3/28/19 4:23 PM, Andrew Haley wrote: > Dijkstra: > > The notation iff is used for "if and only if". A few years ago, > while lecturing in Denmark, I used Fif instead, reasoning that since > "if and only if" was a symmetric concept its notation should be > symmetric also. Without knowing it, I had punned in Danish and the > audience laughed, for fif in Danish means "a little trick". I > resolved thereafter to use fif so I could tell my joke, but my > colleagues talked me out of it. Arggh! Sorry, that was Gries, not Dijkstra. I misremembered. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From adinn at redhat.com Thu Mar 28 16:56:06 2019 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 28 Mar 2019 16:56:06 +0000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> Message-ID: <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> On 28/03/2019 15:22, Thomas St?fe wrote: > The second of those was actually intended to be iff. This is a common > abbreviation used by English/US mathematicians and logicians to write > 'if and only if' (it is also sometimes written as <=>). Since you didn't > recognize it I guess I really need to write it out in full. > > > Oh, don't worry on my account. I am not a native speaker nor a > mathematician. You could leave iff and add [sic] to make everyone > curious and start googling "iff" :) I changed it nevertheless. It would be remiss to presume readers need be conversant with elliptical, English logico-mathematical parlance (I'm explaining this in as plain English as I can ... no, wait! ;-). Anyway, I addressed this and your other concerns in a new webrev. JIRA: https://bugs.openjdk.java.net/browse/JDK-8221477 Webrev: http://cr.openjdk.java.net/~adinn/8221477/webrev.02 Input from a second reviewer would be welcome ... regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From thomas.stuefe at gmail.com Thu Mar 28 17:09:22 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 28 Mar 2019 18:09:22 +0100 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> Message-ID: On Thu, Mar 28, 2019 at 5:56 PM Andrew Dinn wrote: > On 28/03/2019 15:22, Thomas St?fe wrote: > > The second of those was actually intended to be iff. This is a common > > abbreviation used by English/US mathematicians and logicians to write > > 'if and only if' (it is also sometimes written as <=>). Since you > didn't > > recognize it I guess I really need to write it out in full. > > > > > > Oh, don't worry on my account. I am not a native speaker nor a > > mathematician. You could leave iff and add [sic] to make everyone > > curious and start googling "iff" :) > I changed it nevertheless. It would be remiss to presume readers need be > conversant with elliptical, English logico-mathematical parlance (I'm > explaining this in as plain English as I can ... no, wait! ;-). > > Anyway, I addressed this and your other concerns in a new webrev. > > JIRA: https://bugs.openjdk.java.net/browse/JDK-8221477 > Webrev: http://cr.openjdk.java.net/~adinn/8221477/webrev.02 > > src/hotspot/share/classfile/javaClasses.cpp Pure nitpicking, but you could remove the member variables too and just set the Unsafe members directly, e.g.: if (fd->name() == vmSymbols::address_size_name()) { mirror->int_field_put(fd->offset(), (jint)sizeof(void*)); } else if (fd->name() == vmSymbols::page_size_name()) { mirror->int_field_put(fd->offset(), (jint) os::vm_page_size()); .. and so on. But I leave this up to you. This is already better than before. + }else { space missing before else -- src/hotspot/share/classfile/vmSymbols.hpp "UNALIGNED_ACCESS" off by one space --- src/java.base/share/classes/java/nio/channels/FileChannel.java This may have creeped in from another change. --- Good otherwise. Cheers, Thomas > Input from a second reviewer would be welcome ... > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From naoto.sato at oracle.com Thu Mar 28 15:59:28 2019 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 28 Mar 2019 08:59:28 -0700 Subject: RFC: Make test failed because of the locale LANG In-Reply-To: <396490c6-9917-2306-c288-2de5ec92bdc9@oracle.com> References: <5fd4d748-c9cf-3779-3a65-b7c7547ab9ce@loongson.cn> <396490c6-9917-2306-c288-2de5ec92bdc9@oracle.com> Message-ID: Hi, I don't think there is any *official* rule for the regression tests to succeed in any locale, but if the test case is locale sensitive such as in this case, I'd suggest it should correctly specify the locale beforehand, or quit gracefully in general. For this specific case, I'd suggest not set the environment variable (as it won't work with Windows), using runtime system properties (user.language/user.country) would help. Naoto On 3/27/19 10:47 PM, David Holmes wrote: > Hi Jing, > > On 28/03/2019 3:22 pm, Jing Tian wrote: >> Hi, >> >> When I am doing the 'make test'.If the local LANG is set as >> 'zh_CN.UTF-8', Test cases will have a lot of error messages. >> >> ============================== >> Test summary >> ============================== >> ??? TEST????????????????????????????????????????????? TOTAL? PASS FAIL >> ERROR >> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1373 1371 2 >> 0 << >> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1860 7 >> 0 << >> ?>> jtreg:test/langtools:tier1???????????????????????? 3922 2470 >> 1450???? 2 << >> ??? jtreg:test/nashorn:tier1????????????????????????????? 0 0???? 0???? 0 >> ??? jtreg:test/jaxp:tier1???????????????????????????????? 0 0???? 0???? 0 >> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3334 3305 29 0 << >> ?>> jtreg:test/langtools:tier2 11???? 9???? 2???? 0 << >> ??? jtreg:test/nashorn:tier2???????????????????????????? 35 35 >> 0???? 0 >> ?>> jtreg:test/jaxp:tier2?????????????????????????????? 438 437 1 >> 0 << >> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1068 36 0 << >> ??? jtreg:test/langtools:tier3??????????????????????????? 0 0???? 0???? 0 >> ??? jtreg:test/nashorn:tier3????????????????????????????? 0 0???? 0???? 0 >> ??? jtreg:test/jaxp:tier3???????????????????????????????? 0 0???? 0???? 0 >> ============================== > > Given most of these are not hotspot issues and given the number of > failures, I think this is something that would need to be discussed much > more broadly. So I've cc'd core-libs-dev and compiler-dev. I recall > there was a very recent discussion regarding some tests failing for the > same reason, but don't recall the outcome. > > David > ----- > >> On the same machine,when i set LANG=en_US.UTF-8. >> >> ============================== >> Test summary >> ============================== >> ??? TEST????????????????????????????????????????????? TOTAL? PASS FAIL >> ERROR >> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1388 1386 2 >> 0 << >> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1843 19 5 << >> ??? jtreg:test/langtools:tier1???????????????????????? 3920 3920 0???? 0 >> ??? jtreg:test/nashorn:tier1????????????????????????????? 0 0???? 0???? 0 >> ??? jtreg:test/jaxp:tier1???????????????????????????????? 0 0???? 0???? 0 >> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3328 3290 31 7 << >> ??? jtreg:test/langtools:tier2?????????????????????????? 11 11 >> 0???? 0 >> ??? jtreg:test/nashorn:tier2???????????????????????????? 35 35 >> 0???? 0 >> ??? jtreg:test/jaxp:tier2?????????????????????????????? 438 438 >> 0???? 0 >> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1067 37 0 << >> ??? jtreg:test/langtools:tier3??????????????????????????? 0 0???? 0???? 0 >> ??? jtreg:test/nashorn:tier3????????????????????????????? 0 0???? 0???? 0 >> ??? jtreg:test/jaxp:tier3???????????????????????????????? 0 0???? 0???? 0 >> >> >> By comparison we can find, lots of(1000+) langtools tests will get >> fail, and other(about 30+, exclude some X11 and sanity that >> >> result problem) test cases will also fail because of local LANG. >> >> >> such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java, >> >> shouldContain("Error: Could not find or load main class " + CLASSNAME) >> >> >> When in the zh_CN.UTF-8 environment, because some of the output >> information is changed to Chinese by some properties file, >> >> the English cannot be matched, which will result in an fail. >> >> When I set? LANG=en_US/LC_ALL=C, this test will pass. >> >> >> I think there are some possible solutions. >> >> >> 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this >> modification is more violent because he will affect all test cases. >> >> >> 2.We modify each individual test?E.g >> test/hotspot/jtreg/compiler/c2/Test8062950.java >> >> diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java >> ??package compiler.c2; >> ??import jdk.test.lib.process.ProcessTools; >> +import java.util.Map; >> +import jdk.test.lib.process.OutputAnalyzer; >> >> ??public class Test8062950 { >> ????? private static final String CLASSNAME = "DoesNotExist"; >> ????? public static void main(String[] args) throws Exception { >> - ProcessTools.executeTestJvm("-Xcomp", >> - "-XX:-TieredCompilation", >> - "-XX:-UseOptoBiasInlining", >> - CLASSNAME) >> - .shouldHaveExitValue(1) >> -??????????????????? .shouldContain("Error: Could not find or load >> main class " + CLASSNAME) >> -??????????????????? .shouldNotContain("A fatal error has been detected") >> -??????????????????? .shouldNotContain("Internal Error") >> -??????????????????? .shouldNotContain("HotSpot Virtual Machine Error"); >> +??????? final ProcessBuilder pb = >> ProcessTools.createJavaProcessBuilder(true, >> + "-Xcomp", >> + "-XX:-TieredCompilation", >> + "-XX:-UseOptoBiasInlining", >> + CLASSNAME); >> +??????? final Map env = pb.environment(); >> +??????? env.put("LC_ALL", "en_US.UTF-8"); >> +??????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); >> + output.shouldHaveExitValue(1); >> +??????? output.shouldContain("Error: Could not find or load main >> class " + CLASSNAME); >> +??????? output.shouldNotContain("A fatal error has been detected"); >> +??????? output.shouldNotContain("Internal Error"); >> +??????? output.shouldNotContain("HotSpot Virtual Machine Error"); >> } >> ??} >> >> But I don't know if this will change the test program too much, >> because the related problems are a lot in langtools tests. >> >> >> 3.And i find that there is a function can judge the locale >> >> ??if (!isEnglishLocale()) { // only english version >> return; >> ??} >> >> But in this case, I think in many test cases, we may not be able to >> return so directly, because some test cases may have other test purposes. >> >> >> So I don't know if you have any ideas or some suggestions to solve the >> problem that the output information and English do not match in this >> >> non-English environment. >> >> >> Cheers, >> >> Jing Tian >> >> >> >> From jcbeyler at google.com Thu Mar 28 18:12:24 2019 From: jcbeyler at google.com (Jean Christophe Beyler) Date: Thu, 28 Mar 2019 11:12:24 -0700 Subject: RFR 8157372: C2: Node::cmp() should return bool In-Reply-To: References: Message-ID: Hi all, I pushed it for Derek. Have a great day, Jc On Mon, Mar 25, 2019 at 4:11 PM Vladimir Ivanov < vladimir.x.ivanov at oracle.com> wrote: > Looks good. > > Best regards, > Vladimir Ivanov > > On 19/03/2019 11:08, Derek Thomson wrote: > > Changed uint to bool throughout. Could someone review my change please? > > Also, what tests should I run to validate this? > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8157372/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8157372 > > > -- Thanks, Jc From vladimir.kozlov at oracle.com Thu Mar 28 19:15:31 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 28 Mar 2019 12:15:31 -0700 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library Message-ID: https://bugs.openjdk.java.net/browse/JDK-8220623 http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ Update JVMCI to support pre-compiled as shared library Graal. Using aoted Graal can offers benefits including: - fast startup - compile time similar to native JIt compilers (C2) - memory usage disjoint from the application Java heap - no profile pollution of JDK code used by the application This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. Changes were collected in Metropolis repo [2] and tested there. Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler group. Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In this set Graal was tested only in tier3. And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our system. Several issue were found which were present before these changes. Thanks, Vladimir [1] https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af [2] http://hg.openjdk.java.net/metropolis/dev/ From doug.simon at oracle.com Thu Mar 28 19:32:23 2019 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 28 Mar 2019 20:32:23 +0100 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: Message-ID: Not a Reviewer, but I like it! ;-) -Doug > On 28 Mar 2019, at 20:15, Vladimir Kozlov wrote: > > https://bugs.openjdk.java.net/browse/JDK-8220623 > http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ > > Update JVMCI to support pre-compiled as shared library Graal. > Using aoted Graal can offers benefits including: > - fast startup > - compile time similar to native JIt compilers (C2) > - memory usage disjoint from the application Java heap > - no profile pollution of JDK code used by the application > > This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. > Changes were collected in Metropolis repo [2] and tested there. > > Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler group. > Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. > > I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In this set Graal was tested only in tier3. > > And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our system. Several issue were found which were present before these changes. > > Thanks, > Vladimir > > [1] https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af > [2] http://hg.openjdk.java.net/metropolis/dev/ From stefan.karlsson at oracle.com Thu Mar 28 19:54:19 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 28 Mar 2019 20:54:19 +0100 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: Message-ID: Hi Vladimir, I started to check the GC code. ======================================================================== I see that you've added guarded includes in the middle of the include list: ? #include "gc/shared/strongRootsScope.hpp" ? #include "gc/shared/weakProcessor.hpp" + #if INCLUDE_JVMCI + #include "jvmci/jvmci.hpp" + #endif ? #include "oops/instanceRefKlass.hpp" ? #include "oops/oop.inline.hpp" The style we use is to put these conditional includes at the end of the include lists. ======================================================================== Could you also change the following: + #if INCLUDE_JVMCI +???? // Clean JVMCI metadata handles. +???? JVMCI::do_unloading(is_alive_closure(), purged_class); + #endif to: +???? // Clean JVMCI metadata handles. +???? JVMCI_ONLY(JVMCI::do_unloading(is_alive_closure(), purged_class);) to get rid of some of the line noise in the GC files. ======================================================================== In the future we will need version of JVMCI::do_unloading that supports concurrent cleaning for ZGC. ======================================================================== What's the performance impact for G1 remark pause with this serial walk over the MetadataHandleBlock? 3275 void G1CollectedHeap::complete_cleaning(BoolObjectClosure* is_alive, 3276???????????????????????????????????????? bool class_unloading_occurred) { 3277?? uint num_workers = workers()->active_workers(); 3278?? ParallelCleaningTask unlink_task(is_alive, num_workers, class_unloading_occurred, false); 3279?? workers()->run_task(&unlink_task); 3280 #if INCLUDE_JVMCI 3281?? // No parallel processing of JVMCI metadata handles for now. 3282?? JVMCI::do_unloading(is_alive, class_unloading_occurred); 3283 #endif 3284 } ======================================================================== Did you consider adding it as a task for one of the worker threads to execute in ParallelCleaningTask? See how other tasks are claimed by one worker: void KlassCleaningTask::work() { ? ResourceMark rm; ? // One worker will clean the subklass/sibling klass tree. ? if (claim_clean_klass_tree_task()) { ??? Klass::clean_subklass_tree(); ? } ======================================================================== In MetadataHandleBlock::do_unloading: +??????? if (klass->class_loader_data()->is_unloading()) { +????????? // This needs to be marked so that it's no longer scanned +????????? // but can't be put on the free list yet. The +????????? // ReferenceCleaner will set this to NULL and +????????? // put it on the free list. I couldn't find the ReferenceCleaner in the patch or in the source. Where can I find this code? Thanks, StefanK On 2019-03-28 20:15, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8220623 > http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ > > Update JVMCI to support pre-compiled as shared library Graal. > Using aoted Graal can offers benefits including: > ?- fast startup > ?- compile time similar to native JIt compilers (C2) > ?- memory usage disjoint from the application Java heap > ?- no profile pollution of JDK code used by the application > > This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. > Changes were collected in Metropolis repo [2] and tested there. > > Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and > our compiler group. > Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. > > I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was > clean. In this set Graal was tested only in tier3. > > And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available > in our system. Several issue were found which were present before > these changes. > > Thanks, > Vladimir > > [1] > https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af > [2] http://hg.openjdk.java.net/metropolis/dev/ From vladimir.kozlov at oracle.com Thu Mar 28 21:44:13 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 28 Mar 2019 14:44:13 -0700 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: Message-ID: Thank you, Stefan On 3/28/19 12:54 PM, Stefan Karlsson wrote: > Hi Vladimir, > > I started to check the GC code. > > ======================================================================== > I see that you've added guarded includes in the middle of the include list: > ? #include "gc/shared/strongRootsScope.hpp" > ? #include "gc/shared/weakProcessor.hpp" > + #if INCLUDE_JVMCI > + #include "jvmci/jvmci.hpp" > + #endif > ? #include "oops/instanceRefKlass.hpp" > ? #include "oops/oop.inline.hpp" > > The style we use is to put these conditional includes at the end of the include lists. okay > > ======================================================================== > Could you also change the following: > > + #if INCLUDE_JVMCI > +???? // Clean JVMCI metadata handles. > +???? JVMCI::do_unloading(is_alive_closure(), purged_class); > + #endif > > to: > +???? // Clean JVMCI metadata handles. > +???? JVMCI_ONLY(JVMCI::do_unloading(is_alive_closure(), purged_class);) > > to get rid of some of the line noise in the GC files. okay > > ======================================================================== > In the future we will need version of JVMCI::do_unloading that supports concurrent cleaning for ZGC. Yes, we need to support concurrent cleaning in a future. > > ======================================================================== > What's the performance impact for G1 remark pause with this serial walk over the MetadataHandleBlock? > > 3275 void G1CollectedHeap::complete_cleaning(BoolObjectClosure* is_alive, > 3276???????????????????????????????????????? bool class_unloading_occurred) { > 3277?? uint num_workers = workers()->active_workers(); > 3278?? ParallelCleaningTask unlink_task(is_alive, num_workers, class_unloading_occurred, false); > 3279?? workers()->run_task(&unlink_task); > 3280 #if INCLUDE_JVMCI > 3281?? // No parallel processing of JVMCI metadata handles for now. > 3282?? JVMCI::do_unloading(is_alive, class_unloading_occurred); > 3283 #endif > 3284 } There should not be impact if Graal is not used. Only cost of call (which most likely is inlined in product VM) and check: http://hg.openjdk.java.net/metropolis/dev/file/530fc1427d02/src/hotspot/share/jvmci/jvmciRuntime.cpp#l1237 If Graal is used it should not have big impact since these metadata has regular pattern (32 handles per array and array per MetadataHandleBlock block which are linked in list) and not large. If there will be noticeable impact - we will work on it as you suggested by using ParallelCleaningTask. > > ======================================================================== > Did you consider adding it as a task for one of the worker threads to execute in ParallelCleaningTask? > > See how other tasks are claimed by one worker: > void KlassCleaningTask::work() { > ? ResourceMark rm; > > ? // One worker will clean the subklass/sibling klass tree. > ? if (claim_clean_klass_tree_task()) { > ??? Klass::clean_subklass_tree(); > ? } These changes were ported from JDK8u based changes in graal-jvmci-8 and there are no ParallelCleaningTask in JDK8. Your suggestion is interesting and I agree that we should investigate it. > > ======================================================================== > In MetadataHandleBlock::do_unloading: > > +??????? if (klass->class_loader_data()->is_unloading()) { > +????????? // This needs to be marked so that it's no longer scanned > +????????? // but can't be put on the free list yet. The > +????????? // ReferenceCleaner will set this to NULL and > +????????? // put it on the free list. > > I couldn't find the ReferenceCleaner in the patch or in the source. Where can I find this code? I think it is typo (I will fix it) - it references new HandleCleaner class: http://cr.openjdk.java.net/~kvn/8220623/webrev.03/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HandleCleaner.java.html Thanks, Vladimir > > Thanks, > StefanK > > On 2019-03-28 20:15, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8220623 >> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ >> >> Update JVMCI to support pre-compiled as shared library Graal. >> Using aoted Graal can offers benefits including: >> ?- fast startup >> ?- compile time similar to native JIt compilers (C2) >> ?- memory usage disjoint from the application Java heap >> ?- no profile pollution of JDK code used by the application >> >> This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. >> Changes were collected in Metropolis repo [2] and tested there. >> >> Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler group. >> Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. >> >> I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In this set Graal was >> tested only in tier3. >> >> And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our system. Several issue >> were found which were present before these changes. >> >> Thanks, >> Vladimir >> >> [1] https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af >> [2] http://hg.openjdk.java.net/metropolis/dev/ > From david.holmes at oracle.com Thu Mar 28 23:54:06 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 29 Mar 2019 09:54:06 +1000 Subject: RFC: Make test failed because of the locale LANG In-Reply-To: References: <5fd4d748-c9cf-3779-3a65-b7c7547ab9ce@loongson.cn> <396490c6-9917-2306-c288-2de5ec92bdc9@oracle.com> Message-ID: On 29/03/2019 1:59 am, Naoto Sato wrote: > Hi, > > I don't think there is any *official* rule for the regression tests to > succeed in any locale, but if the test case is locale sensitive such as > in this case, I'd suggest it should correctly specify the locale > beforehand, or quit gracefully in general. > > For this specific case, I'd suggest not set the environment variable (as > it won't work with Windows), using runtime system properties > (user.language/user.country) would help. Which specific case are you referring to? :) Given there are well over a thousand failures in total I think we need to set a reasonable initial locale as part of the test setup. Many of the hotspot tests check actual and expected output and we, in general, have no idea what kinds of output may be subject to locale specific changes. Cheers, David > Naoto > > On 3/27/19 10:47 PM, David Holmes wrote: >> Hi Jing, >> >> On 28/03/2019 3:22 pm, Jing Tian wrote: >>> Hi, >>> >>> When I am doing the 'make test'.If the local LANG is set as >>> 'zh_CN.UTF-8', Test cases will have a lot of error messages. >>> >>> ============================== >>> Test summary >>> ============================== >>> ??? TEST????????????????????????????????????????????? TOTAL? PASS >>> FAIL ERROR >>> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1373 1371 2 0 << >>> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1860 7 0 << >>> ?>> jtreg:test/langtools:tier1???????????????????????? 3922 2470 >>> 1450???? 2 << >>> ??? jtreg:test/nashorn:tier1????????????????????????????? 0 0 >>> 0???? 0 >>> ??? jtreg:test/jaxp:tier1???????????????????????????????? 0 0 >>> 0???? 0 >>> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3334 3305 29 0 << >>> ?>> jtreg:test/langtools:tier2 11???? 9???? 2???? 0 << >>> ??? jtreg:test/nashorn:tier2???????????????????????????? 35 35 0???? 0 >>> ?>> jtreg:test/jaxp:tier2?????????????????????????????? 438 437 1 0 << >>> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1068 36 0 << >>> ??? jtreg:test/langtools:tier3??????????????????????????? 0 0 >>> 0???? 0 >>> ??? jtreg:test/nashorn:tier3????????????????????????????? 0 0 >>> 0???? 0 >>> ??? jtreg:test/jaxp:tier3???????????????????????????????? 0 0 >>> 0???? 0 >>> ============================== >> >> Given most of these are not hotspot issues and given the number of >> failures, I think this is something that would need to be discussed >> much more broadly. So I've cc'd core-libs-dev and compiler-dev. I >> recall there was a very recent discussion regarding some tests failing >> for the same reason, but don't recall the outcome. >> >> David >> ----- >> >>> On the same machine,when i set LANG=en_US.UTF-8. >>> >>> ============================== >>> Test summary >>> ============================== >>> ??? TEST????????????????????????????????????????????? TOTAL? PASS >>> FAIL ERROR >>> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1388 1386 2 0 << >>> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1843 19 5 << >>> ??? jtreg:test/langtools:tier1???????????????????????? 3920 3920 0???? 0 >>> ??? jtreg:test/nashorn:tier1????????????????????????????? 0 0 >>> 0???? 0 >>> ??? jtreg:test/jaxp:tier1???????????????????????????????? 0 0 >>> 0???? 0 >>> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3328 3290 31 7 << >>> ??? jtreg:test/langtools:tier2?????????????????????????? 11 11 0???? 0 >>> ??? jtreg:test/nashorn:tier2???????????????????????????? 35 35 0???? 0 >>> ??? jtreg:test/jaxp:tier2?????????????????????????????? 438 438 0???? 0 >>> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1067 37 0 << >>> ??? jtreg:test/langtools:tier3??????????????????????????? 0 0 >>> 0???? 0 >>> ??? jtreg:test/nashorn:tier3????????????????????????????? 0 0 >>> 0???? 0 >>> ??? jtreg:test/jaxp:tier3???????????????????????????????? 0 0 >>> 0???? 0 >>> >>> >>> By comparison we can find, lots of(1000+) langtools tests will get >>> fail, and other(about 30+, exclude some X11 and sanity that >>> >>> result problem) test cases will also fail because of local LANG. >>> >>> >>> such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java, >>> >>> shouldContain("Error: Could not find or load main class " + CLASSNAME) >>> >>> >>> When in the zh_CN.UTF-8 environment, because some of the output >>> information is changed to Chinese by some properties file, >>> >>> the English cannot be matched, which will result in an fail. >>> >>> When I set? LANG=en_US/LC_ALL=C, this test will pass. >>> >>> >>> I think there are some possible solutions. >>> >>> >>> 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this >>> modification is more violent because he will affect all test cases. >>> >>> >>> 2.We modify each individual test?E.g >>> test/hotspot/jtreg/compiler/c2/Test8062950.java >>> >>> diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java >>> ??package compiler.c2; >>> ??import jdk.test.lib.process.ProcessTools; >>> +import java.util.Map; >>> +import jdk.test.lib.process.OutputAnalyzer; >>> >>> ??public class Test8062950 { >>> ????? private static final String CLASSNAME = "DoesNotExist"; >>> ????? public static void main(String[] args) throws Exception { >>> - ProcessTools.executeTestJvm("-Xcomp", >>> - "-XX:-TieredCompilation", >>> - "-XX:-UseOptoBiasInlining", >>> - CLASSNAME) >>> - .shouldHaveExitValue(1) >>> -??????????????????? .shouldContain("Error: Could not find or load >>> main class " + CLASSNAME) >>> -??????????????????? .shouldNotContain("A fatal error has been >>> detected") >>> -??????????????????? .shouldNotContain("Internal Error") >>> -??????????????????? .shouldNotContain("HotSpot Virtual Machine Error"); >>> +??????? final ProcessBuilder pb = >>> ProcessTools.createJavaProcessBuilder(true, >>> + "-Xcomp", >>> + "-XX:-TieredCompilation", >>> + "-XX:-UseOptoBiasInlining", >>> + CLASSNAME); >>> +??????? final Map env = pb.environment(); >>> +??????? env.put("LC_ALL", "en_US.UTF-8"); >>> +??????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>> + output.shouldHaveExitValue(1); >>> +??????? output.shouldContain("Error: Could not find or load main >>> class " + CLASSNAME); >>> +??????? output.shouldNotContain("A fatal error has been detected"); >>> +??????? output.shouldNotContain("Internal Error"); >>> +??????? output.shouldNotContain("HotSpot Virtual Machine Error"); >>> } >>> ??} >>> >>> But I don't know if this will change the test program too much, >>> because the related problems are a lot in langtools tests. >>> >>> >>> 3.And i find that there is a function can judge the locale >>> >>> ??if (!isEnglishLocale()) { // only english version >>> return; >>> ??} >>> >>> But in this case, I think in many test cases, we may not be able to >>> return so directly, because some test cases may have other test >>> purposes. >>> >>> >>> So I don't know if you have any ideas or some suggestions to solve >>> the problem that the output information and English do not match in this >>> >>> non-English environment. >>> >>> >>> Cheers, >>> >>> Jing Tian >>> >>> >>> >>> From david.holmes at oracle.com Fri Mar 29 00:15:48 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 29 Mar 2019 10:15:48 +1000 Subject: RFR: 8219241: Provide basic virtualization related info in the hs_error file on linux/windows x86_64 In-Reply-To: References: <98b6c488-a935-9b43-7382-5bfc732f0ac7@oracle.com> <2f68463c-be1c-9d34-6202-72613cb7438d@oracle.com> <2c61ef7c-91f0-9636-5567-eaf8bbff7c94@oracle.com> <30b2b3c5-38f7-8569-10cd-e781129b5da5@oracle.com> Message-ID: Hi Matthias, I have two issues with this API: 1. Why do we have/need os::::print_virtualization_info when we also have VM_Version::print_platform_virtualization_info? 2. I don't like the fact that the there are two ways to define the platform specific information: a) override VM_Version::print_platform_virtualization_info; or b) hook into the get_detected_virtualization switch And IIUC code that uses (a) relies on the switch doing nothing (NoVirtualization) and code that uses (b) relies on print_platform_virtualization_info doing nothing! Why doesn't the default implementation of VM_Version::print_platform_virtualization_info define the get_detected_virtualization() switch logic, and do away with Abstract_VM_Version::print_detected_virtualization? Code that wants to print this info can just call VM_Version::print_platform_virtualization_info(). Sorry but I just find the current proposal has too many methods and an unclear structure. David ----- On 29/03/2019 2:12 am, Baesken, Matthias wrote: >> >> which calls >> >> VM_Version::print_platform_virtualization_info >> >> which you have defined for s390 and PPC, but I don't see it for other >> architectures? > > > Hi David , we have in class Abstract_VM_Version > > static void print_detected_virtualization(outputStream*); > > http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/src/hotspot/share/runtime/vm_version.hpp.frames.html > > > ( same approach that is used for other Abstract_VM_Version methods like > > ..... Platforms that > 83 // need to specialize this define VM_Version::early_initialize(). > 84 static void early_initialize() { } > > ) > >> >> Are the other platforms handled by the switch on >> VM_Version::get_detected_virtualization() ? Why isn't that encapsulated >> in print_platform_virtualization_info for those platforms (which I think >> is only x86 at this stage?). >> > > Yes (and true, it is x86 only because all the other platforms have _detected_virtualization = NoDetectedVirtualization; ). > > > > Best regards , Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 28. M?rz 2019 08:21 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' >> Cc: Doerr, Martin >> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >> hs_error file on linux/windows x86_64 >> >> On 26/03/2019 6:32 pm, Baesken, Matthias wrote: >>> Hi David, >>> >>>> This is looking better though I'm struggling to keep all the dots >>>> connected in terms of the APIs - I can't quite see the pattern for what >>>> calls what and what overrides what. >>> >>> I just followed here what is done at other places of VM_Version / >> Abstract_VM_Version , see >>> for example : >> >> Sorry but I'm having trouble following all the levels of indirection. >> You have: >> >> VM_version::print_detected_virtualization >> >> which calls >> >> VM_Version::print_platform_virtualization_info >> >> which you have defined for s390 and PPC, but I don't see it for other >> architectures? >> >> Then you have os::Linux and os::Windows print_virtualization_info that >> then calls VM_version::print_detected_virtualization >> >> Are the other platforms handled by the switch on >> VM_Version::get_detected_virtualization() ? Why isn't that encapsulated >> in print_platform_virtualization_info for those platforms (which I think >> is only x86 at this stage?). >> >> I'm finding it extremely hard to see the clear shape of this API - sorry. >> >> David >> ----- >> >>> src/hotspot/share/runtime/vm_version.hpp >>> >>> 82 // ..... Platforms that >>> 83 // need to specialize this define VM_Version::early_initialize(). >>> 84 static void early_initialize() { } >>> 85 >>> 86 // Called to initialize VM variables needing initialization >>> 87 // after command line parsing. Platforms that need to specialize >>> 88 // this should define VM_Version::init_before_ergo(). >>> 89 static void init_before_ergo() {} >>> >>> >>>> 2. os_windows.cpp >>>> >>>> Why the indirection through os::win32::print_virtualization_info instead >>>> of just calling VM_Version::print_detected_virtualization directly? >>> >>> Yes that's a good point - I think I should change this . >>> >>> Best regards, Matthias >>> >>> >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Mittwoch, 20. M?rz 2019 07:15 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' >>>> Cc: Doerr, Martin >>>> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >>>> hs_error file on linux/windows x86_64 >>>> >>>> Hi Matthias, >>>> >>>> On 19/03/2019 11:53 pm, Baesken, Matthias wrote: >>>>> Hello, here is a new webrev : >>>>> >>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8219241.3/ >>>>> >>>>> This moved the ppc/s390 virtualization related info output into the >>>> platform/cpu - specific coding . >>>> >>>> This is looking better though I'm struggling to keep all the dots >>>> connected in terms of the APIs - I can't quite see the pattern for what >>>> calls what and what overrides what. A couple of initial comments: >>>> >>>> 1. We already have a ton of cpuid related parsing in >>>> ./cpu/x86/vm_version_x86.hpp - can't you hook into that to "check >>>> virtualizations" and store the information for later use? >>>> >>>> 2. os_windows.cpp >>>> >>>> Why the indirection through os::win32::print_virtualization_info instead >>>> of just calling VM_Version::print_detected_virtualization directly? >>>> >>>> I'll keep trying to piece this together. :) >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> Best regards, Matthias >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Montag, 11. M?rz 2019 08:36 >>>>>> To: Baesken, Matthias ; 'hotspot- >>>>>> dev at openjdk.java.net' >>>>>> Cc: Doerr, Martin >>>>>> Subject: Re: RFR: 8219241: Provide basic virtualization related info in the >>>>>> hs_error file on linux/windows x86_64 >>>>>> >>>>>> On 7/03/2019 3:13 am, Baesken, Matthias wrote: >>>>>>> Hello , could I push the latest version of 8219241 ? >>>>>> >>>>>> Sorry based on your earlier email I thought you were looking at doing >>>>>> the restructuring. I was then on vacation most of last week. >>>>>> >>>>>>> A change that moves the coding more into platform/cpu - specific >>>> coding >>>>>> (if this is wanted) >>>>>>> could be done when bringing in the AIX virtualization related info >> in >>>>>> another patch which I plan to do . >>>>>> >>>>>> If that is imminent then okay - I don't like the code as it is. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>> From naoto.sato at oracle.com Fri Mar 29 01:23:29 2019 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Thu, 28 Mar 2019 18:23:29 -0700 Subject: RFC: Make test failed because of the locale LANG In-Reply-To: References: <5fd4d748-c9cf-3779-3a65-b7c7547ab9ce@loongson.cn> <396490c6-9917-2306-c288-2de5ec92bdc9@oracle.com> Message-ID: <794486e3-0a12-7605-2be4-abd3916d030f@oracle.com> On 3/28/19 4:54 PM, David Holmes wrote: > On 29/03/2019 1:59 am, Naoto Sato wrote: >> Hi, >> >> I don't think there is any *official* rule for the regression tests to >> succeed in any locale, but if the test case is locale sensitive such >> as in this case, I'd suggest it should correctly specify the locale >> beforehand, or quit gracefully in general. >> >> For this specific case, I'd suggest not set the environment variable >> (as it won't work with Windows), using runtime system properties >> (user.language/user.country) would help. > > Which specific case are you referring to? :) I was referring to the test case below, Test8062950.java, where it issues "env.put("LC_ALL", "en_US.UTF-8")". Instead I would recommend "-Duser.language=en -Duser.country=US" runtime system property. > > Given there are well over a thousand failures in total I think we need > to set a reasonable initial locale as part of the test setup. Many of > the hotspot tests check actual and expected output and we, in general, > have no idea what kinds of output may be subject to locale specific > changes. Agree. Specifying a reasonable default locale would not be a bad idea. Naoto > > Cheers, > David > >> Naoto >> >> On 3/27/19 10:47 PM, David Holmes wrote: >>> Hi Jing, >>> >>> On 28/03/2019 3:22 pm, Jing Tian wrote: >>>> Hi, >>>> >>>> When I am doing the 'make test'.If the local LANG is set as >>>> 'zh_CN.UTF-8', Test cases will have a lot of error messages. >>>> >>>> ============================== >>>> Test summary >>>> ============================== >>>> ??? TEST????????????????????????????????????????????? TOTAL? PASS >>>> FAIL ERROR >>>> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1373 1371 2 0 << >>>> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1860 7 0 << >>>> ?>> jtreg:test/langtools:tier1???????????????????????? 3922 2470 >>>> 1450???? 2 << >>>> ??? jtreg:test/nashorn:tier1????????????????????????????? 0 0 0???? 0 >>>> ??? jtreg:test/jaxp:tier1???????????????????????????????? 0 0 0???? 0 >>>> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3334 3305 29 >>>> 0 << >>>> ?>> jtreg:test/langtools:tier2 11???? 9???? 2???? 0 << >>>> ??? jtreg:test/nashorn:tier2???????????????????????????? 35 35 0???? 0 >>>> ?>> jtreg:test/jaxp:tier2?????????????????????????????? 438 437 1 0 << >>>> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1068 36 >>>> 0 << >>>> ??? jtreg:test/langtools:tier3??????????????????????????? 0 0 0???? 0 >>>> ??? jtreg:test/nashorn:tier3????????????????????????????? 0 0 0???? 0 >>>> ??? jtreg:test/jaxp:tier3???????????????????????????????? 0 0 0???? 0 >>>> ============================== >>> >>> Given most of these are not hotspot issues and given the number of >>> failures, I think this is something that would need to be discussed >>> much more broadly. So I've cc'd core-libs-dev and compiler-dev. I >>> recall there was a very recent discussion regarding some tests >>> failing for the same reason, but don't recall the outcome. >>> >>> David >>> ----- >>> >>>> On the same machine,when i set LANG=en_US.UTF-8. >>>> >>>> ============================== >>>> Test summary >>>> ============================== >>>> ??? TEST????????????????????????????????????????????? TOTAL? PASS >>>> FAIL ERROR >>>> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1388 1386 2 0 << >>>> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1843 19 >>>> 5 << >>>> ??? jtreg:test/langtools:tier1???????????????????????? 3920 3920 >>>> 0???? 0 >>>> ??? jtreg:test/nashorn:tier1????????????????????????????? 0 0 0???? 0 >>>> ??? jtreg:test/jaxp:tier1???????????????????????????????? 0 0 0???? 0 >>>> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3328 3290 31 >>>> 7 << >>>> ??? jtreg:test/langtools:tier2?????????????????????????? 11 11 0???? 0 >>>> ??? jtreg:test/nashorn:tier2???????????????????????????? 35 35 0???? 0 >>>> ??? jtreg:test/jaxp:tier2?????????????????????????????? 438 438 0???? 0 >>>> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1067 37 >>>> 0 << >>>> ??? jtreg:test/langtools:tier3??????????????????????????? 0 0 0???? 0 >>>> ??? jtreg:test/nashorn:tier3????????????????????????????? 0 0 0???? 0 >>>> ??? jtreg:test/jaxp:tier3???????????????????????????????? 0 0 0???? 0 >>>> >>>> >>>> By comparison we can find, lots of(1000+) langtools tests will get >>>> fail, and other(about 30+, exclude some X11 and sanity that >>>> >>>> result problem) test cases will also fail because of local LANG. >>>> >>>> >>>> such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java, >>>> >>>> shouldContain("Error: Could not find or load main class " + CLASSNAME) >>>> >>>> >>>> When in the zh_CN.UTF-8 environment, because some of the output >>>> information is changed to Chinese by some properties file, >>>> >>>> the English cannot be matched, which will result in an fail. >>>> >>>> When I set? LANG=en_US/LC_ALL=C, this test will pass. >>>> >>>> >>>> I think there are some possible solutions. >>>> >>>> >>>> 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this >>>> modification is more violent because he will affect all test cases. >>>> >>>> >>>> 2.We modify each individual test?E.g >>>> test/hotspot/jtreg/compiler/c2/Test8062950.java >>>> >>>> diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java >>>> ??package compiler.c2; >>>> ??import jdk.test.lib.process.ProcessTools; >>>> +import java.util.Map; >>>> +import jdk.test.lib.process.OutputAnalyzer; >>>> >>>> ??public class Test8062950 { >>>> ????? private static final String CLASSNAME = "DoesNotExist"; >>>> ????? public static void main(String[] args) throws Exception { >>>> - ProcessTools.executeTestJvm("-Xcomp", >>>> - "-XX:-TieredCompilation", >>>> - "-XX:-UseOptoBiasInlining", >>>> - CLASSNAME) >>>> - .shouldHaveExitValue(1) >>>> -??????????????????? .shouldContain("Error: Could not find or load >>>> main class " + CLASSNAME) >>>> -??????????????????? .shouldNotContain("A fatal error has been >>>> detected") >>>> -??????????????????? .shouldNotContain("Internal Error") >>>> -??????????????????? .shouldNotContain("HotSpot Virtual Machine >>>> Error"); >>>> +??????? final ProcessBuilder pb = >>>> ProcessTools.createJavaProcessBuilder(true, >>>> + "-Xcomp", >>>> + "-XX:-TieredCompilation", >>>> + "-XX:-UseOptoBiasInlining", >>>> + CLASSNAME); >>>> +??????? final Map env = pb.environment(); >>>> +??????? env.put("LC_ALL", "en_US.UTF-8"); >>>> +??????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>>> + output.shouldHaveExitValue(1); >>>> +??????? output.shouldContain("Error: Could not find or load main >>>> class " + CLASSNAME); >>>> +??????? output.shouldNotContain("A fatal error has been detected"); >>>> +??????? output.shouldNotContain("Internal Error"); >>>> +??????? output.shouldNotContain("HotSpot Virtual Machine Error"); >>>> } >>>> ??} >>>> >>>> But I don't know if this will change the test program too much, >>>> because the related problems are a lot in langtools tests. >>>> >>>> >>>> 3.And i find that there is a function can judge the locale >>>> >>>> ??if (!isEnglishLocale()) { // only english version >>>> return; >>>> ??} >>>> >>>> But in this case, I think in many test cases, we may not be able to >>>> return so directly, because some test cases may have other test >>>> purposes. >>>> >>>> >>>> So I don't know if you have any ideas or some suggestions to solve >>>> the problem that the output information and English do not match in >>>> this >>>> >>>> non-English environment. >>>> >>>> >>>> Cheers, >>>> >>>> Jing Tian >>>> >>>> >>>> >>>> From david.holmes at oracle.com Fri Mar 29 01:25:00 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 29 Mar 2019 11:25:00 +1000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> Message-ID: <945d9546-5195-ab33-0e20-57871c2f1ef9@oracle.com> Hi Andrew, This seems fine in general but I have a few queries on some details: src/hotspot/share/classfile/javaClasses.hpp f(java_lang_Thread) \ + f(jdk_internal_misc_UnsafeConstants) \ f(java_lang_ThreadGroup) \ Is there a reason this needs to be shoved in there? Similarly with src/hotspot/share/classfile/systemDictionary.hpp: do_klass(Thread_klass, java_lang_Thread ) \ + do_klass(UnsafeConstants_klass, jdk_internal_misc_UnsafeConstants ) \ do_klass(ThreadGroup_klass, java_lang_ThreadGroup ) \ ? --- src/hotspot/share/runtime/thread.cpp main_thread->set_threadObj(thread_object); + + // initialize the hardware-specific constants needed by Unsafe + initialize_class(vmSymbols::jdk_internal_misc_UnsafeConstants(), CHECK); + jdk_internal_misc_UnsafeConstants::set_unsafe_constants(); + // Set thread status to running since main thread has // been started and running. java_lang_Thread::set_thread_status(thread_object, That seems a very odd place to insert the new initialization. I can't see any reason you'd need to split the Thread object initialization like that. ?? More generally exactly when do you need to initialize this new class by and how does the initialization order change before/after this fix? (I'm expecting only to see the new class inserted where needed without any other perturbations.) --- src/hotspot/share/classfile/vmSymbols.hpp + template(big_endian_name, "BIG_ENDIAN") \ + template(use_unaligned_access_name, "UNALIGNED_ACCESS") \ Nit: There's an extra space before "UNALIGNED... --- src/java.base/share/classes/jdk/internal/misc/UnsafeConstants.java 31 * package-protected ... s/protected/private/ 37 * The JVM injects values into all the static fields of this class ... 43 * hardware-specific constants installed by the JVM. I get this gist of this note but still found it rather difficult to intepret. There are I think two key points: 1. The fields must not be constant variables, hence the need for the static initialization block. 2. The true value of each field is injected by the VM. How about: * The JVM injects hardware-specific values into all the static fields * of this class during JVM initialization. The static initialization * block exists to prevent the fields from being considered constant * variables, so the field values will be not be compiled directly into * any class that uses them. 60 * @implNote 61 * The actual value for this field is injected by JVM. A static 62 * initialization block is used to set the value here to 63 * communicate that this static final field is not statically 64 * foldable, and to avoid any possible circular dependency during 65 * vm initialization. I think all you need for each field is just: * @implNote * The actual value for this field is injected by _the_ JVM. 85 * flag whose value ... 92 * flag whose value ... s/flag/Flag/ Thanks, David On 29/03/2019 2:56 am, Andrew Dinn wrote: > On 28/03/2019 15:22, Thomas St?fe wrote: >> The second of those was actually intended to be iff. This is a common >> abbreviation used by English/US mathematicians and logicians to write >> 'if and only if' (it is also sometimes written as <=>). Since you didn't >> recognize it I guess I really need to write it out in full. >> >> >> Oh, don't worry on my account. I am not a native speaker nor a >> mathematician. You could leave iff and add [sic] to make everyone >> curious and start googling "iff" :) > I changed it nevertheless. It would be remiss to presume readers need be > conversant with elliptical, English logico-mathematical parlance (I'm > explaining this in as plain English as I can ... no, wait! ;-). > > Anyway, I addressed this and your other concerns in a new webrev. > > JIRA: https://bugs.openjdk.java.net/browse/JDK-8221477 > Webrev: http://cr.openjdk.java.net/~adinn/8221477/webrev.02 > > Input from a second reviewer would be welcome ... > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From vladimir.kozlov at oracle.com Fri Mar 29 02:07:14 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 28 Mar 2019 19:07:14 -0700 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: Message-ID: <4b759550-f185-724f-7139-9bab648a1966@oracle.com> Hi Stefan, I collected some data on MetadataHandleBlock. First, do_unloading() code is executed only when class_unloading_occurred is 'true' - it is rare case. It should not affect normal G1 remark pause. Second, I run a test with -Xcomp. I got about 10,000 compilations by Graal and next data at the end of execution: max_blocks = 232 max_handles_per_block = 32 (since handles array has 32 elements) max_total_alive_values = 4631 Thanks, Vladimir On 3/28/19 2:44 PM, Vladimir Kozlov wrote: > Thank you, Stefan > > On 3/28/19 12:54 PM, Stefan Karlsson wrote: >> Hi Vladimir, >> >> I started to check the GC code. >> >> ======================================================================== >> I see that you've added guarded includes in the middle of the include list: >> ?? #include "gc/shared/strongRootsScope.hpp" >> ?? #include "gc/shared/weakProcessor.hpp" >> + #if INCLUDE_JVMCI >> + #include "jvmci/jvmci.hpp" >> + #endif >> ?? #include "oops/instanceRefKlass.hpp" >> ?? #include "oops/oop.inline.hpp" >> >> The style we use is to put these conditional includes at the end of the include lists. > > okay > >> >> ======================================================================== >> Could you also change the following: >> >> + #if INCLUDE_JVMCI >> +???? // Clean JVMCI metadata handles. >> +???? JVMCI::do_unloading(is_alive_closure(), purged_class); >> + #endif >> >> to: >> +???? // Clean JVMCI metadata handles. >> +???? JVMCI_ONLY(JVMCI::do_unloading(is_alive_closure(), purged_class);) >> >> to get rid of some of the line noise in the GC files. > > okay > >> >> ======================================================================== >> In the future we will need version of JVMCI::do_unloading that supports concurrent cleaning for ZGC. > > Yes, we need to support concurrent cleaning in a future. > >> >> ======================================================================== >> What's the performance impact for G1 remark pause with this serial walk over the MetadataHandleBlock? >> >> 3275 void G1CollectedHeap::complete_cleaning(BoolObjectClosure* is_alive, >> 3276???????????????????????????????????????? bool class_unloading_occurred) { >> 3277?? uint num_workers = workers()->active_workers(); >> 3278?? ParallelCleaningTask unlink_task(is_alive, num_workers, class_unloading_occurred, false); >> 3279?? workers()->run_task(&unlink_task); >> 3280 #if INCLUDE_JVMCI >> 3281?? // No parallel processing of JVMCI metadata handles for now. >> 3282?? JVMCI::do_unloading(is_alive, class_unloading_occurred); >> 3283 #endif >> 3284 } > > There should not be impact if Graal is not used. Only cost of call (which most likely is inlined in > product VM) and check: > > http://hg.openjdk.java.net/metropolis/dev/file/530fc1427d02/src/hotspot/share/jvmci/jvmciRuntime.cpp#l1237 > > > If Graal is used it should not have big impact since these metadata has regular pattern (32? handles > per array and array per MetadataHandleBlock block which are linked in list) and not large. > If there will be noticeable impact - we will work on it as you suggested by using ParallelCleaningTask. > >> >> ======================================================================== >> Did you consider adding it as a task for one of the worker threads to execute in >> ParallelCleaningTask? >> >> See how other tasks are claimed by one worker: >> void KlassCleaningTask::work() { >> ?? ResourceMark rm; >> >> ?? // One worker will clean the subklass/sibling klass tree. >> ?? if (claim_clean_klass_tree_task()) { >> ???? Klass::clean_subklass_tree(); >> ?? } > > These changes were ported from JDK8u based changes in graal-jvmci-8 and there are no > ParallelCleaningTask in JDK8. > > Your suggestion is interesting and I agree that we should investigate it. > >> >> ======================================================================== >> In MetadataHandleBlock::do_unloading: >> >> +??????? if (klass->class_loader_data()->is_unloading()) { >> +????????? // This needs to be marked so that it's no longer scanned >> +????????? // but can't be put on the free list yet. The >> +????????? // ReferenceCleaner will set this to NULL and >> +????????? // put it on the free list. >> >> I couldn't find the ReferenceCleaner in the patch or in the source. Where can I find this code? > > I think it is typo (I will fix it) - it references new HandleCleaner class: > > http://cr.openjdk.java.net/~kvn/8220623/webrev.03/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HandleCleaner.java.html > > > Thanks, > Vladimir > >> >> Thanks, >> StefanK >> >> On 2019-03-28 20:15, Vladimir Kozlov wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8220623 >>> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ >>> >>> Update JVMCI to support pre-compiled as shared library Graal. >>> Using aoted Graal can offers benefits including: >>> ?- fast startup >>> ?- compile time similar to native JIt compilers (C2) >>> ?- memory usage disjoint from the application Java heap >>> ?- no profile pollution of JDK code used by the application >>> >>> This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. >>> Changes were collected in Metropolis repo [2] and tested there. >>> >>> Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler group. >>> Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. >>> >>> I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In this set Graal was >>> tested only in tier3. >>> >>> And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our system. Several issue >>> were found which were present before these changes. >>> >>> Thanks, >>> Vladimir >>> >>> [1] https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af >>> [2] http://hg.openjdk.java.net/metropolis/dev/ >> From jingtian at loongson.cn Fri Mar 29 02:20:29 2019 From: jingtian at loongson.cn (Jing Tian) Date: Fri, 29 Mar 2019 10:20:29 +0800 Subject: RFC: Make test failed because of the locale LANG In-Reply-To: <794486e3-0a12-7605-2be4-abd3916d030f@oracle.com> References: <5fd4d748-c9cf-3779-3a65-b7c7547ab9ce@loongson.cn> <396490c6-9917-2306-c288-2de5ec92bdc9@oracle.com> <794486e3-0a12-7605-2be4-abd3916d030f@oracle.com> Message-ID: Hi, First of all, I want to say sorry . Because of the problems with my mail client, this typography looks bad.I found the typography so bad on the webpage today. T_T These suggestions I just want to show some ideas. We can't make changes in each test case, because it is too much, and the changes I made are not good. The following is the fail test cases because of the locale(exclude some X11 fail and sanity fail, because some of fails are in the list because of them): java/lang/System/LoggerFinder/internal/BootstrapLogger/BootstrapLoggerTest.java java/util/logging/LocalizedLevelName.java java/util/logging/SimpleFormatterFormat.java sun/util/logging/SourceClassName.java tools/pack200/DeprecatePack200.java java/security/KeyStore/PKCS12/KeytoolReaderP12Test.java java/time/tck/java/time/format/TCKDateTimeFormatterBuilder.java javax/security/auth/login/JAASConfigSyntaxCheck/JAASConfigSyntaxTest.java javax/security/auth/login/LoginContext/DynamicConfigurationTest.java jdk/security/logging/TestSecurityPropertyModificationLog.java jdk/security/logging/TestTLSHandshakeLog.java jdk/security/logging/TestX509CertificateLog.java jdk/security/logging/TestX509ValidationLog.java tools/jar/DeprecateOptionN.java tools/jar/compat/CLICompatibility.java tools/jar/mmrjar/Basic.java tools/jar/modularJar/Basic.java tools/jar/multiRelease/ApiValidatorTest.java tools/jar/multiRelease/Basic.java tools/jar/multiRelease/Basic1.java tools/jlink/JLinkNegativeTest.java tools/jlink/JLinkOptionsTest.java tools/jlink/JLinkPluginsTest.java tools/jlink/JLinkTest.java tools/jlink/bindservices/SuggestProviders.java tools/jlink/plugins/LegalFilePluginTest.java tools/jmod/JmodNegativeTest.java tools/jmod/JmodTest.java tools/launcher/FXLauncherTest.java tools/launcher/InfoStreams.java tools/launcher/SourceMode.java tools/launcher/TestSpecialArgs.java javax/xml/jaxp/functional/org/xml/sax/ptests/EHFatalTest.java test/hotspot/jtreg/compiler/c2/Test8062950.java test/hotspot/jtreg/runtime/classFileParserBug/TestEmptyBootstrapMethodsAttr.java The following is the langtools fail test cases.(1450/3922) jdk/javadoc/doclet/AccessAsciiArt/AccessAsciiArt.java jdk/javadoc/doclet/AccessFrameTitle/AccessFrameTitle.java jdk/javadoc/doclet/AccessSkipNav/AccessSkipNav.java jdk/javadoc/doclet/AuthorDD/AuthorDD.java jdk/javadoc/doclet/MetaTag/MetaTag.java jdk/javadoc/doclet/WindowTitles/WindowTitles.java jdk/javadoc/doclet/testAbstractMethod/TestAbstractMethod.java jdk/javadoc/doclet/testAnchorNames/TestAnchorNames.java jdk/javadoc/doclet/testAnnotationTypes/TestAnnotationTypes.java jdk/javadoc/doclet/testAuthor/TestAuthor.java jdk/javadoc/doclet/testBadPackageFileInJar/TestBadPackageFileInJar.java jdk/javadoc/doclet/testCharsetDocencodingOptions/TestCharsetDocencodingOptions.java jdk/javadoc/doclet/testClassCrossReferences/TestClassCrossReferences.java jdk/javadoc/doclet/testClassDocCatalog/TestClassDocCatalog.java .......... I don't know how to solve this problem is a better solution. ? 2019/3/29 9:23, naoto.sato at oracle.com ??: > On 3/28/19 4:54 PM, David Holmes wrote: >> On 29/03/2019 1:59 am, Naoto Sato wrote: >>> Hi, >>> >>> I don't think there is any *official* rule for the regression tests >>> to succeed in any locale, but if the test case is locale sensitive >>> such as in this case, I'd suggest it should correctly specify the >>> locale beforehand, or quit gracefully in general. >>> >>> For this specific case, I'd suggest not set the environment variable >>> (as it won't work with Windows), using runtime system properties >>> (user.language/user.country) would help. >> >> Which specific case are you referring to? :) > > I was referring to the test case below, Test8062950.java, where it > issues "env.put("LC_ALL", "en_US.UTF-8")". Instead I would recommend > "-Duser.language=en -Duser.country=US" runtime system property. > >> >> Given there are well over a thousand failures in total I think we >> need to set a reasonable initial locale as part of the test setup. >> Many of the hotspot tests check actual and expected output and we, in >> general, have no idea what kinds of output may be subject to locale >> specific > changes. > > Agree. Specifying a reasonable default locale would not be a bad idea. > > Naoto > >> >> Cheers, >> David >> >>> Naoto >>> >>> On 3/27/19 10:47 PM, David Holmes wrote: >>>> Hi Jing, >>>> >>>> On 28/03/2019 3:22 pm, Jing Tian wrote: >>>>> Hi, >>>>> >>>>> When I am doing the 'make test'.If the local LANG is set as >>>>> 'zh_CN.UTF-8', Test cases will have a lot of error messages. >>>>> >>>>> ============================== >>>>> Test summary >>>>> ============================== >>>>> ??? TEST TOTAL? PASS FAIL ERROR >>>>> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1373 1371 2 >>>>> 0 << >>>>> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1860 7 >>>>> 0 << >>>>> ?>> jtreg:test/langtools:tier1???????????????????????? 3922 2470 >>>>> 1450???? 2 << >>>>> ??? jtreg:test/nashorn:tier1 0 0 0???? 0 >>>>> ??? jtreg:test/jaxp:tier1 0 0 0???? 0 >>>>> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3334 3305 >>>>> 29 0 << >>>>> ?>> jtreg:test/langtools:tier2 11???? 9???? 2???? 0 << >>>>> ??? jtreg:test/nashorn:tier2 35 35 0???? 0 >>>>> ?>> jtreg:test/jaxp:tier2?????????????????????????????? 438 437 1 >>>>> 0 << >>>>> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1068 >>>>> 36 0 << >>>>> ??? jtreg:test/langtools:tier3 0 0 0???? 0 >>>>> ??? jtreg:test/nashorn:tier3 0 0 0???? 0 >>>>> ??? jtreg:test/jaxp:tier3 0 0 0???? 0 >>>>> ============================== >>>> >>>> Given most of these are not hotspot issues and given the number of >>>> failures, I think this is something that would need to be discussed >>>> much more broadly. So I've cc'd core-libs-dev and compiler-dev. I >>>> recall there was a very recent discussion regarding some tests >>>> failing for the same reason, but don't recall the outcome. >>>> >>>> David >>>> ----- >>>> >>>>> On the same machine,when i set LANG=en_US.UTF-8. >>>>> >>>>> ============================== >>>>> Test summary >>>>> ============================== >>>>> ??? TEST TOTAL? PASS FAIL ERROR >>>>> ?>> jtreg:test/hotspot/jtreg:tier1???????????????????? 1388 1386 2 >>>>> 0 << >>>>> ?>> jtreg:test/jdk:tier1?????????????????????????????? 1867 1843 >>>>> 19 5 << >>>>> ??? jtreg:test/langtools:tier1 3920 3920 0???? 0 >>>>> ??? jtreg:test/nashorn:tier1 0 0 0???? 0 >>>>> ??? jtreg:test/jaxp:tier1 0 0 0???? 0 >>>>> ?>> jtreg:test/jdk:tier2?????????????????????????????? 3328 3290 >>>>> 31 7 << >>>>> ??? jtreg:test/langtools:tier2 11 11 0???? 0 >>>>> ??? jtreg:test/nashorn:tier2 35 35 0???? 0 >>>>> ??? jtreg:test/jaxp:tier2 438 438 0???? 0 >>>>> ?>> jtreg:test/jdk:tier3?????????????????????????????? 1104 1067 >>>>> 37 0 << >>>>> ??? jtreg:test/langtools:tier3 0 0 0???? 0 >>>>> ??? jtreg:test/nashorn:tier3 0 0 0???? 0 >>>>> ??? jtreg:test/jaxp:tier3 0 0 0???? 0 >>>>> >>>>> >>>>> By comparison we can find, lots of(1000+) langtools tests will get >>>>> fail, and other(about 30+, exclude some X11 and sanity that >>>>> >>>>> result problem) test cases will also fail because of local LANG. >>>>> >>>>> >>>>> such as in the test/hotspot/jtreg/compiler/c2/Test8062950.java, >>>>> >>>>> shouldContain("Error: Could not find or load main class " + >>>>> CLASSNAME) >>>>> >>>>> >>>>> When in the zh_CN.UTF-8 environment, because some of the output >>>>> information is changed to Chinese by some properties file, >>>>> >>>>> the English cannot be matched, which will result in an fail. >>>>> >>>>> When I set? LANG=en_US/LC_ALL=C, this test will pass. >>>>> >>>>> >>>>> I think there are some possible solutions. >>>>> >>>>> >>>>> 1.we set LC_ALL=C/LANG=EN_us in the Runtests.gmk, but this >>>>> modification is more violent because he will affect all test cases. >>>>> >>>>> >>>>> 2.We modify each individual test?E.g >>>>> test/hotspot/jtreg/compiler/c2/Test8062950.java >>>>> >>>>> diff -r 0421d49b6217 test/hotspot/jtreg/compiler/c2/Test8062950.java >>>>> ??package compiler.c2; >>>>> ??import jdk.test.lib.process.ProcessTools; >>>>> +import java.util.Map; >>>>> +import jdk.test.lib.process.OutputAnalyzer; >>>>> >>>>> ??public class Test8062950 { >>>>> ????? private static final String CLASSNAME = "DoesNotExist"; >>>>> ????? public static void main(String[] args) throws Exception { >>>>> - ProcessTools.executeTestJvm("-Xcomp", >>>>> - "-XX:-TieredCompilation", >>>>> - "-XX:-UseOptoBiasInlining", >>>>> - CLASSNAME) >>>>> - .shouldHaveExitValue(1) >>>>> -??????????????????? .shouldContain("Error: Could not find or load >>>>> main class " + CLASSNAME) >>>>> -??????????????????? .shouldNotContain("A fatal error has been >>>>> detected") >>>>> -??????????????????? .shouldNotContain("Internal Error") >>>>> -??????????????????? .shouldNotContain("HotSpot Virtual Machine >>>>> Error"); >>>>> +??????? final ProcessBuilder pb = >>>>> ProcessTools.createJavaProcessBuilder(true, >>>>> + "-Xcomp", >>>>> + "-XX:-TieredCompilation", >>>>> + "-XX:-UseOptoBiasInlining", >>>>> + CLASSNAME); >>>>> +??????? final Map env = pb.environment(); >>>>> +??????? env.put("LC_ALL", "en_US.UTF-8"); >>>>> +??????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); >>>>> + output.shouldHaveExitValue(1); >>>>> +??????? output.shouldContain("Error: Could not find or load main >>>>> class " + CLASSNAME); >>>>> +??????? output.shouldNotContain("A fatal error has been detected"); >>>>> +??????? output.shouldNotContain("Internal Error"); >>>>> +??????? output.shouldNotContain("HotSpot Virtual Machine Error"); >>>>> } >>>>> ??} >>>>> >>>>> But I don't know if this will change the test program too much, >>>>> because the related problems are a lot in langtools tests. >>>>> >>>>> >>>>> 3.And i find that there is a function can judge the locale >>>>> >>>>> ??if (!isEnglishLocale()) { // only english version >>>>> return; >>>>> ??} >>>>> >>>>> But in this case, I think in many test cases, we may not be able >>>>> to return so directly, because some test cases may have other test >>>>> purposes. >>>>> >>>>> >>>>> So I don't know if you have any ideas or some suggestions to solve >>>>> the problem that the output information and English do not match >>>>> in this >>>>> >>>>> non-English environment. >>>>> >>>>> >>>>> Cheers, >>>>> >>>>> Jing Tian >>>>> >>>>> >>>>> >>>>> From david.holmes at oracle.com Fri Mar 29 05:06:44 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 29 Mar 2019 15:06:44 +1000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: Message-ID: <6e54431a-8e23-160a-5757-948f1041819e@oracle.com> Hi Matthias, This generally seems okay. On 28/03/2019 2:50 am, Baesken, Matthias wrote: > Hello, here is a second webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.1/ > > I followed Davids idea to use the os_perf coding . > For reusing I moved os::Linux::get_tick_information (former name get_total_ticks , but it is now not only total but also steal ) into os_linux . > The CPUPerfTicks struct has been enhanced by steal and a boolean flag has_steal_ticks . > > 28 struct CPUPerfTicks { > 29 uint64_t used; > 30 uint64_t usedKernel; > 31 uint64_t total; > 32 uint64_t steal; > 33 bool has_steal_ticks; > 34 }; Okay, but one minor nit: why put struct CPUPerfTicks in the os:: namespace with the code that uses it in the os::Linux namespace? A few other minor comments: src/hotspot/os/linux/os_linux.cpp + if (-1 == which_logical_cpu) { hotspot style puts the variable first in a comparison against a constant. + #ifdef DEBUG_LINUX_PROC_STAT This should either just be DEBUG code or else deleted. The vm_printfs would ideally be logging statements if that is possible. + if (n > expected_tickinfo_count+3) { Spaces around + please. + st->print_cr("Steal ticks from vm start: %llu", (long long unsigned) steal_ticks_difference); %llu should be UINT64_FORMAT. The cast should not be needed. Thanks, David ----- > > I found the "big API" idea a bit too much for just 2 use cases . > > > Best regards , Matthias > > > >> -----Original Message----- >> From: David Holmes >> Sent: Mittwoch, 27. M?rz 2019 11:22 >> To: Baesken, Matthias ; Thomas St?fe >> >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: RFR: 8221535: add steal tick related information to hs_error file >> [linux] >> >> On 27/03/2019 8:04 pm, Baesken, Matthias wrote: >>> Hi Thomas, interesting idea . David ? would you be fine with? what >>> Thomas proposes ? >> >> Isn't adding the missing fields to the existing os_perf code basically >> the same thing? I don't see any point in duplicating existing code like >> this. >> >> David >> ----- >> >>> Best regards, Matthias >>> >>> *From:*Thomas St?fe >>> *Sent:* Mittwoch, 27. M?rz 2019 10:28 >>> *To:* Baesken, Matthias >>> *Cc:* hotspot-dev at openjdk.java.net >>> *Subject:* Re: RFR: 8221535: add steal tick related information to >>> hs_error file [linux] >>> >>> Hi Matthias, >>> >>> this is a lot of coding for just one counter... >>> >>> Also os_perf_linux.cpp does something very similar. However, that code >>> does not get the steal ticks you want, does not seem to care about >>> kernel differences and could also be improved. >>> >>> Proposal: Why do you not tweak your new API to be a general purpose API >>> to read all ticks from /proc/stat. That would be way more useful. Later, >>> if we feel like it, could change os_perf_linux.cpp to use your function. >>> >>> We did something similar for the statistical history in the SapMachine, see >>> >>> >> https://github.com/SAP/SapMachine/blob/sapmachine/src/hotspot/os/linu >> x/stathist_linux.cpp >>> >>> . So I would suggest to have a function reading proc stat and returning >>> all values in a struct. That struct should have provisions for "value >>> not found" since which values you get depends in kernel version. >>> >>> -------- >>> >>> Example: >>> >>> struct value_t { bool valid; uint64_t v; } >>> >>> struct cpu_values_t { >>> ?value_t user; >>> ?value_t nice; >>> ?value_t system; >>> ?value_t idle; >>> ?value_t iowait; >>> ?value_t steal; >>> ?value_t guest; >>> ?value_t guest_nice; >>> }; >>> >>> void parse_proc_stat_cpu_line(const char* line, cpu_values_t* out) { >>> ? // Note: existence of some of these values depends on kernel version >>> ? out->user.valid = out->nice.valid?= out->system.valid?= >>> out->idle.valid?= out->iowait.valid?= out->steal.valid?= >>> out->guest.valid?= out->guest_nice.valid?= false; >>> ? int user, nice, system, idle, iowait, irq, softirq, steal, guest, >>> guest_nice; >>> ? int num = ::sscanf(line, "cpu %d %d %d %d %d %d %d %d %d %d", >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &user, &nice, &system, &idle, &iowait, >>> &irq, &softirq, &steal, &guest, &guest_nice); >>> ? if (num >= 4) { >>> ? ? out->user.v = user; out->user.valid = true; >>> ? ? out->nice.v?= nice; out->nice.valid = true; >>> ? ? out->system.v?= system; out->system.valid = true; >>> ? ? out->idle.v?= idle; out->idle.valid = true; >>> ? ? if (num >= 5) { // iowait (5) (since Linux 2.5.41) >>> ? ? ? out->iowait = iowait; out->iowait.valid = true; >>> ? ? ? if (num >= 8) { // steal (8) (since Linux 2.6.11) >>> ? ? ? ? out->steal = steal; out->steal.valid = true; >>> ? ? ? ? if (num >= 9) { // guest (9) (since Linux 2.6.24) >>> ? ? ? ? ? out->guest = guest; out->guest.valid = true; >>> ? ? ? ? ? if (num >= 10) { // guest (9) (since Linux 2.6.33) >>> ? ? ? ? ? ? out->guest_nice = guest_nice; out->guest_nice.valid = true; >>> ? ? ? ? ? } >>> ? ? ? } >>> ? ?} >>> } >>> >>> Cheers Thomas >>> >>> On Wed, Mar 27, 2019 at 9:42 AM Baesken, Matthias >>> > >> wrote: >>> >>> Hello, please review? the? following change . >>> >>> Recent Linux kernels provide in the /proc filesystem? ?so called >>> "steal" information, see the the proc man page : >>> >>> ? .. >>> "there is an eighth column, steal - stolen time, which is the time >>> spent in other operating systems when running in a virtualized >>> environment" >>> >>> The info was useful? in our? internal VM,? when looking into >>> performance issues? (especially in virtualized environments), so it >>> should be added to the hs_err file . >>> >>> Bug/webrev : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8221535 >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.0/ >>> >>> >>> Best Regards, Matthias >>> From stefan.karlsson at oracle.com Fri Mar 29 07:36:40 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 29 Mar 2019 08:36:40 +0100 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: <4b759550-f185-724f-7139-9bab648a1966@oracle.com> References: <4b759550-f185-724f-7139-9bab648a1966@oracle.com> Message-ID: On 2019-03-29 03:07, Vladimir Kozlov wrote: > Hi Stefan, > > I collected some data on MetadataHandleBlock. > > First, do_unloading() code is executed only when > class_unloading_occurred is 'true' - it is rare case. It should not > affect normal G1 remark pause. It's only rare for applications that don't do dynamic class loading and unloading. The applications that do, will be affected. > > Second, I run a test with -Xcomp. I got about 10,000 compilations by > Graal and next data at the end of execution: > > max_blocks = 232 > max_handles_per_block = 32 (since handles array has 32 elements) > max_total_alive_values = 4631 OK. Thanks for the info. StefanK > > Thanks, > Vladimir > > On 3/28/19 2:44 PM, Vladimir Kozlov wrote: >> Thank you, Stefan >> >> On 3/28/19 12:54 PM, Stefan Karlsson wrote: >>> Hi Vladimir, >>> >>> I started to check the GC code. >>> >>> ======================================================================== >>> I see that you've added guarded includes in the middle of the include >>> list: >>> ?? #include "gc/shared/strongRootsScope.hpp" >>> ?? #include "gc/shared/weakProcessor.hpp" >>> + #if INCLUDE_JVMCI >>> + #include "jvmci/jvmci.hpp" >>> + #endif >>> ?? #include "oops/instanceRefKlass.hpp" >>> ?? #include "oops/oop.inline.hpp" >>> >>> The style we use is to put these conditional includes at the end of >>> the include lists. >> >> okay >> >>> >>> ======================================================================== >>> Could you also change the following: >>> >>> + #if INCLUDE_JVMCI >>> +???? // Clean JVMCI metadata handles. >>> +???? JVMCI::do_unloading(is_alive_closure(), purged_class); >>> + #endif >>> >>> to: >>> +???? // Clean JVMCI metadata handles. >>> +???? JVMCI_ONLY(JVMCI::do_unloading(is_alive_closure(), purged_class);) >>> >>> to get rid of some of the line noise in the GC files. >> >> okay >> >>> >>> ======================================================================== >>> In the future we will need version of JVMCI::do_unloading that >>> supports concurrent cleaning for ZGC. >> >> Yes, we need to support concurrent cleaning in a future. >> >>> >>> ======================================================================== >>> What's the performance impact for G1 remark pause with this serial >>> walk over the MetadataHandleBlock? >>> >>> 3275 void G1CollectedHeap::complete_cleaning(BoolObjectClosure* >>> is_alive, >>> 3276???????????????????????????????????????? bool >>> class_unloading_occurred) { >>> 3277?? uint num_workers = workers()->active_workers(); >>> 3278?? ParallelCleaningTask unlink_task(is_alive, num_workers, >>> class_unloading_occurred, false); >>> 3279?? workers()->run_task(&unlink_task); >>> 3280 #if INCLUDE_JVMCI >>> 3281?? // No parallel processing of JVMCI metadata handles for now. >>> 3282?? JVMCI::do_unloading(is_alive, class_unloading_occurred); >>> 3283 #endif >>> 3284 } >> >> There should not be impact if Graal is not used. Only cost of call >> (which most likely is inlined in product VM) and check: >> >> http://hg.openjdk.java.net/metropolis/dev/file/530fc1427d02/src/hotspot/share/jvmci/jvmciRuntime.cpp#l1237 >> >> >> If Graal is used it should not have big impact since these metadata >> has regular pattern (32? handles per array and array per >> MetadataHandleBlock block which are linked in list) and not large. >> If there will be noticeable impact - we will work on it as you >> suggested by using ParallelCleaningTask. >> >>> >>> ======================================================================== >>> Did you consider adding it as a task for one of the worker threads to >>> execute in ParallelCleaningTask? >>> >>> See how other tasks are claimed by one worker: >>> void KlassCleaningTask::work() { >>> ?? ResourceMark rm; >>> >>> ?? // One worker will clean the subklass/sibling klass tree. >>> ?? if (claim_clean_klass_tree_task()) { >>> ???? Klass::clean_subklass_tree(); >>> ?? } >> >> These changes were ported from JDK8u based changes in graal-jvmci-8 >> and there are no ParallelCleaningTask in JDK8. >> >> Your suggestion is interesting and I agree that we should investigate it. >> >>> >>> ======================================================================== >>> In MetadataHandleBlock::do_unloading: >>> >>> +??????? if (klass->class_loader_data()->is_unloading()) { >>> +????????? // This needs to be marked so that it's no longer scanned >>> +????????? // but can't be put on the free list yet. The >>> +????????? // ReferenceCleaner will set this to NULL and >>> +????????? // put it on the free list. >>> >>> I couldn't find the ReferenceCleaner in the patch or in the source. >>> Where can I find this code? >> >> I think it is typo (I will fix it) - it references new HandleCleaner >> class: >> >> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HandleCleaner.java.html >> >> >> Thanks, >> Vladimir >> >>> >>> Thanks, >>> StefanK >>> >>> On 2019-03-28 20:15, Vladimir Kozlov wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8220623 >>>> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ >>>> >>>> Update JVMCI to support pre-compiled as shared library Graal. >>>> Using aoted Graal can offers benefits including: >>>> ?- fast startup >>>> ?- compile time similar to native JIt compilers (C2) >>>> ?- memory usage disjoint from the application Java heap >>>> ?- no profile pollution of JDK code used by the application >>>> >>>> This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to >>>> date. >>>> Changes were collected in Metropolis repo [2] and tested there. >>>> >>>> Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and >>>> our compiler group. >>>> Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI >>>> flags. >>>> >>>> I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was >>>> clean. In this set Graal was tested only in tier3. >>>> >>>> And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available >>>> in our system. Several issue were found which were present before >>>> these changes. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> [1] >>>> https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af >>>> >>>> [2] http://hg.openjdk.java.net/metropolis/dev/ >>> From matthias.baesken at sap.com Fri Mar 29 07:56:43 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 29 Mar 2019 07:56:43 +0000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: <6e54431a-8e23-160a-5757-948f1041819e@oracle.com> References: <6e54431a-8e23-160a-5757-948f1041819e@oracle.com> Message-ID: Hi David , thanks for your comments . New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.2/ moved os::CPUPerfTicks to os::Linux::CPUPerfTicks removed DEBUG_LINUX_PROC_STAT from os_linux.cpp addressed the minor comments Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Freitag, 29. M?rz 2019 06:07 > To: Baesken, Matthias ; Thomas St?fe > > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8221535: add steal tick related information to hs_error file > [linux] > > Hi Matthias, > > This generally seems okay. > > On 28/03/2019 2:50 am, Baesken, Matthias wrote: > > Hello, here is a second webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.1/ > > > > I followed Davids idea to use the os_perf coding . > > For reusing I moved os::Linux::get_tick_information (former name > get_total_ticks , but it is now not only total but also steal ) into os_linux . > > The CPUPerfTicks struct has been enhanced by steal and a boolean flag > has_steal_ticks . > > > > 28 struct CPUPerfTicks { > > 29 uint64_t used; > > 30 uint64_t usedKernel; > > 31 uint64_t total; > > 32 uint64_t steal; > > 33 bool has_steal_ticks; > > 34 }; > > Okay, but one minor nit: why put struct CPUPerfTicks in the os:: > namespace with the code that uses it in the os::Linux namespace? > > A few other minor comments: > > src/hotspot/os/linux/os_linux.cpp > > + if (-1 == which_logical_cpu) { > > hotspot style puts the variable first in a comparison against a constant. > > + #ifdef DEBUG_LINUX_PROC_STAT > > This should either just be DEBUG code or else deleted. The vm_printfs > would ideally be logging statements if that is possible. > > + if (n > expected_tickinfo_count+3) { > > Spaces around + please. > > + st->print_cr("Steal ticks from vm start: %llu", (long long > unsigned) steal_ticks_difference); > > %llu should be UINT64_FORMAT. > The cast should not be needed. > > Thanks, > David > ----- > > > > > I found the "big API" idea a bit too much for just 2 use cases . > > > > > > Best regards , Matthias > > From adinn at redhat.com Fri Mar 29 08:43:10 2019 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 29 Mar 2019 08:43:10 +0000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> Message-ID: <7eb52b3a-bd9b-62ba-9451-54cb42871ce0@redhat.com> On 28/03/2019 17:09, Thomas St?fe wrote: > src/hotspot/share/classfile/javaClasses.cpp > > Pure nitpicking, but you could remove the member variables too and just > set the Unsafe members directly, e.g.: > > ? ? if (fd->name() == vmSymbols::address_size_name()) { > ? ? ? mirror->int_field_put(fd->offset(), (jint)sizeof(void*)); > ? ? } else if (fd->name() == vmSymbols::page_size_name()) { > ? ? ? mirror->int_field_put(fd->offset(), (jint) os::vm_page_size()); > .. > > and so on. But I leave this up to you. This is already better than before. I thought about that but it seemed to me to be better for clarity and readability to init all the members in a single block in the constructor rather than spread the computations of the field values through the code that updates the fields. I think it's easier that way to see where all the values are coming from and align them up with the fields in the target class. > +? ? }else { > space missing before else Yes. > src/hotspot/share/classfile/vmSymbols.hpp > "UNALIGNED_ACCESS" off by one space Yes. > src/java.base/share/classes/java/nio/channels/FileChannel.java > > This may have creeped in from another change. Ah, yes. That was the next patch that I forgot to back out. I have updated the webrev in place to remove it and also fix the whitespace & alignment issues. > Good otherwise. Thanks. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From thomas.stuefe at gmail.com Fri Mar 29 08:44:19 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 29 Mar 2019 09:44:19 +0100 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: <6e54431a-8e23-160a-5757-948f1041819e@oracle.com> Message-ID: Hi Matthias, as usual, thanks for your patience and perseverance. Unfortunately you touched a piece of coding which was not very pretty (IMHO) in the first place, so there are a number of further remarks: - Return type: please use a simple bool value, not OSReturn. Keeps this in line with all other OS functions. Plus I do not see us returning anything else than OS_OK and OS_ERR anyway, and callers seem not to care either. - Please use lowercase-underscore naming of the return struct, not camel case. Again, like all other os functions around. I suggest a simple "cpu_ticks". - Please change parameter order to get_tick_information and make the default of "which_logical_cpu" -1. And add a small comment that -1 means "all". // which_logical_cpu=-1 returns accumulated ticks for all cpus. bool os::Linux::get_tick_information(cpu_ticks* out, int which_logical_cpu = -1) { - Please initialize the output structure to zero at the begin of the function. Just a simple memset is sufficient. - I would rename "expected_tickinfo_count" to "required_tickinfo_count". That clearly states that we have a number of values which are mandatory but are prepared to handle optional values too. - I would completely remove this section: +#ifdef DEBUG_LINUX_PROC_STAT + vm_fprintf(stderr, "[stat] read " + UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " + UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " \n", + userTicks, niceTicks, systemTicks, idleTicks, + iowTicks, irqTicks, sirqTicks, stealTicks, guestNiceTicks); +#endif vm_fprintf is nowhere defined and will not build when activated. Seems to be a relict. We call it only from here and from os_perf_aix.cpp. Please remove it from os_aix.cpp too. - Backward compatibility: I do not see the value in "has_steal_ticks". See manpage for /proc: http://man7.org/linux/man-pages/man5/proc.5.html Every item past the "idle" column in /proc/stat has a (admittedly old) kernel version dependency. Why do you think it is important to communicate the existence of "steal", but not of the items before (introduced before Linux 2.6.11)? Seems arbitrary. I mean, lets do this either right or not at all. If done right, I would prefer my suggested solution which somehow maps an INVALID value onto the output value. If you dislike my prior proposal (value_t) you could just define a special value to mean INVALID. E.g. #define INVALID_VALUE ((uint64_t)(0xFFFFFFFFFFFFFFFFUL)) or similar. Alternatively, we could decide to just not care. For values we do not find we could report zero (code does that now already). You may report "0" steals and not know if thats true or if the kernel is just too old to tell you, but again, that is the case for other values too. ------ BTW, this has nothing to do with your change, but this whole desig seems inefficient: In os_perf_linux.cpp - which is used e.g. from JFR - we call "os::Linux::get_tick_information" for every expected cpu, and so we parse /proc/stat n times for one reading. We could instead just have a function like os::Linux::get_tick_information just return all information in one go. Would be way easier. Cheers, Thomas On Fri, Mar 29, 2019 at 8:56 AM Baesken, Matthias wrote: > Hi David , thanks for your comments . > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.2/ > > moved os::CPUPerfTicks to os::Linux::CPUPerfTicks > removed DEBUG_LINUX_PROC_STAT from os_linux.cpp > addressed the minor comments > > > Best regards, Matthias > > > > -----Original Message----- > > From: David Holmes > > Sent: Freitag, 29. M?rz 2019 06:07 > > To: Baesken, Matthias ; Thomas St?fe > > > > Cc: hotspot-dev at openjdk.java.net > > Subject: Re: RFR: 8221535: add steal tick related information to > hs_error file > > [linux] > > > > Hi Matthias, > > > > This generally seems okay. > > > > On 28/03/2019 2:50 am, Baesken, Matthias wrote: > > > Hello, here is a second webrev : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.1/ > > > > > > I followed Davids idea to use the os_perf coding . > > > For reusing I moved os::Linux::get_tick_information (former name > > get_total_ticks , but it is now not only total but also steal ) > into os_linux . > > > The CPUPerfTicks struct has been enhanced by steal and a > boolean flag > > has_steal_ticks . > > > > > > 28 struct CPUPerfTicks { > > > 29 uint64_t used; > > > 30 uint64_t usedKernel; > > > 31 uint64_t total; > > > 32 uint64_t steal; > > > 33 bool has_steal_ticks; > > > 34 }; > > > > Okay, but one minor nit: why put struct CPUPerfTicks in the os:: > > namespace with the code that uses it in the os::Linux namespace? > > > > A few other minor comments: > > > > src/hotspot/os/linux/os_linux.cpp > > > > + if (-1 == which_logical_cpu) { > > > > hotspot style puts the variable first in a comparison against a constant. > > > > + #ifdef DEBUG_LINUX_PROC_STAT > > > > This should either just be DEBUG code or else deleted. The vm_printfs > > would ideally be logging statements if that is possible. > > > > + if (n > expected_tickinfo_count+3) { > > > > Spaces around + please. > > > > + st->print_cr("Steal ticks from vm start: %llu", (long long > > unsigned) steal_ticks_difference); > > > > %llu should be UINT64_FORMAT. > > The cast should not be needed. > > > > Thanks, > > David > > ----- > > > > > > > > I found the "big API" idea a bit too much for just 2 use > cases . > > > > > > > > > Best regards , Matthias > > > > > From thomas.stuefe at gmail.com Fri Mar 29 08:47:57 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 29 Mar 2019 09:47:57 +0100 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: <7eb52b3a-bd9b-62ba-9451-54cb42871ce0@redhat.com> References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> <7eb52b3a-bd9b-62ba-9451-54cb42871ce0@redhat.com> Message-ID: Thanks Andrew. Looks all good to me now. Cheers, Thomas On Fri, Mar 29, 2019 at 9:43 AM Andrew Dinn wrote: > On 28/03/2019 17:09, Thomas St?fe wrote: > > src/hotspot/share/classfile/javaClasses.cpp > > > > Pure nitpicking, but you could remove the member variables too and just > > set the Unsafe members directly, e.g.: > > > > if (fd->name() == vmSymbols::address_size_name()) { > > mirror->int_field_put(fd->offset(), (jint)sizeof(void*)); > > } else if (fd->name() == vmSymbols::page_size_name()) { > > mirror->int_field_put(fd->offset(), (jint) os::vm_page_size()); > > .. > > > > and so on. But I leave this up to you. This is already better than > before. > > I thought about that but it seemed to me to be better for clarity and > readability to init all the members in a single block in the constructor > rather than spread the computations of the field values through the code > that updates the fields. I think it's easier that way to see where all > the values are coming from and align them up with the fields in the > target class. > > > + }else { > > space missing before else > Yes. > > > src/hotspot/share/classfile/vmSymbols.hpp > > "UNALIGNED_ACCESS" off by one space > > Yes. > > > src/java.base/share/classes/java/nio/channels/FileChannel.java > > > > This may have creeped in from another change. > > Ah, yes. That was the next patch that I forgot to back out. I have > updated the webrev in place to remove it and also fix the whitespace & > alignment issues. > > > Good otherwise. > Thanks. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From matthias.baesken at sap.com Fri Mar 29 10:01:58 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 29 Mar 2019 10:01:58 +0000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: <6e54431a-8e23-160a-5757-948f1041819e@oracle.com> Message-ID: * I would completely remove this section: * +#ifdef DEBUG_LINUX_PROC_STAT * + vm_fprintf(stderr, "[stat] read " * Hi Thomas, There are no vm_printf any more in http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.2/src/hotspot/os/linux/os_linux.cpp.frames.html they have been removed already. * Every item past the "idle" column in /proc/stat has a (admittedly old) kernel version dependency. * Why do you think it is important to communicate the existence of "steal", In case we are sure that we are always on 2.6.11+ kernels, then indeed I can remove the special handling. I was sure that we are on 2.6+ but only 95% sure that we are on 2.6.11+ . >BTW, this has nothing to do with your change, but this whole desig seems inefficient: >In os_perf_linux.cpp - which is used e.g. from JFR - we call "os::Linux::get_tick_information" for every expected cpu, and so we parse /proc/stat n times for one reading. Sure this could/should be addressed ; but in another change . Regards, Matthias From: Thomas St?fe Sent: Freitag, 29. M?rz 2019 09:44 To: Baesken, Matthias Cc: David Holmes ; hotspot-dev at openjdk.java.net Subject: Re: RFR: 8221535: add steal tick related information to hs_error file [linux] Hi Matthias, as usual, thanks for your patience and perseverance. Unfortunately you touched a piece of coding which was not very pretty (IMHO) in the first place, so there are a number of further remarks: - Return type: please use a simple bool value, not OSReturn. Keeps this in line with all other OS functions. Plus I do not see us returning anything else than OS_OK and OS_ERR anyway, and callers seem not to care either. - Please use lowercase-underscore naming of the return struct, not camel case. Again, like all other os functions around. I suggest a simple "cpu_ticks". - Please change parameter order to get_tick_information and make the default of "which_logical_cpu" -1. And add a small comment that -1 means "all". // which_logical_cpu=-1 returns accumulated ticks for all cpus. bool os::Linux::get_tick_information(cpu_ticks* out, int which_logical_cpu = -1) { - Please initialize the output structure to zero at the begin of the function. Just a simple memset is sufficient. - I would rename "expected_tickinfo_count" to "required_tickinfo_count". That clearly states that we have a number of values which are mandatory but are prepared to handle optional values too. - I would completely remove this section: +#ifdef DEBUG_LINUX_PROC_STAT + vm_fprintf(stderr, "[stat] read " + UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " + UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " \n", + userTicks, niceTicks, systemTicks, idleTicks, + iowTicks, irqTicks, sirqTicks, stealTicks, guestNiceTicks); +#endif vm_fprintf is nowhere defined and will not build when activated. Seems to be a relict. We call it only from here and from os_perf_aix.cpp. Please remove it from os_aix.cpp too. - Backward compatibility: I do not see the value in "has_steal_ticks". See manpage for /proc: http://man7.org/linux/man-pages/man5/proc.5.html Every item past the "idle" column in /proc/stat has a (admittedly old) kernel version dependency. Why do you think it is important to communicate the existence of "steal", but not of the items before (introduced before Linux 2.6.11)? Seems arbitrary. I mean, lets do this either right or not at all. If done right, I would prefer my suggested solution which somehow maps an INVALID value onto the output value. If you dislike my prior proposal (value_t) you could just define a special value to mean INVALID. E.g. #define INVALID_VALUE ((uint64_t)(0xFFFFFFFFFFFFFFFFUL)) or similar. Alternatively, we could decide to just not care. For values we do not find we could report zero (code does that now already). You may report "0" steals and not know if thats true or if the kernel is just too old to tell you, but again, that is the case for other values too. ------ BTW, this has nothing to do with your change, but this whole desig seems inefficient: In os_perf_linux.cpp - which is used e.g. from JFR - we call "os::Linux::get_tick_information" for every expected cpu, and so we parse /proc/stat n times for one reading. We could instead just have a function like os::Linux::get_tick_information just return all information in one go. Would be way easier. Cheers, Thomas On Fri, Mar 29, 2019 at 8:56 AM Baesken, Matthias > wrote: Hi David , thanks for your comments . New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.2/ moved os::CPUPerfTicks to os::Linux::CPUPerfTicks removed DEBUG_LINUX_PROC_STAT from os_linux.cpp addressed the minor comments Best regards, Matthias > -----Original Message----- > From: David Holmes > > Sent: Freitag, 29. M?rz 2019 06:07 > To: Baesken, Matthias >; Thomas St?fe > > > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8221535: add steal tick related information to hs_error file > [linux] > > Hi Matthias, > > This generally seems okay. > > On 28/03/2019 2:50 am, Baesken, Matthias wrote: > > Hello, here is a second webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.1/ > > > > I followed Davids idea to use the os_perf coding . > > For reusing I moved os::Linux::get_tick_information (former name > get_total_ticks , but it is now not only total but also steal ) into os_linux . > > The CPUPerfTicks struct has been enhanced by steal and a boolean flag > has_steal_ticks . > > > > 28 struct CPUPerfTicks { > > 29 uint64_t used; > > 30 uint64_t usedKernel; > > 31 uint64_t total; > > 32 uint64_t steal; > > 33 bool has_steal_ticks; > > 34 }; > > Okay, but one minor nit: why put struct CPUPerfTicks in the os:: > namespace with the code that uses it in the os::Linux namespace? > > A few other minor comments: > > src/hotspot/os/linux/os_linux.cpp > > + if (-1 == which_logical_cpu) { > > hotspot style puts the variable first in a comparison against a constant. > > + #ifdef DEBUG_LINUX_PROC_STAT > > This should either just be DEBUG code or else deleted. The vm_printfs > would ideally be logging statements if that is possible. > > + if (n > expected_tickinfo_count+3) { > > Spaces around + please. > > + st->print_cr("Steal ticks from vm start: %llu", (long long > unsigned) steal_ticks_difference); > > %llu should be UINT64_FORMAT. > The cast should not be needed. > > Thanks, > David > ----- > > > > > I found the "big API" idea a bit too much for just 2 use cases . > > > > > > Best regards , Matthias > > From adinn at redhat.com Fri Mar 29 10:40:06 2019 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 29 Mar 2019 10:40:06 +0000 Subject: RFR: 8221477: Inject os/cpu-specific constants into Unsafe from JVM In-Reply-To: <945d9546-5195-ab33-0e20-57871c2f1ef9@oracle.com> References: <2fc62430-2e2e-9ee5-ccf1-f1cde5cea416@redhat.com> <8b0f5b4c-0386-1440-6939-646daf36aa55@redhat.com> <24b4bb72-732e-0447-d1be-dd71507e7787@redhat.com> <945d9546-5195-ab33-0e20-57871c2f1ef9@oracle.com> Message-ID: <2a26715f-fa64-9de4-1071-b18d07c05511@redhat.com> Hi David, Thanks very much for reviewing this patch. On 29/03/2019 01:25, David Holmes wrote: > This seems fine in general but I have a few queries on some details: > > src/hotspot/share/classfile/javaClasses.hpp > > ??? f(java_lang_Thread) \ > +?? f(jdk_internal_misc_UnsafeConstants) \ > ??? f(java_lang_ThreadGroup) \ > > Is there a reason this needs to be shoved in there? Similarly with > src/hotspot/share/classfile/systemDictionary.hpp: > > ?? do_klass(Thread_klass, > java_lang_Thread????????????????????????????????????? ) \ > +?? do_klass(UnsafeConstants_klass, > jdk_internal_misc_UnsafeConstants???????????????????? ) \ > ??? do_klass(ThreadGroup_klass, > java_lang_ThreadGroup???????????????????????????????? ) \ > > ? I'm not sure what you are asking here. Are you talking about the positioning of these entries? If so then the reason for choosing those positions was because they match the position of the corresponding class initialization calls in the file thread.cpp. As to that choice ... see below. > src/hotspot/share/runtime/thread.cpp > > ??? main_thread->set_threadObj(thread_object); > + > +?? // initialize the hardware-specific constants needed by Unsafe > +?? initialize_class(vmSymbols::jdk_internal_misc_UnsafeConstants(), > CHECK); > +?? jdk_internal_misc_UnsafeConstants::set_unsafe_constants(); > + > ??? // Set thread status to running since main thread has > ??? // been started and running. > ??? java_lang_Thread::set_thread_status(thread_object, > > That seems a very odd place to insert the new initialization. I can't > see any reason you'd need to split the Thread object initialization like > that. ?? Well, yes, indeed :-). I was not sure where this init needed to go so I simply put it in as early as I thought was safe. Clearly, it is an interloper into intitialise_java_lang_classes() but I was not sure where else was more appropriate. I don't think it matters too much where this init happens so long as it is early enough to precede any clinit dependencies on Unsafe (see below). I'm very happy to take advice on this (indeed I was hoping/ expecting it would be brought up at review) and also on where the entries in the headers would best be placed. > More generally exactly when do you need to initialize this new class by > and how does the initialization order change before/after this fix? (I'm > expecting only to see the new class inserted where needed without any > other perturbations.) As I said, I put the class initialization in at this early point simply to guarantee that the constants are available before Unsafe or any class which might recursively initialize Unsafe could end up needing them. I am sure they could move later and also earlier, although the latter would probably not make any sense. The important thing is that they don't really have any hard dependencies on other class inits apart, perhaps, from 'magic' classes like Object, Class etc which need to exist in order to init /any/ other class. I deliberately factored these constants out of Unsafe into a separate all static class UnsafeConstants so as to decouple this init from any current or future dependencies on Unsafe (and also to make them more clearly visible). Since the fields only hold os/hw specific constants they can be set any time after VM_Version/CPU-specific init and OS-specific init has completed. > src/hotspot/share/classfile/vmSymbols.hpp > > +?? template(big_endian_name,?????????????????????????? "BIG_ENDIAN") > ?????????????????????????? \ > +?? template(use_unaligned_access_name, > "UNALIGNED_ACCESS")??????????????????????? \ > > Nit: There's an extra space before "UNALIGNED... Thanks. Thomas Stuefe already spotted that and I have updated the webrev in place to fix it. > src/java.base/share/classes/jdk/internal/misc/UnsafeConstants.java > > 31? * package-protected? ... > > s/protected/private/ Thanks, will correct it. > ?37? * The JVM injects values into all the static fields of this class > ?... > ?43? * hardware-specific constants installed by the JVM. > > I get this gist of this note but still found it rather difficult to > intepret. There are I think two key points: > > 1. The fields must not be constant variables, hence the need for the > static initialization block. > 2. The true value of each field is injected by the VM. > > How about: > > * The JVM injects hardware-specific values into all the static fields > * of this class during JVM initialization. The static initialization > * block exists to prevent the fields from being considered constant > * variables, so the field values will be not be compiled directly into > * any class that uses them. Yes, I agree that is clearer. > 60????? * @implNote > 61????? * The actual value for this field is injected by JVM. A static > 62????? * initialization block is used to set the value here to > 63????? * communicate that this static final field is not statically > 64????? * foldable, and to avoid any possible circular dependency during > 65????? * vm initialization. > > I think all you need for each field is just: > > * @implNote > * The actual value for this field is injected by _the_ JVM. Yes, also better. > 85????? * flag whose value ... > 92????? * flag whose value ... > > s/flag/Flag/ Thanks, will correct this. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From robbin.ehn at oracle.com Fri Mar 29 13:19:49 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 29 Mar 2019 14:19:49 +0100 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: Message-ID: <282d3eca-b715-dd43-16ae-586fa7a4621e@oracle.com> Hi, 434 for (; JavaThread *thr = jtiwh.next(); ) { 435 if (thr!=thr_cur && thr->thread_state() == _thread_in_native) { 436 num_active++; 437 if (thr->is_Compiler_thread()) { 438 CompilerThread* ct = (CompilerThread*) thr; 439 if (ct->compiler() == NULL || !ct->compiler()->is_jvmci()) { 440 num_active_compiler_thread++; 441 } else { 442 // When using a Java based JVMCI compiler, it's possible 443 // for one compiler thread to grab a Java lock, enter 444 // HotSpot and go to sleep on the shutdown safepoint. 445 // Another JVMCI compiler thread can then attempt grab 446 // the lock and thus never make progress. 447 } 448 } 449 } 450 } We inc num_active on threads in native. If such thread is a compiler thread we also inc num_active_compiler_thread. JavaThread blocking on safepoint would be state blocked. JavaThread waiting on the 'Java lock' would also be blocked. Why are you not blocked when waiting on that contended Java lock ? /Robbin On 3/28/19 8:15 PM, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8220623 > http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ > > Update JVMCI to support pre-compiled as shared library Graal. > Using aoted Graal can offers benefits including: > ?- fast startup > ?- compile time similar to native JIt compilers (C2) > ?- memory usage disjoint from the application Java heap > ?- no profile pollution of JDK code used by the application > > This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. > Changes were collected in Metropolis repo [2] and tested there. > > Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler > group. > Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. > > I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In > this set Graal was tested only in tier3. > > And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our > system. Several issue were found which were present before these changes. > > Thanks, > Vladimir > > [1] > https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af > > [2] http://hg.openjdk.java.net/metropolis/dev/ From matthias.baesken at sap.com Fri Mar 29 13:37:11 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 29 Mar 2019 13:37:11 +0000 Subject: RFR: 8221535: add steal tick related information to hs_error file [linux] In-Reply-To: References: <6e54431a-8e23-160a-5757-948f1041819e@oracle.com> Message-ID: Hello, new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.3/ changed the function to bool os::Linux::get_tick_information(cpu_ticks* out, int which_logical_cpu = -1) added memset at beginning of get_tick_information removed the remaining vm_printf from src/hotspot/os/aix/os_perf_aix.cpp (at first I did not notice it was there as well ). [ kept the struct naming CPUPerfTicks ( I find plenty of such struct name styles in hotspot codebase ). ] > >In case we are sure that we are always on 2.6.11+ kernels, then indeed I can remove the special handling. >I was sure that we are on 2.6+ but only 95% sure that we are on 2.6.11+ . > I kept the has_steal_ticks for now; in case someone can confirm 2.6.11+ for jdk13 I would remove it . Best regards, Matthias From: Baesken, Matthias Sent: Freitag, 29. M?rz 2019 11:02 To: 'Thomas St?fe' Cc: David Holmes ; hotspot-dev at openjdk.java.net Subject: RE: RFR: 8221535: add steal tick related information to hs_error file [linux] * I would completely remove this section: * +#ifdef DEBUG_LINUX_PROC_STAT * + vm_fprintf(stderr, "[stat] read " * Hi Thomas, There are no vm_printf any more in http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.2/src/hotspot/os/linux/os_linux.cpp.frames.html they have been removed already. * Every item past the "idle" column in /proc/stat has a (admittedly old) kernel version dependency. * Why do you think it is important to communicate the existence of "steal", In case we are sure that we are always on 2.6.11+ kernels, then indeed I can remove the special handling. I was sure that we are on 2.6+ but only 95% sure that we are on 2.6.11+ . >BTW, this has nothing to do with your change, but this whole desig seems inefficient: >In os_perf_linux.cpp - which is used e.g. from JFR - we call "os::Linux::get_tick_information" for every expected cpu, and so we parse /proc/stat n times for one reading. Sure this could/should be addressed ; but in another change . Regards, Matthias From: Thomas St?fe > Sent: Freitag, 29. M?rz 2019 09:44 To: Baesken, Matthias > Cc: David Holmes >; hotspot-dev at openjdk.java.net Subject: Re: RFR: 8221535: add steal tick related information to hs_error file [linux] Hi Matthias, as usual, thanks for your patience and perseverance. Unfortunately you touched a piece of coding which was not very pretty (IMHO) in the first place, so there are a number of further remarks: - Return type: please use a simple bool value, not OSReturn. Keeps this in line with all other OS functions. Plus I do not see us returning anything else than OS_OK and OS_ERR anyway, and callers seem not to care either. - Please use lowercase-underscore naming of the return struct, not camel case. Again, like all other os functions around. I suggest a simple "cpu_ticks". - Please change parameter order to get_tick_information and make the default of "which_logical_cpu" -1. And add a small comment that -1 means "all". // which_logical_cpu=-1 returns accumulated ticks for all cpus. bool os::Linux::get_tick_information(cpu_ticks* out, int which_logical_cpu = -1) { - Please initialize the output structure to zero at the begin of the function. Just a simple memset is sufficient. - I would rename "expected_tickinfo_count" to "required_tickinfo_count". That clearly states that we have a number of values which are mandatory but are prepared to handle optional values too. - I would completely remove this section: +#ifdef DEBUG_LINUX_PROC_STAT + vm_fprintf(stderr, "[stat] read " + UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " + UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " " UINT64_FORMAT " \n", + userTicks, niceTicks, systemTicks, idleTicks, + iowTicks, irqTicks, sirqTicks, stealTicks, guestNiceTicks); +#endif vm_fprintf is nowhere defined and will not build when activated. Seems to be a relict. We call it only from here and from os_perf_aix.cpp. Please remove it from os_aix.cpp too. - Backward compatibility: I do not see the value in "has_steal_ticks". See manpage for /proc: http://man7.org/linux/man-pages/man5/proc.5.html Every item past the "idle" column in /proc/stat has a (admittedly old) kernel version dependency. Why do you think it is important to communicate the existence of "steal", but not of the items before (introduced before Linux 2.6.11)? Seems arbitrary. I mean, lets do this either right or not at all. If done right, I would prefer my suggested solution which somehow maps an INVALID value onto the output value. If you dislike my prior proposal (value_t) you could just define a special value to mean INVALID. E.g. #define INVALID_VALUE ((uint64_t)(0xFFFFFFFFFFFFFFFFUL)) or similar. Alternatively, we could decide to just not care. For values we do not find we could report zero (code does that now already). You may report "0" steals and not know if thats true or if the kernel is just too old to tell you, but again, that is the case for other values too. ------ BTW, this has nothing to do with your change, but this whole desig seems inefficient: In os_perf_linux.cpp - which is used e.g. from JFR - we call "os::Linux::get_tick_information" for every expected cpu, and so we parse /proc/stat n times for one reading. We could instead just have a function like os::Linux::get_tick_information just return all information in one go. Would be way easier. Cheers, Thomas On Fri, Mar 29, 2019 at 8:56 AM Baesken, Matthias > wrote: Hi David , thanks for your comments . New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.2/ moved os::CPUPerfTicks to os::Linux::CPUPerfTicks removed DEBUG_LINUX_PROC_STAT from os_linux.cpp addressed the minor comments Best regards, Matthias > -----Original Message----- > From: David Holmes > > Sent: Freitag, 29. M?rz 2019 06:07 > To: Baesken, Matthias >; Thomas St?fe > > > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8221535: add steal tick related information to hs_error file > [linux] > > Hi Matthias, > > This generally seems okay. > > On 28/03/2019 2:50 am, Baesken, Matthias wrote: > > Hello, here is a second webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221535.1/ > > > > I followed Davids idea to use the os_perf coding . > > For reusing I moved os::Linux::get_tick_information (former name > get_total_ticks , but it is now not only total but also steal ) into os_linux . > > The CPUPerfTicks struct has been enhanced by steal and a boolean flag > has_steal_ticks . > > > > 28 struct CPUPerfTicks { > > 29 uint64_t used; > > 30 uint64_t usedKernel; > > 31 uint64_t total; > > 32 uint64_t steal; > > 33 bool has_steal_ticks; > > 34 }; > > Okay, but one minor nit: why put struct CPUPerfTicks in the os:: > namespace with the code that uses it in the os::Linux namespace? > > A few other minor comments: > > src/hotspot/os/linux/os_linux.cpp > > + if (-1 == which_logical_cpu) { > > hotspot style puts the variable first in a comparison against a constant. > > + #ifdef DEBUG_LINUX_PROC_STAT > > This should either just be DEBUG code or else deleted. The vm_printfs > would ideally be logging statements if that is possible. > > + if (n > expected_tickinfo_count+3) { > > Spaces around + please. > > + st->print_cr("Steal ticks from vm start: %llu", (long long > unsigned) steal_ticks_difference); > > %llu should be UINT64_FORMAT. > The cast should not be needed. > > Thanks, > David > ----- > > > > > I found the "big API" idea a bit too much for just 2 use cases . > > > > > > Best regards , Matthias > > From erik.osterlund at oracle.com Fri Mar 29 13:39:23 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 29 Mar 2019 14:39:23 +0100 Subject: RFR (M) 8221183: Avoid code cache walk in MetadataOnStackMark In-Reply-To: References: Message-ID: <780ddef6-9af2-7fdb-abf0-93c4680f9a08@oracle.com> Hi Coleen, Seems like the recreate_old_table() should be called reset_old_table() as it does not really create anything. The CodeCache::mark_all_nmethods_for_deoptimization function seems like a general facility. I don't think I like passing in a boolean if class redefinition-specific stuff should be done in there or not. It's a very small method that just iterates over the nmethods. I'd rather duplicate that while loop in a new function and have the class redefinition logic be separate. Otherwise this looks good! Thanks, /Erik On 2019-03-27 14:09, coleen.phillimore at oracle.com wrote: > Summary: Note nmethods with "old" Methods in them in table to walk instead. > > See RFE for more details.? Tested with RedefineClasses tests > with/without -Xcomp, mach5 tier1-5. > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8221183.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8221183 > > Thanks, > Coleen From jamsheed.c.m at oracle.com Fri Mar 29 14:19:38 2019 From: jamsheed.c.m at oracle.com (Jamsheed) Date: Fri, 29 Mar 2019 19:49:38 +0530 Subject: RFR 8191278: MappedByteBuffer bulk access memory failures are not handled gracefully In-Reply-To: References: <8ecf4f01-b7e2-c1a7-921d-51aee6e49777@oracle.com> <571f3d72-a22a-3b52-2942-6408072e353f@oracle.com> <144af13e-33e2-2089-f8d2-902e94485eb1@oracle.com> <62ad6130-0498-5222-43c7-befa3cea9741@oracle.com> Message-ID: <9dc5704c-7f87-e231-f3cc-7508c1a6d75c@oracle.com> Hi Tobias, Please find the revised webrev. http://cr.openjdk.java.net/~jcm/8191278/webrev.02/ 1) checks were made in StubRoutines unaligned arraycopy code range. 2) tried to implement fastexit for x64(mac,linux,windows,solaris). others continue with old implementation. 3) implementation like arm has auto increment mode for from and to addresses,? even aligning is done this way. this may make to proceed with unaligned addresses. Best regards, Jamsheed On 15/03/19 1:14 PM, Tobias Hartmann wrote: > Hi Jamsheed, > > okay, that looks reasonable to me! > > Best regards, > Tobias > > On 15.03.19 01:07, Jamsheed wrote: >> Hi Tobias, >> >> as a fix, for handling? the bugs in hand written assembly stubs, i will put checks like.[1]? instead >> of StubRoutines::contains(pc). >> >> Best regards, >> >> Jamsheed >> >> [1] >> >> bool StubRoutines::is_unsafe_copy(address pc) { >> ??? if ((pc > StubRoutines::_jbyte_disjoint_arraycopy && pc < >> StubRoutines::_arrayof_jbyte_disjoint_arraycopy) || >> ??????? (pc > StubRoutines::_jshort_disjoint_arraycopy && pc < >> StubRoutines::_arrayof_jshort_disjoint_arraycopy) || >> ??????? (pc > StubRoutines::_jint_disjoint_arraycopy && pc < >> StubRoutines::_arrayof_oop_disjoint_arraycopy)) { >> ????? return true; >> ??? } >> ??? return false; >> ? } >> >> PS: these code ranges are shared by others too. >> >> On 14/03/19 8:46 PM, Jamsheed wrote: >>> Hi Tobias, >>> >>> On 14/03/19 1:58 PM, Tobias Hartmann wrote: >>>> Hi Jamsheed, >>>> >>>> On 14.03.19 02:29, Jamsheed wrote: >>>>> i will try to limit my checks to array copy stubs. will make changes and send it for review. >>>> Okay, if there's a simple way to do that. Otherwise, I'm fine with the current version of the fix. >>> either i should have separate code generated for unsafe copy. or setting/resetting >>> doing_unsafe_access before after unsafe_copy should do. >>> >>> Best regards, >>> >>> Jamsheed >>>> Best regards, >>>> Tobias From coleen.phillimore at oracle.com Fri Mar 29 14:54:02 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 29 Mar 2019 10:54:02 -0400 Subject: RFR (M) 8221183: Avoid code cache walk in MetadataOnStackMark In-Reply-To: <780ddef6-9af2-7fdb-abf0-93c4680f9a08@oracle.com> References: <780ddef6-9af2-7fdb-abf0-93c4680f9a08@oracle.com> Message-ID: Hi Erik, Thank you for reviewing this! On 3/29/19 9:39 AM, Erik ?sterlund wrote: > Hi Coleen, > > Seems like the recreate_old_table() should be called reset_old_table() > as it does not really create anything. > I can change the name.? I guess the table isn't old but the methods are.? So I could call it reset_old_method_table instead. > The CodeCache::mark_all_nmethods_for_deoptimization function seems > like a general facility. I don't think I like passing in a boolean if > class redefinition-specific stuff should be done in there or not. It's > a very small method that just iterates over the nmethods. I'd rather > duplicate that while loop in a new function and have the class > redefinition logic be separate. > Yes, I could do this too.? I hate passing a boolean to a function to do something different.? Sort of implies one should write a different function in the first place. Thanks! Coleen > Otherwise this looks good! > > Thanks, > /Erik > > On 2019-03-27 14:09, coleen.phillimore at oracle.com wrote: >> Summary: Note nmethods with "old" Methods in them in table to walk >> instead. >> >> See RFE for more details.? Tested with RedefineClasses tests >> with/without -Xcomp, mach5 tier1-5. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8221183.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8221183 >> >> Thanks, >> Coleen From doug.simon at oracle.com Fri Mar 29 15:16:36 2019 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 29 Mar 2019 16:16:36 +0100 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: Message-ID: <82C212AE-CE8A-4B6A-A1A0-DD08B41A06A5@oracle.com> Hi Robbin, > From: Robbin Ehn > > Hi, > > 434 for (; JavaThread *thr = jtiwh.next(); ) { > 435 if (thr!=thr_cur && thr->thread_state() == _thread_in_native) { > 436 num_active++; > 437 if (thr->is_Compiler_thread()) { > 438 CompilerThread* ct = (CompilerThread*) thr; > 439 if (ct->compiler() == NULL || !ct->compiler()->is_jvmci()) { > 440 num_active_compiler_thread++; > 441 } else { > 442 // When using a Java based JVMCI compiler, it's possible > 443 // for one compiler thread to grab a Java lock, enter > 444 // HotSpot and go to sleep on the shutdown safepoint. > 445 // Another JVMCI compiler thread can then attempt grab > 446 // the lock and thus never make progress. > 447 } > 448 } > 449 } > 450 } > > We inc num_active on threads in native. > If such thread is a compiler thread we also inc num_active_compiler_thread. > JavaThread blocking on safepoint would be state blocked. > JavaThread waiting on the 'Java lock' would also be blocked. > > Why are you not blocked when waiting on that contended Java lock ? This change was made primarily in the context of libgraal. It can happen that a JVMCI compiler thread acquires a lock in libgraal, enters HotSpot and goes to sleep in the shutdown safepoint. Another JVMCI compiler thread then attempts to acquire the same lock and goes to sleep in libgraal which from HotSpot?s perspective is the _thread_in_native state. This is the original fix I had for this: CompilerThread* ct = (CompilerThread*) thr; if (ct->compiler() == NULL || !ct->compiler()->is_jvmci() JVMCI_ONLY(|| !UseJVMCINativeLibrary)) { num_active_compiler_thread++; } else { // When using a compiler in a JVMCI shared library, it's possible // for one compiler thread to grab a lock in the shared library, // enter HotSpot and go to sleep on the shutdown safepoint. Another // JVMCI shared library compiler thread can then attempt to grab the // lock and thus never make progress. } which is probably the right one. I hadn?t realized that a JavaGraal (as opposed to libgraal) JVMCI compiler thread blocked on a lock will be in the blocked state, not in the _thread_in_native state. -Doug From martin.doerr at sap.com Fri Mar 29 15:33:50 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 29 Mar 2019 15:33:50 +0000 Subject: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? In-Reply-To: <58dcefc5-4bdf-9994-f0b6-484d627d5cfa@oracle.com> References: <58dcefc5-4bdf-9994-f0b6-484d627d5cfa@oracle.com> Message-ID: Hi Matthias, looks good. Best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of Vladimir Kozlov Sent: Mittwoch, 27. M?rz 2019 17:36 To: hotspot-dev at openjdk.java.net Subject: Re: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? Changes looks good. It was part of debug functionality in C2 which was removed in jdk7u but adlc part was not: http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/a5dd6e3ef9f3#l6.7 Thanks, Vladimir On 3/27/19 2:29 AM, Baesken, Matthias wrote: > Hi Thomas, good point . > I removed the 2 functions . > Btw. I checked them (using nm ) in the product libjvm.so and they showed up in the binary . > So it might help also to get the binary a little bit smaller ? > > > Please review ! > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8221541 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8221541.0/ > > > Thanks, Matthias > > > From: Thomas St?fe > Sent: Dienstag, 26. M?rz 2019 20:40 > To: Baesken, Matthias > Cc: hotspot-dev at openjdk.java.net > Subject: Re: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? > > Hi Matthias, > > On Tue, Mar 26, 2019 at 4:17 PM Baesken, Matthias > wrote: > > Hello, it seems that the fopen calls in CompilerOracle::append_comment_to_file and > CompilerOracle::append_exclude_to_file do not have a corresponding fclose. > Or do I miss something ? > > No, you are right, this is a bug. However, I think those functions are nowhere used, so maybe they could be removed completely. > > Cheers, Thomas > > We could also add a "true" for the second argument of the fileStream constructor calls ( fileStream(FILE* file, bool need_close = false) ) > to get the closing. > > > coding : > ------------ > > jdk/src/hotspot/share/compiler/compilerOracle.cpp > > > 741void CompilerOracle::append_comment_to_file(const char* message) { > 742 assert(has_command_file(), "command file must be specified"); > 743 fileStream stream(fopen(cc_file(), "at")); > 744 stream.print("# "); > 745 for (int index = 0; message[index] != '\0'; index++) { > 746 stream.put(message[index]); > 747 if (message[index] == '\n') stream.print("# "); > 748 } > 749 stream.cr(); > 750} > 751 > 752void CompilerOracle::append_exclude_to_file(const methodHandle& method) { > 753 assert(has_command_file(), "command file must be specified"); > 754 fileStream stream(fopen(cc_file(), "at")); > 755 stream.print("exclude "); > 756 method->method_holder()->name()->print_symbol_on(&stream); > 757 stream.print("."); > 758 method->name()->print_symbol_on(&stream); > 759 method->signature()->print_symbol_on(&stream); > 760 stream.cr(); > 761 stream.cr(); > 762} > > Best regards, Matthias > From coleen.phillimore at oracle.com Fri Mar 29 15:41:52 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 29 Mar 2019 11:41:52 -0400 Subject: RFR (M) 8221183: Avoid code cache walk in MetadataOnStackMark In-Reply-To: References: <780ddef6-9af2-7fdb-abf0-93c4680f9a08@oracle.com> Message-ID: <557dd7b9-99bf-1613-1b89-a39dc993371e@oracle.com> On 3/29/19 10:54 AM, coleen.phillimore at oracle.com wrote: > > Hi Erik, > Thank you for reviewing this! > > On 3/29/19 9:39 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Seems like the recreate_old_table() should be called >> reset_old_table() as it does not really create anything. >> > > I can change the name.? I guess the table isn't old but the methods > are.? So I could call it reset_old_method_table instead. > >> The CodeCache::mark_all_nmethods_for_deoptimization function seems >> like a general facility. I don't think I like passing in a boolean if >> class redefinition-specific stuff should be done in there or not. >> It's a very small method that just iterates over the nmethods. I'd >> rather duplicate that while loop in a new function and have the class >> redefinition logic be separate. >> > > Yes, I could do this too.? I hate passing a boolean to a function to > do something different.? Sort of implies one should write a different > function in the first place. I made these changes (added functionmark_all_nmethods_for_evol_deoptimization) and moved all the redefinition functions in codeCache.cpp together under #ifndef INCLUDE_JVMTI. http://cr.openjdk.java.net/~coleenp/2019/8221183.02/webrev/index.html Thanks, Coleen > > Thanks! > Coleen >> Otherwise this looks good! >> >> Thanks, >> /Erik >> >> On 2019-03-27 14:09, coleen.phillimore at oracle.com wrote: >>> Summary: Note nmethods with "old" Methods in them in table to walk >>> instead. >>> >>> See RFE for more details.? Tested with RedefineClasses tests >>> with/without -Xcomp, mach5 tier1-5. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2019/8221183.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8221183 >>> >>> Thanks, >>> Coleen > From matthias.baesken at sap.com Fri Mar 29 16:00:40 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 29 Mar 2019 16:00:40 +0000 Subject: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? In-Reply-To: References: <58dcefc5-4bdf-9994-f0b6-484d627d5cfa@oracle.com>, Message-ID: <8B103531-14BE-440D-97B0-5CB80220F7E6@sap.com> Thanks! Can this be pushed as XS change with 1 review? Von meinem iPhone gesendet > Am 29.03.2019 um 16:33 schrieb Doerr, Martin : > > Hi Matthias, > > looks good. > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of Vladimir Kozlov > Sent: Mittwoch, 27. M?rz 2019 17:36 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? > > Changes looks good. > > It was part of debug functionality in C2 which was removed in jdk7u but adlc part was not: > > http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/a5dd6e3ef9f3#l6.7 > > Thanks, > Vladimir > >> On 3/27/19 2:29 AM, Baesken, Matthias wrote: >> Hi Thomas, good point . >> I removed the 2 functions . >> Btw. I checked them (using nm ) in the product libjvm.so and they showed up in the binary . >> So it might help also to get the binary a little bit smaller ? >> >> >> Please review ! >> >> Bug/webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8221541 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8221541.0/ >> >> >> Thanks, Matthias >> >> >> From: Thomas St?fe >> Sent: Dienstag, 26. M?rz 2019 20:40 >> To: Baesken, Matthias >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? >> >> Hi Matthias, >> >> On Tue, Mar 26, 2019 at 4:17 PM Baesken, Matthias > wrote: >> >> Hello, it seems that the fopen calls in CompilerOracle::append_comment_to_file and >> CompilerOracle::append_exclude_to_file do not have a corresponding fclose. >> Or do I miss something ? >> >> No, you are right, this is a bug. However, I think those functions are nowhere used, so maybe they could be removed completely. >> >> Cheers, Thomas >> >> We could also add a "true" for the second argument of the fileStream constructor calls ( fileStream(FILE* file, bool need_close = false) ) >> to get the closing. >> >> >> coding : >> ------------ >> >> jdk/src/hotspot/share/compiler/compilerOracle.cpp >> >> >> 741void CompilerOracle::append_comment_to_file(const char* message) { >> 742 assert(has_command_file(), "command file must be specified"); >> 743 fileStream stream(fopen(cc_file(), "at")); >> 744 stream.print("# "); >> 745 for (int index = 0; message[index] != '\0'; index++) { >> 746 stream.put(message[index]); >> 747 if (message[index] == '\n') stream.print("# "); >> 748 } >> 749 stream.cr(); >> 750} >> 751 >> 752void CompilerOracle::append_exclude_to_file(const methodHandle& method) { >> 753 assert(has_command_file(), "command file must be specified"); >> 754 fileStream stream(fopen(cc_file(), "at")); >> 755 stream.print("exclude "); >> 756 method->method_holder()->name()->print_symbol_on(&stream); >> 757 stream.print("."); >> 758 method->name()->print_symbol_on(&stream); >> 759 method->signature()->print_symbol_on(&stream); >> 760 stream.cr(); >> 761 stream.cr(); >> 762} >> >> Best regards, Matthias >> From martin.doerr at sap.com Fri Mar 29 16:02:40 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 29 Mar 2019 16:02:40 +0000 Subject: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? In-Reply-To: <8B103531-14BE-440D-97B0-5CB80220F7E6@sap.com> References: <58dcefc5-4bdf-9994-f0b6-484d627d5cfa@oracle.com>, <8B103531-14BE-440D-97B0-5CB80220F7E6@sap.com> Message-ID: Could get pushed under trivial rule. But you already have 2 reviews. Best regards, Martin -----Original Message----- From: Baesken, Matthias Sent: Freitag, 29. M?rz 2019 17:01 To: Doerr, Martin Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? Thanks! Can this be pushed as XS change with 1 review? Von meinem iPhone gesendet > Am 29.03.2019 um 16:33 schrieb Doerr, Martin : > > Hi Matthias, > > looks good. > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of Vladimir Kozlov > Sent: Mittwoch, 27. M?rz 2019 17:36 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8221541: clean up functions in CompilerOracle - was : RE: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? > > Changes looks good. > > It was part of debug functionality in C2 which was removed in jdk7u but adlc part was not: > > http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/a5dd6e3ef9f3#l6.7 > > Thanks, > Vladimir > >> On 3/27/19 2:29 AM, Baesken, Matthias wrote: >> Hi Thomas, good point . >> I removed the 2 functions . >> Btw. I checked them (using nm ) in the product libjvm.so and they showed up in the binary . >> So it might help also to get the binary a little bit smaller ? >> >> >> Please review ! >> >> Bug/webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8221541 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8221541.0/ >> >> >> Thanks, Matthias >> >> >> From: Thomas St?fe >> Sent: Dienstag, 26. M?rz 2019 20:40 >> To: Baesken, Matthias >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: CompilerOracle::append_comment_to_file and CompilerOracle::append_exclude_to_file - missing fclose calls ? >> >> Hi Matthias, >> >> On Tue, Mar 26, 2019 at 4:17 PM Baesken, Matthias > wrote: >> >> Hello, it seems that the fopen calls in CompilerOracle::append_comment_to_file and >> CompilerOracle::append_exclude_to_file do not have a corresponding fclose. >> Or do I miss something ? >> >> No, you are right, this is a bug. However, I think those functions are nowhere used, so maybe they could be removed completely. >> >> Cheers, Thomas >> >> We could also add a "true" for the second argument of the fileStream constructor calls ( fileStream(FILE* file, bool need_close = false) ) >> to get the closing. >> >> >> coding : >> ------------ >> >> jdk/src/hotspot/share/compiler/compilerOracle.cpp >> >> >> 741void CompilerOracle::append_comment_to_file(const char* message) { >> 742 assert(has_command_file(), "command file must be specified"); >> 743 fileStream stream(fopen(cc_file(), "at")); >> 744 stream.print("# "); >> 745 for (int index = 0; message[index] != '\0'; index++) { >> 746 stream.put(message[index]); >> 747 if (message[index] == '\n') stream.print("# "); >> 748 } >> 749 stream.cr(); >> 750} >> 751 >> 752void CompilerOracle::append_exclude_to_file(const methodHandle& method) { >> 753 assert(has_command_file(), "command file must be specified"); >> 754 fileStream stream(fopen(cc_file(), "at")); >> 755 stream.print("exclude "); >> 756 method->method_holder()->name()->print_symbol_on(&stream); >> 757 stream.print("."); >> 758 method->name()->print_symbol_on(&stream); >> 759 method->signature()->print_symbol_on(&stream); >> 760 stream.cr(); >> 761 stream.cr(); >> 762} >> >> Best regards, Matthias >> From tom.rodriguez at oracle.com Fri Mar 29 16:44:50 2019 From: tom.rodriguez at oracle.com (Tom Rodriguez) Date: Fri, 29 Mar 2019 09:44:50 -0700 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: <82C212AE-CE8A-4B6A-A1A0-DD08B41A06A5@oracle.com> References: <82C212AE-CE8A-4B6A-A1A0-DD08B41A06A5@oracle.com> Message-ID: <9f3b0352-3474-24f3-6c0f-b82e29754444@oracle.com> > This is the original fix I had for this: > > CompilerThread* ct = (CompilerThread*) thr; > if (ct->compiler() == NULL || !ct->compiler()->is_jvmci() JVMCI_ONLY(|| !UseJVMCINativeLibrary)) { > num_active_compiler_thread++; > } else { > // When using a compiler in a JVMCI shared library, it's possible > // for one compiler thread to grab a lock in the shared library, > // enter HotSpot and go to sleep on the shutdown safepoint. Another > // JVMCI shared library compiler thread can then attempt to grab the > // lock and thus never make progress. > } > > which is probably the right one. I hadn?t realized that a JavaGraal > (as opposed to libgraal) JVMCI compiler thread blocked on a lock will be in > the blocked state, not in the _thread_in_native state. I think it would be ok to go back to your original fix. tom From vladimir.kozlov at oracle.com Fri Mar 29 16:55:18 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 29 Mar 2019 09:55:18 -0700 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: <4b759550-f185-724f-7139-9bab648a1966@oracle.com> Message-ID: Stefan, Do you have a test (and flags) which can allow me to measure effect of this code on G1 remark pause? Thanks, Vladimir On 3/29/19 12:36 AM, Stefan Karlsson wrote: > On 2019-03-29 03:07, Vladimir Kozlov wrote: >> Hi Stefan, >> >> I collected some data on MetadataHandleBlock. >> >> First, do_unloading() code is executed only when class_unloading_occurred is 'true' - it is rare >> case. It should not affect normal G1 remark pause. > > It's only rare for applications that don't do dynamic class loading and unloading. The applications > that do, will be affected. > >> >> Second, I run a test with -Xcomp. I got about 10,000 compilations by Graal and next data at the >> end of execution: >> >> max_blocks = 232 >> max_handles_per_block = 32 (since handles array has 32 elements) >> max_total_alive_values = 4631 > > OK. Thanks for the info. > > StefanK > >> >> Thanks, >> Vladimir >> >> On 3/28/19 2:44 PM, Vladimir Kozlov wrote: >>> Thank you, Stefan >>> >>> On 3/28/19 12:54 PM, Stefan Karlsson wrote: >>>> Hi Vladimir, >>>> >>>> I started to check the GC code. >>>> >>>> ======================================================================== >>>> I see that you've added guarded includes in the middle of the include list: >>>> ?? #include "gc/shared/strongRootsScope.hpp" >>>> ?? #include "gc/shared/weakProcessor.hpp" >>>> + #if INCLUDE_JVMCI >>>> + #include "jvmci/jvmci.hpp" >>>> + #endif >>>> ?? #include "oops/instanceRefKlass.hpp" >>>> ?? #include "oops/oop.inline.hpp" >>>> >>>> The style we use is to put these conditional includes at the end of the include lists. >>> >>> okay >>> >>>> >>>> ======================================================================== >>>> Could you also change the following: >>>> >>>> + #if INCLUDE_JVMCI >>>> +???? // Clean JVMCI metadata handles. >>>> +???? JVMCI::do_unloading(is_alive_closure(), purged_class); >>>> + #endif >>>> >>>> to: >>>> +???? // Clean JVMCI metadata handles. >>>> +???? JVMCI_ONLY(JVMCI::do_unloading(is_alive_closure(), purged_class);) >>>> >>>> to get rid of some of the line noise in the GC files. >>> >>> okay >>> >>>> >>>> ======================================================================== >>>> In the future we will need version of JVMCI::do_unloading that supports concurrent cleaning for >>>> ZGC. >>> >>> Yes, we need to support concurrent cleaning in a future. >>> >>>> >>>> ======================================================================== >>>> What's the performance impact for G1 remark pause with this serial walk over the >>>> MetadataHandleBlock? >>>> >>>> 3275 void G1CollectedHeap::complete_cleaning(BoolObjectClosure* is_alive, >>>> 3276???????????????????????????????????????? bool class_unloading_occurred) { >>>> 3277?? uint num_workers = workers()->active_workers(); >>>> 3278?? ParallelCleaningTask unlink_task(is_alive, num_workers, class_unloading_occurred, false); >>>> 3279?? workers()->run_task(&unlink_task); >>>> 3280 #if INCLUDE_JVMCI >>>> 3281?? // No parallel processing of JVMCI metadata handles for now. >>>> 3282?? JVMCI::do_unloading(is_alive, class_unloading_occurred); >>>> 3283 #endif >>>> 3284 } >>> >>> There should not be impact if Graal is not used. Only cost of call (which most likely is inlined >>> in product VM) and check: >>> >>> http://hg.openjdk.java.net/metropolis/dev/file/530fc1427d02/src/hotspot/share/jvmci/jvmciRuntime.cpp#l1237 >>> >>> >>> If Graal is used it should not have big impact since these metadata has regular pattern (32 >>> handles per array and array per MetadataHandleBlock block which are linked in list) and not large. >>> If there will be noticeable impact - we will work on it as you suggested by using >>> ParallelCleaningTask. >>> >>>> >>>> ======================================================================== >>>> Did you consider adding it as a task for one of the worker threads to execute in >>>> ParallelCleaningTask? >>>> >>>> See how other tasks are claimed by one worker: >>>> void KlassCleaningTask::work() { >>>> ?? ResourceMark rm; >>>> >>>> ?? // One worker will clean the subklass/sibling klass tree. >>>> ?? if (claim_clean_klass_tree_task()) { >>>> ???? Klass::clean_subklass_tree(); >>>> ?? } >>> >>> These changes were ported from JDK8u based changes in graal-jvmci-8 and there are no >>> ParallelCleaningTask in JDK8. >>> >>> Your suggestion is interesting and I agree that we should investigate it. >>> >>>> >>>> ======================================================================== >>>> In MetadataHandleBlock::do_unloading: >>>> >>>> +??????? if (klass->class_loader_data()->is_unloading()) { >>>> +????????? // This needs to be marked so that it's no longer scanned >>>> +????????? // but can't be put on the free list yet. The >>>> +????????? // ReferenceCleaner will set this to NULL and >>>> +????????? // put it on the free list. >>>> >>>> I couldn't find the ReferenceCleaner in the patch or in the source. Where can I find this code? >>> >>> I think it is typo (I will fix it) - it references new HandleCleaner class: >>> >>> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HandleCleaner.java.html >>> >>> >>> Thanks, >>> Vladimir >>> >>>> >>>> Thanks, >>>> StefanK >>>> >>>> On 2019-03-28 20:15, Vladimir Kozlov wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8220623 >>>>> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ >>>>> >>>>> Update JVMCI to support pre-compiled as shared library Graal. >>>>> Using aoted Graal can offers benefits including: >>>>> ?- fast startup >>>>> ?- compile time similar to native JIt compilers (C2) >>>>> ?- memory usage disjoint from the application Java heap >>>>> ?- no profile pollution of JDK code used by the application >>>>> >>>>> This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. >>>>> Changes were collected in Metropolis repo [2] and tested there. >>>>> >>>>> Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler group. >>>>> Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. >>>>> >>>>> I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In this set Graal >>>>> was tested only in tier3. >>>>> >>>>> And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our system. Several >>>>> issue were found which were present before these changes. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> [1] https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af >>>>> [2] http://hg.openjdk.java.net/metropolis/dev/ >>>> From nils.eliasson at oracle.com Fri Mar 29 21:09:49 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 29 Mar 2019 22:09:49 +0100 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: References: Message-ID: <6f31b2e5-ab7a-cad3-d610-296e80174e01@oracle.com> Hi Vladimir, I've started going through the review. This one caught my eye: *+ if (UseJVMCINativeLibrary) {* *+ // SVM compiled code requires more stack space* *+ if (FLAG_IS_DEFAULT(CompilerThreadStackSize)) {* *+ FLAG_SET_DEFAULT(CompilerThreadStackSize, 2*M);* *+ } *CompilerThreadStackSize is in Ks, so that default will turn into 2G stacks. I guess that isn't your intention :) Regards, Nils ** On 2019-03-28 20:15, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8220623 > http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ > > Update JVMCI to support pre-compiled as shared library Graal. > Using aoted Graal can offers benefits including: > ?- fast startup > ?- compile time similar to native JIt compilers (C2) > ?- memory usage disjoint from the application Java heap > ?- no profile pollution of JDK code used by the application > > This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. > Changes were collected in Metropolis repo [2] and tested there. > > Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and > our compiler group. > Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. > > I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was > clean. In this set Graal was tested only in tier3. > > And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available > in our system. Several issue were found which were present before > these changes. > > Thanks, > Vladimir > > [1] > https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af > [2] http://hg.openjdk.java.net/metropolis/dev/ From nils.eliasson at oracle.com Fri Mar 29 21:22:12 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 29 Mar 2019 22:22:12 +0100 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: <6f31b2e5-ab7a-cad3-d610-296e80174e01@oracle.com> References: <6f31b2e5-ab7a-cad3-d610-296e80174e01@oracle.com> Message-ID: <577392af-a512-bd13-aa60-46c9dccf9723@oracle.com> I killed the formatting somehow. Second try: Hi Vladimir, I've started going through the review. This one caught my eye: compilerDefinitions.cpp: + if (UseJVMCINativeLibrary) { +?? // SVM compiled code requires more stack space +?? if (FLAG_IS_DEFAULT(CompilerThreadStackSize)) { +?? FLAG_SET_DEFAULT(CompilerThreadStackSize, 2*M); + } CompilerThreadStackSize is in Ks, so that default will turn into 2G stacks. I guess that isn't your intention :) Regards, Nils On 2019-03-29 22:09, Nils Eliasson wrote: > Hi Vladimir, I've started going through the review. This one caught my > eye: *+ if (UseJVMCINativeLibrary) {* > *+ // SVM compiled code requires more stack space* > *+ if (FLAG_IS_DEFAULT(CompilerThreadStackSize)) {* > *+ FLAG_SET_DEFAULT(CompilerThreadStackSize, 2*M);* > *+ } *CompilerThreadStackSize is in Ks, so that default will turn into > 2G stacks. I guess that isn't your intention :) Regards, Nils ** > > On 2019-03-28 20:15, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8220623 >> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ >> >> Update JVMCI to support pre-compiled as shared library Graal. >> Using aoted Graal can offers benefits including: >> ?- fast startup >> ?- compile time similar to native JIt compilers (C2) >> ?- memory usage disjoint from the application Java heap >> ?- no profile pollution of JDK code used by the application >> >> This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to >> date. >> Changes were collected in Metropolis repo [2] and tested there. >> >> Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and >> our compiler group. >> Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. >> >> I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was >> clean. In this set Graal was tested only in tier3. >> >> And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available >> in our system. Several issue were found which were present before >> these changes. >> >> Thanks, >> Vladimir >> >> [1] >> https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af >> [2] http://hg.openjdk.java.net/metropolis/dev/ From coleen.phillimore at oracle.com Fri Mar 29 21:53:49 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 29 Mar 2019 17:53:49 -0400 Subject: RFR (M) 8221183: Avoid code cache walk in MetadataOnStackMark In-Reply-To: <8970aca2-faa7-268e-6043-a7a269af4706@oracle.com> References: <8970aca2-faa7-268e-6043-a7a269af4706@oracle.com> Message-ID: <8714487a-c2a8-0b07-e7fb-7b43dbc2c090@oracle.com> Serguei, Thank you for reviewing. On 3/29/19 3:06 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > It looks good to me. > I agree with Erik on reset_old_table(). > > I have one question. > > http://cr.openjdk.java.net/~coleenp/2019/8221183.01/webrev/src/hotspot/share/code/codeCache.cpp.frames.html > Yes, I changed it to reset_old_method_table. > 1077 void CodeCache::mark_for_evol_deoptimization(InstanceKlass* > dependee) { > *1078 MutexLockerEx mu(CodeCache_lock, Mutex::_no_safepoint_check_flag);* > 1079 > 1080 // Mark dependent AOT nmethods, which are only found via the class redefined. > 1081 // TODO: add dependencies to aotCompiledMethod's metadata section > so this isn't > 1082 // needed. > 1083 AOTLoader::mark_evol_dependent_methods(dependee); > 1084 } > > Is it still necessary to grab the CodeCache_lock here? > It's actually not necessary because we're at a safepoint, and this code is always at a safepoint. I'll assert it's at a safepoint.? flush_evol_dependents has this: ? // --- Compile_lock is not held. However we are at a safepoint. ? assert_locked_or_safepoint(Compile_lock); Thanks, Coleen > > Thanks, > Serguei > > > On 3/27/19 06:09, coleen.phillimore at oracle.com wrote: >> Summary: Note nmethods with "old" Methods in them in table to walk >> instead. >> >> See RFE for more details.? Tested with RedefineClasses tests >> with/without -Xcomp, mach5 tier1-5. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8221183.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8221183 >> >> Thanks, >> Coleen > From vladimir.kozlov at oracle.com Sat Mar 30 00:23:10 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 29 Mar 2019 17:23:10 -0700 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: <577392af-a512-bd13-aa60-46c9dccf9723@oracle.com> References: <6f31b2e5-ab7a-cad3-d610-296e80174e01@oracle.com> <577392af-a512-bd13-aa60-46c9dccf9723@oracle.com> Message-ID: <28412d92-f173-0372-a2d0-4227b6f43b46@oracle.com> Thank you, Nils Yes, it looks like typo but it is the same in graal-jvmci-8 code. I asked Graal guys to make sure it is typo which should be fixed (or not). Vladimir On 3/29/19 2:22 PM, Nils Eliasson wrote: > I killed the formatting somehow. Second try: > > Hi Vladimir, > > I've started going through the review. This one caught my eye: > > compilerDefinitions.cpp: > > + if (UseJVMCINativeLibrary) { > +?? // SVM compiled code requires more stack space > +?? if (FLAG_IS_DEFAULT(CompilerThreadStackSize)) { > +?? FLAG_SET_DEFAULT(CompilerThreadStackSize, 2*M); > + } > > CompilerThreadStackSize is in Ks, so that default will turn into 2G stacks. I guess that isn't your > intention :) > > Regards, > > Nils > > On 2019-03-29 22:09, Nils Eliasson wrote: >> Hi Vladimir, I've started going through the review. This one caught my eye: *+ if >> (UseJVMCINativeLibrary) {* >> *+ // SVM compiled code requires more stack space* >> *+ if (FLAG_IS_DEFAULT(CompilerThreadStackSize)) {* >> *+ FLAG_SET_DEFAULT(CompilerThreadStackSize, 2*M);* >> *+ } *CompilerThreadStackSize is in Ks, so that default will turn into 2G stacks. I guess that >> isn't your intention :) Regards, Nils ** >> >> On 2019-03-28 20:15, Vladimir Kozlov wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8220623 >>> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ >>> >>> Update JVMCI to support pre-compiled as shared library Graal. >>> Using aoted Graal can offers benefits including: >>> ?- fast startup >>> ?- compile time similar to native JIt compilers (C2) >>> ?- memory usage disjoint from the application Java heap >>> ?- no profile pollution of JDK code used by the application >>> >>> This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. >>> Changes were collected in Metropolis repo [2] and tested there. >>> >>> Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler group. >>> Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. >>> >>> I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In this set Graal was >>> tested only in tier3. >>> >>> And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our system. Several issue >>> were found which were present before these changes. >>> >>> Thanks, >>> Vladimir >>> >>> [1] https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af >>> [2] http://hg.openjdk.java.net/metropolis/dev/ From vladimir.kozlov at oracle.com Sat Mar 30 00:38:45 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 29 Mar 2019 17:38:45 -0700 Subject: [13] RFR(L) 8220623: [JVMCI] Update JVMCI to support JVMCI based Compiler compiled into shared library In-Reply-To: <28412d92-f173-0372-a2d0-4227b6f43b46@oracle.com> References: <6f31b2e5-ab7a-cad3-d610-296e80174e01@oracle.com> <577392af-a512-bd13-aa60-46c9dccf9723@oracle.com> <28412d92-f173-0372-a2d0-4227b6f43b46@oracle.com> Message-ID: I did additional changes based on reviews and tested them with tier1-3 testing: http://cr.openjdk.java.net/~kvn/8220623/webrev_delta.04/ - For G1 moved JVMCI::do_unloading() call to ParallelCleaningTask to execute by one worker thread. - Added #if INCLUDE_JVMCI for code which is used only by JVMCI. - Used JVMCI_ONLY() macro for one line JVMCI code. - Fixed JVMCI code which count compiler threads in vmOperations.cpp as discussed. - Fixed typo in CompilerThreadStackSize setting in libgraal case. Thanks, Vladimir On 3/29/19 5:23 PM, Vladimir Kozlov wrote: > Thank you, Nils > > Yes, it looks like typo but it is the same in graal-jvmci-8 code. I asked Graal guys to make sure it > is typo which should be fixed (or not). > > Vladimir > > On 3/29/19 2:22 PM, Nils Eliasson wrote: >> I killed the formatting somehow. Second try: >> >> Hi Vladimir, >> >> I've started going through the review. This one caught my eye: >> >> compilerDefinitions.cpp: >> >> + if (UseJVMCINativeLibrary) { >> +?? // SVM compiled code requires more stack space >> +?? if (FLAG_IS_DEFAULT(CompilerThreadStackSize)) { >> +?? FLAG_SET_DEFAULT(CompilerThreadStackSize, 2*M); >> + } >> >> CompilerThreadStackSize is in Ks, so that default will turn into 2G stacks. I guess that isn't >> your intention :) >> >> Regards, >> >> Nils >> >> On 2019-03-29 22:09, Nils Eliasson wrote: >>> Hi Vladimir, I've started going through the review. This one caught my eye: *+ if >>> (UseJVMCINativeLibrary) {* >>> *+ // SVM compiled code requires more stack space* >>> *+ if (FLAG_IS_DEFAULT(CompilerThreadStackSize)) {* >>> *+ FLAG_SET_DEFAULT(CompilerThreadStackSize, 2*M);* >>> *+ } *CompilerThreadStackSize is in Ks, so that default will turn into 2G stacks. I guess that >>> isn't your intention :) Regards, Nils ** >>> >>> On 2019-03-28 20:15, Vladimir Kozlov wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8220623 >>>> http://cr.openjdk.java.net/~kvn/8220623/webrev.03/ >>>> >>>> Update JVMCI to support pre-compiled as shared library Graal. >>>> Using aoted Graal can offers benefits including: >>>> ?- fast startup >>>> ?- compile time similar to native JIt compilers (C2) >>>> ?- memory usage disjoint from the application Java heap >>>> ?- no profile pollution of JDK code used by the application >>>> >>>> This is JDK13 port of JVMCI changes done in graal-jvmci-8 [1] up to date. >>>> Changes were collected in Metropolis repo [2] and tested there. >>>> >>>> Changes we reviewed by Oracle Labs (authors of JVMCI and Graal) and our compiler group. >>>> Changes in shared code are guarded by #if INCLUDE_JVMCI and JVMCI flags. >>>> >>>> I ran tier1-tier8 (which includes HotSpot and JDK tests) and it was clean. In this set Graal was >>>> tested only in tier3. >>>> >>>> And I ran all hs-tier3-graal .. hs-tier8-graal Graal tests available in our system. Several >>>> issue were found which were present before these changes. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> [1] https://github.com/graalvm/graal-jvmci-8/commit/49ff2045fb603e35516a3a427d8023c00e1607af >>>> [2] http://hg.openjdk.java.net/metropolis/dev/ From shade at redhat.com Sun Mar 31 21:35:38 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 31 Mar 2019 23:35:38 +0200 Subject: RFR (XS) 8221725: AArch64 build failures after JDK-8221408 (Windows 32bit build build errors/warnings in hotspot) Message-ID: <593aa6a5-407d-4415-1d37-f304c4d4e6d5@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8221725 Fix: diff -r f062188117ad src/hotspot/cpu/aarch64/aarch64.ad --- a/src/hotspot/cpu/aarch64/aarch64.ad Sat Mar 30 21:29:37 2019 +0100 +++ b/src/hotspot/cpu/aarch64/aarch64.ad Sun Mar 31 23:30:41 2019 +0200 @@ -3443,11 +3443,11 @@ // Check if the owner is self by comparing the value in the // markOop of object (disp_hdr) with the stack pointer. __ mov(rscratch1, sp); __ sub(disp_hdr, disp_hdr, rscratch1); - __ mov(tmp, (address) (~(os::vm_page_size()-1) | markOopDesc::lock_mask_in_place)); + __ mov(tmp, (address) (~(os::vm_page_size()-1) | (uintptr_t)markOopDesc::lock_mask_in_place)); // If condition is true we are cont and hence we can store 0 as the // displaced header in the box, which indicates that it is a recursive lock. __ ands(tmp/*==0?*/, disp_hdr, tmp); // Sets flags for result The offending change pulled some enum constants out, which apparently made enum representation much smaller. Not sure if putting the constants back does not break Win32 build again. So, instead, fix it up in one place that blows up with build warning-as-error. The fix has some level of ewwness, but it seems to be the same as what bytecodeInterpreter.cpp is doing. The alternative is to cast to uintx, as macroAssembler_arm.cpp is doing. Testing: Linux aarch64 fastdebug build, ad-hoc tests -- Thanks, -Aleksey From david.holmes at oracle.com Sun Mar 31 22:18:24 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Apr 2019 08:18:24 +1000 Subject: RFR (XS) 8221725: AArch64 build failures after JDK-8221408 (Windows 32bit build build errors/warnings in hotspot) In-Reply-To: <593aa6a5-407d-4415-1d37-f304c4d4e6d5@redhat.com> References: <593aa6a5-407d-4415-1d37-f304c4d4e6d5@redhat.com> Message-ID: Hi Aleksey, On 1/04/2019 7:35 am, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8221725 > > Fix: > > diff -r f062188117ad src/hotspot/cpu/aarch64/aarch64.ad > --- a/src/hotspot/cpu/aarch64/aarch64.ad Sat Mar 30 21:29:37 2019 +0100 > +++ b/src/hotspot/cpu/aarch64/aarch64.ad Sun Mar 31 23:30:41 2019 +0200 > @@ -3443,11 +3443,11 @@ > // Check if the owner is self by comparing the value in the > // markOop of object (disp_hdr) with the stack pointer. > __ mov(rscratch1, sp); > __ sub(disp_hdr, disp_hdr, rscratch1); > - __ mov(tmp, (address) (~(os::vm_page_size()-1) | markOopDesc::lock_mask_in_place)); > + __ mov(tmp, (address) (~(os::vm_page_size()-1) | (uintptr_t)markOopDesc::lock_mask_in_place)); > // If condition is true we are cont and hence we can store 0 as the > // displaced header in the box, which indicates that it is a recursive lock. > __ ands(tmp/*==0?*/, disp_hdr, tmp); // Sets flags for result > > The offending change pulled some enum constants out, which apparently made enum representation much > smaller. Not sure if putting the constants back does not break Win32 build again. So, instead, fix > it up in one place that blows up with build warning-as-error. The fix has some level of ewwness, but > it seems to be the same as what bytecodeInterpreter.cpp is doing. The alternative is to cast to > uintx, as macroAssembler_arm.cpp is doing. The fix seems quite reasonable (and again trivial - but I'm not an Aarch64 expert ...) Thanks, David > Testing: Linux aarch64 fastdebug build, ad-hoc tests >