From coleen.phillimore at oracle.com Thu May 1 01:56:18 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 30 Apr 2014 21:56:18 -0400 Subject: RFR(S): 6885993: Named Thread: introduce print() and print_on(outputStream* st) methods In-Reply-To: <5360F0EA.7010104@oracle.com> References: <5360F0EA.7010104@oracle.com> Message-ID: <5361A9C2.3010503@oracle.com> This looks good. Coleen On 4/30/14, 8:47 AM, Zhengyu Gu wrote: > This is more of code cleanup. Instead of duplicating similar code all > over subclasses of NamedThread, NamedThread should implements > print_on(outputStream* str) and its subclasses should only set "name" > to use default the implementation. > > Webrev: http://cr.openjdk.java.net/~zgu/6885993/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-6885993 > > Tests: > - JPRT > - Ran a Java program on Linux x64, do "CTRL-\" to dump Java stacks, > make sure the output remains unchange. > > Thanks, > > -Zhengyu From david.holmes at oracle.com Thu May 1 03:59:10 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 01 May 2014 13:59:10 +1000 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> Message-ID: <5361C68E.9040002@oracle.com> On 30/04/2014 9:59 PM, David Chase wrote: > > On 2014-04-30, at 5:09 AM, David Holmes wrote: >>> How do you like it otherwise? I'm itching to get reviews and get this thing out of my hair. >> >> I appreciate what you are doing but I dislike the details - in particular print_raw and, even worse, cr(). >> >> One thing we do need from this is some clear documentation somewhere on what format specific macros to use for which types. > > You do understand that those are both existing public methods, and I did not add or innovate, right? I didn't blame you for their existence, I just dislike the proliferation of their use. I was not aware of either of them before this thread, and have no recollection of encountering them. > It's also completely nonintuitive to me that you would dislike cr more than print_raw. s->print_raw(...) at least is obviously printing s->cr() is what? print_cr at least puts the cr in the context of printing. > Rather than document, we could make them not be public, if we really meant for them not to be called. > I prefer Vladimir's proposal to rename print_raw into puts. I would not be averse to that. :) > One thing that is a little puzzling to me is why we have both INTX and INT64. > I see no purpose whatsoever for INTX; it appears to be a 64-bit integer type, > but we have a well-defined 64-bit integer type (INT64) and a not-well-defined > 64-bit integer type (JLONG) that we own because it is Ours. We also have a > pointer-sized integer type (INTPTR) and if this is actually a distinction, > an "I intend to print this integer as a pointer type" (PTR). As Vladimir points out intx is not a 64-bit integer type, but an intptr_t that varies in size. Why we needed that instead of just using intptr_t I have no idea. David H. > David > From david.holmes at oracle.com Thu May 1 06:25:05 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 01 May 2014 16:25:05 +1000 Subject: RFR(S): 6885993: Named Thread: introduce print() and print_on(outputStream* st) methods In-Reply-To: <5360F0EA.7010104@oracle.com> References: <5360F0EA.7010104@oracle.com> Message-ID: <5361E8C1.2040000@oracle.com> Hi Zhengyu, On 30/04/2014 10:47 PM, Zhengyu Gu wrote: > This is more of code cleanup. Instead of duplicating similar code all > over subclasses of NamedThread, NamedThread should implements > print_on(outputStream* str) and its subclasses should only set "name" to > use default the implementation. > > Webrev: http://cr.openjdk.java.net/~zgu/6885993/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-6885993 I agree with Christian that print_on should probably not be inlined in thread.hpp > Tests: > - JPRT > - Ran a Java program on Linux x64, do "CTRL-\" to dump Java stacks, > make sure the output remains unchange. Note that none of the above tests printing for the GC threads. Of course if it works for one NamedThread then it should work for them all :) The actual Java threads are unaffected by this change. Thanks, David > Thanks, > > -Zhengyu From zhengyu.gu at oracle.com Thu May 1 11:14:26 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Thu, 01 May 2014 07:14:26 -0400 Subject: RFR(S): 6885993: Named Thread: introduce print() and print_on(outputStream* st) methods In-Reply-To: <5361A9C2.3010503@oracle.com> References: <5360F0EA.7010104@oracle.com> <5361A9C2.3010503@oracle.com> Message-ID: <53622C92.9070601@oracle.com> Thanks for the review. -Zhengyu On 4/30/2014 9:56 PM, Coleen Phillimore wrote: > > This looks good. > Coleen > > On 4/30/14, 8:47 AM, Zhengyu Gu wrote: >> This is more of code cleanup. Instead of duplicating similar code all >> over subclasses of NamedThread, NamedThread should implements >> print_on(outputStream* str) and its subclasses should only set "name" >> to use default the implementation. >> >> Webrev: http://cr.openjdk.java.net/~zgu/6885993/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-6885993 >> >> Tests: >> - JPRT >> - Ran a Java program on Linux x64, do "CTRL-\" to dump Java stacks, >> make sure the output remains unchange. >> >> Thanks, >> >> -Zhengyu > From zhengyu.gu at oracle.com Thu May 1 11:57:46 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Thu, 01 May 2014 07:57:46 -0400 Subject: RFR(S): 6885993: Named Thread: introduce print() and print_on(outputStream* st) methods In-Reply-To: <87A5C4F6-A5A7-426E-9E90-C8BC06575360@oracle.com> References: <5360F0EA.7010104@oracle.com> <87A5C4F6-A5A7-426E-9E90-C8BC06575360@oracle.com> Message-ID: <536236BA.606@oracle.com> Thanks for the review. On 4/30/2014 6:58 PM, Christian Thalinger wrote: > I?d prefer to have the method body of: > *+ virtual void print_on(outputStream* st) const {* > in the .cpp file. Will move to .cpp file. -Zhengyu > > Otherwise this is very nice and looks good. > > On Apr 30, 2014, at 2:47 AM, Zhengyu Gu > wrote: > >> This is more of code cleanup. Instead of duplicating similar code all >> over subclasses of NamedThread, NamedThread should implements >> print_on(outputStream* str) and its subclasses should only set "name" >> to use default the implementation. >> >> Webrev: http://cr.openjdk.java.net/~zgu/6885993/webrev.00/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-6885993 >> >> Tests: >> - JPRT >> - Ran a Java program on Linux x64, do "CTRL-\" to dump Java stacks, >> make sure the output remains unchange. >> >> Thanks, >> >> -Zhengyu > From zhengyu.gu at oracle.com Thu May 1 12:00:51 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Thu, 01 May 2014 08:00:51 -0400 Subject: RFR(S): 6885993: Named Thread: introduce print() and print_on(outputStream* st) methods In-Reply-To: <5361E8C1.2040000@oracle.com> References: <5360F0EA.7010104@oracle.com> <5361E8C1.2040000@oracle.com> Message-ID: <53623773.6060007@oracle.com> Thanks for the review. -Zhengyu On 5/1/2014 2:25 AM, David Holmes wrote: > Hi Zhengyu, > > On 30/04/2014 10:47 PM, Zhengyu Gu wrote: >> This is more of code cleanup. Instead of duplicating similar code all >> over subclasses of NamedThread, NamedThread should implements >> print_on(outputStream* str) and its subclasses should only set "name" to >> use default the implementation. >> >> Webrev: http://cr.openjdk.java.net/~zgu/6885993/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-6885993 > > I agree with Christian that print_on should probably not be inlined in > thread.hpp > >> Tests: >> - JPRT >> - Ran a Java program on Linux x64, do "CTRL-\" to dump Java stacks, >> make sure the output remains unchange. > > Note that none of the above tests printing for the GC threads. Of > course if it works for one NamedThread then it should work for them > all :) > > The actual Java threads are unaffected by this change. > > Thanks, > David > >> Thanks, >> >> -Zhengyu From lois.foltan at oracle.com Thu May 1 14:26:13 2014 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 01 May 2014 10:26:13 -0400 Subject: RFR(S): 8039805: Fix the signature of the global new/delete operators in allocation.cpp In-Reply-To: <5360C95A.3050309@oracle.com> References: <5346066A.3000401@oracle.com> <53460DBA.4010708@oracle.com> <53474C4B.9000006@oracle.com> <5347C745.2040901@oracle.com> <5347CED1.3000908@oracle.com> <535FB05B.6030605@oracle.com> <5360C95A.3050309@oracle.com> Message-ID: <53625985.1090909@oracle.com> On 4/30/2014 5:58 AM, David Holmes wrote: > On 30/04/2014 3:24 AM, Volker Simonis wrote: >> Hi Lois, >> >> sure, the latest webrev was: >> >> http://cr.openjdk.java.net/~simonis/webrevs/8039805.v3/ >> >> As far as I understood, David recalled his concerns in his last mail >> ("..Ignore that. I just realized it is not the external linkage to >> these methods that is the main issue, but some internal hotspot code >> calling the global operator new/delete.." - see >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013523.html) >> >> >> @David: are you OK with the change now? > > Yes. To clarify for Lois my "Ignore that" retracted my comment that > everything in the "#ifndef ALLOW_OPERATOR_NEW_USAGE" block could be > deleted. I'm okay with getting rid of ALLOW_OPERATOR_NEW_USAGE itself. Thank you David for the clarification, just wanted to make sure all review comments were covered! Lois > > Thanks, > David > > >> Thanks, >> Volker >> >> PS: jdk9 is just fine for now - thanks for helping out! >> >> >> On Tue, Apr 29, 2014 at 3:59 PM, Lois Foltan >> wrote: >>> >>> On 4/29/2014 9:43 AM, Volker Simonis wrote: >>>> >>>> Hi everybody, >>>> >>>> I'm still looking for a sponsor for this change. >>> >>> >>> Hi Volker, >>> >>> Can I ask a favor, could you send out your latest webrev. Maybe my >>> misunderstanding, but I thought this was left off at David's last >>> comment >>> that ALLOW_OPERATOR_NEW should not be removed after all which you had >>> removed in your last webrev. >>> >>> I can sponsor you for JDK 9 but am not a committer for JDK 8 so >>> would not be >>> able to backport the change. >>> >>> Thanks, >>> Lois >>> >>> >>>> >>>> Thanks, >>>> Volker >>>> >>>> On Fri, Apr 11, 2014 at 1:15 PM, David Holmes >>>> >>>> wrote: >>>>> >>>>> On 11/04/2014 8:43 PM, David Holmes wrote: >>>>>> >>>>>> On 11/04/2014 8:34 PM, Volker Simonis wrote: >>>>>>> >>>>>>> On Fri, Apr 11, 2014 at 3:58 AM, David Holmes >>>>>>> >>>>>> > wrote: >>>>>>> >>>>>>> On 11/04/2014 4:33 AM, Volker Simonis wrote: >>>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> thanks for looking at this issue. >>>>>>> >>>>>>> I agree with you and I've completely removed >>>>>>> ALLOW_OPERATOR_NEW now: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~__simonis/webrevs/8039805.v2/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> I actually meant delete the whole block guarded by >>>>>>> ALLOW_OPERATOR_NEW - we don't need these error-trapping >>>>>>> definitions >>>>>>> now we have fixed the export problem. >>>>>>> >>>>>>> >>>>>>> OK, but arguing this way, we could remove remove all asserts >>>>>>> from the >>>>>>> code, once we fixed an error revealed by them. >>>>>> >>>>>> >>>>>> Obviously you have to draw a line somewhere. But I don't think these >>>>>> particular methods are worth the problem they are now causing. >>>>> >>>>> >>>>> Ignore that. I just realized it is not the external linkage to these >>>>> methods >>>>> that is the main issue, but some internal hotspot code calling the >>>>> global >>>>> operator new/delete. >>>>> >>>>> David >>>>> >>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>>> I think the error-trapping new/delete operators are still >>>>>>> valuable in >>>>>>> protecting people from accidentally calling them (and they don't >>>>>>> really >>>>>>> hurt anybody in terms of performance or space). >>>>>>> >>>>>>> Regards, >>>>>>> Volker >>>>>>> >>>>>>> FYI I'm not in the runtime group. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> >>>>>>> I've also changed the "guarantee" into "fatal" as >>>>>>> proposed by >>>>>>> Vladimir. >>>>>>> >>>>>>> I've thought a little while about the problem that >>>>>>> some other >>>>>>> code may >>>>>>> unintentionally use these operators but I couldn?t >>>>>>> really find >>>>>>> a >>>>>>> scenario where this could happen. Because as you >>>>>>> correctly >>>>>>> pointed >>>>>>> out, these operators aren't exported from libjvm.so - >>>>>>> after >>>>>>> all, >>>>>>> that's the whole reason for compiling with >>>>>>> visibility=hidden >>>>>>> and >>>>>>> using >>>>>>> of export maps. So if another program/library will load >>>>>>> libjvm.so, the >>>>>>> operators won't be visible. On the other hand, if the >>>>>>> libjvm.so >>>>>>> loads >>>>>>> another shared libraries which use these operators >>>>>>> they either >>>>>>> have >>>>>>> their own, private versions of them or they are >>>>>>> dynamically >>>>>>> linked >>>>>>> against another library (most probably the standard >>>>>>> library) >>>>>>> which >>>>>>> provides versions of the operators. >>>>>>> >>>>>>> So if I'm not totally wrong, we could in principle >>>>>>> also enable >>>>>>> these >>>>>>> operators in the product build. However, I'm not >>>>>>> proposing >>>>>>> that for >>>>>>> this change. Let's first fix the signatures and get >>>>>>> rid of >>>>>>> ALLOW_OPERATOR_NEW with this change. If everything >>>>>>> works fine, >>>>>>> we can >>>>>>> think about enabling these global operators in product >>>>>>> builds >>>>>>> as >>>>>>> well. >>>>>>> >>>>>>> By the way - are you from the runtime group? >>>>>>> I was asked to get a review from a runtime-group member - >>>>>>> anybody out >>>>>>> there willing to volunteer? >>>>>>> >>>>>>> Thank you and best regards, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>> On Thu, Apr 10, 2014 at 5:19 AM, David Holmes >>>>>>> >>>>>> > >>>>>>> wrote: >>>>>>> >>>>>>> I think we should just delete the whole >>>>>>> ALLOW_OPERATOR_NEW >>>>>>> block. We fixed >>>>>>> the problem of it being called outside the VM under >>>>>>> 8014326. >>>>>>> >>>>>>> David >>>>>>> >>>>>>> >>>>>>> On 10/04/2014 12:48 PM, David Holmes wrote: >>>>>>> >>>>>>> >>>>>>> Hi Volker, >>>>>>> >>>>>>> Need more time to consider this in full but >>>>>>> from the >>>>>>> existing code: >>>>>>> >>>>>>> 689 // On certain platforms, such as Mac OS X >>>>>>> (Darwin), in debug >>>>>>> version, new is being called >>>>>>> 690 // from jdk source and causing data >>>>>>> corruption. >>>>>>> Such as >>>>>>> 691 // >>>>>>> >>>>>>> Java_sun_security_ec___ECKeyPairGenerator___generateECKeyPair >>>>>>> 692 // define ALLOW_OPERATOR_NEW_USAGE for >>>>>>> platform >>>>>>> on which global >>>>>>> operator new allowed. >>>>>>> >>>>>>> we actually fixed that by using the mapfiles >>>>>>> to ensure >>>>>>> the hotspot >>>>>>> operator new was not visible externally. The >>>>>>> existence >>>>>>> of >>>>>>> ALLOW_OPERATOR_NEW_USAGE wasn't even raised at >>>>>>> the >>>>>>> time :( >>>>>>> >>>>>>> https://bugs.openjdk.java.net/__browse/JDK-8014326 >>>>>>> >>>>>>> >>>>>>> David >>>>>>> >>>>>>> On 10/04/2014 2:34 AM, Volker Simonis wrote: >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> could you please review and sponsor the >>>>>>> following >>>>>>> small change: >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~__simonis/webrevs/8039805/ >>>>>>> >>>>>>> >>>>>>> https://bugs.openjdk.java.net/__browse/JDK-8039805 >>>>>>> >>>>>>> >>>>>>> which fixes the signature of the global >>>>>>> new/delete >>>>>>> operators in >>>>>>> allocation.cpp >>>>>>> >>>>>>> For non-product builds allocation.cpp defines >>>>>>> global >>>>>>> new/delete >>>>>>> operators which shut down the VM if they get >>>>>>> called >>>>>>> at runtime. The >>>>>>> rational behind this is that the these global >>>>>>> operators should never >>>>>>> be used in HotSpot. >>>>>>> >>>>>>> Unfortunately, the signature of some of these >>>>>>> operators doesn't >>>>>>> conform to the C++ standard which confuses >>>>>>> some >>>>>>> C++ >>>>>>> compilers. For a >>>>>>> more detailed explanation of the C++ >>>>>>> background of >>>>>>> this issue see the >>>>>>> new comments in allcoation.cpp and the end >>>>>>> of this >>>>>>> mail. >>>>>>> >>>>>>> This change also replaces the asserts in the >>>>>>> operators with guarantees >>>>>>> because the code may also be active in >>>>>>> not-product >>>>>>> (aka. 'optimized') >>>>>>> builds. >>>>>>> >>>>>>> Finally, the webrev fixes two places in the >>>>>>> AIX-port >>>>>>> which used the >>>>>>> global new operator. After the change we >>>>>>> can now >>>>>>> remove the definition >>>>>>> of ALLOW_OPERATOR_NEW_USAGE from >>>>>>> aix/makefiles/vm.make. >>>>>>> >>>>>>> I have also removed >>>>>>> ALLOW_OPERATOR_NEW_USAGE from >>>>>>> bsd/makefiles/vm.make and the corresponding >>>>>>> comments >>>>>>> in allcoation.cpp >>>>>>> which state that on Mac OS X the global >>>>>>> new/delete >>>>>>> operators from the >>>>>>> HotSpot cause problems together with >>>>>>> usages of >>>>>>> these >>>>>>> operators from >>>>>>> the class library such as the ones from >>>>>>> >>>>>>> Java_sun_security_ec___ECKeyPairGenerator___generateECKeyPair. >>>>>>> I couldn?t >>>>>>> observe any such problems but if anybody >>>>>>> has some >>>>>>> concrete concerns >>>>>>> I'm ready to remove this part from the >>>>>>> webrev. >>>>>>> >>>>>>> I've build and tested these changes on >>>>>>> Linux/x86_64, >>>>>>> Linux/ppc64, >>>>>>> Solaris/Sparc, Windows/x86_64, MacOS X and >>>>>>> AIX/ppc64. I've especially >>>>>>> run the regression tests from sun/security/ec >>>>>>> which >>>>>>> exercise the >>>>>>> method >>>>>>> >>>>>>> Java_sun_security_ec___ECKeyPairGenerator___generateECKeyPair >>>>>>> which >>>>>>> was mentioned to cause problems in >>>>>>> conjunction >>>>>>> with >>>>>>> the globally >>>>>>> defined new/delete operators from the >>>>>>> HotSpot but >>>>>>> couldn't see any >>>>>>> issues, neither in the slowdebug nor in the >>>>>>> optimized build. >>>>>>> >>>>>>> Following some C++ background regarding >>>>>>> the global >>>>>>> new/delete operators: >>>>>>> >>>>>>> In C++98/03 the throwing new operators are >>>>>>> defined >>>>>>> with the following >>>>>>> signature: >>>>>>> >>>>>>> void* operator new(std::size_tsize) >>>>>>> throw(std::bad_alloc); >>>>>>> void* operator new[](std::size_tsize) >>>>>>> throw(std::bad_alloc); >>>>>>> >>>>>>> while all the other (non-throwing) new and >>>>>>> delete >>>>>>> operators are >>>>>>> defined with an empty throw clause (i.e. >>>>>>> "operator >>>>>>> delete(void* p) >>>>>>> throw()") which means that they do not >>>>>>> throw any >>>>>>> exceptions (see >>>>>>> section 18.4 of the C++ standard >>>>>>> >>>>>>> http://www.open-std.org/jtc1/__sc22/wg21/docs/papers/2005/__n1905.pdf >>>>>>> >>>>>>> >>>>>>> ). >>>>>>> >>>>>>> >>>>>>> In the new C++11/14 standard >>>>>>> >>>>>>> (http://www.open-std.org/jtc1/__sc22/wg21/docs/papers/2013/__n3797.pdf >>>>>>> >>>>>>> >>>>>>> ), >>>>>>> >>>>>>> the signature of the throwing new >>>>>>> operators was >>>>>>> changed by completely >>>>>>> omitting the throw clause (which >>>>>>> effectively means >>>>>>> they could throw >>>>>>> any exception) while all the other new/delete >>>>>>> operators where changed >>>>>>> to have a 'nothrow' clause instead of an >>>>>>> empty >>>>>>> throw >>>>>>> clause. >>>>>>> >>>>>>> Unfortunately, the support for exception >>>>>>> specifications among C++ >>>>>>> compilers is still very fragile. While >>>>>>> some more >>>>>>> strict compilers like >>>>>>> AIX xlC or HP aCC reject to override the >>>>>>> default >>>>>>> throwing new operator >>>>>>> with a user operator with an empty throw() >>>>>>> clause, >>>>>>> the MS Visual C++ >>>>>>> compiler warns for every non-empty throw >>>>>>> clause >>>>>>> like >>>>>>> throw(std::bad_alloc) that it will ignore the >>>>>>> exception specification >>>>>>> (see >>>>>>> >>>>>>> http://msdn.microsoft.com/en-__us/library/sa28fef8.aspx >>>>>>> >>>>>>> ). >>>>>>> >>>>>>> I've therefore changed the operator >>>>>>> definitions >>>>>>> such >>>>>>> that they >>>>>>> correctly work with all currently supported >>>>>>> compilers and in way which >>>>>>> should be upwards compatible with C++11/14. >>>>>>> >>>>>>> Please notice that I'm aware of the >>>>>>> discussion >>>>>>> around "8021954: VM >>>>>>> SIGSEGV during classloading on MacOS; >>>>>>> hs_err_pid >>>>>>> file produced" >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> (http://mail.openjdk.java.net/__pipermail/hotspot-runtime-dev/__2013-August/thread.html#8924 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> , >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://hg.openjdk.java.net/__hsx/hotspot-comp/hotspot/rev/__9758d9f36299 >>>>>>> >>>>>>> >>>>>>> ) >>>>>>> >>>>>>> which introduced empty throw() clauses on >>>>>>> all user >>>>>>> defined new >>>>>>> operators. But I think the rational used >>>>>>> for that >>>>>>> change doesn't apply >>>>>>> here, because these global, user user >>>>>>> defined new >>>>>>> operators changed in >>>>>>> this webrev aren't meant to be really >>>>>>> used. There >>>>>>> only task is to >>>>>>> override the default, global operators (and >>>>>>> therefore they need to >>>>>>> have the right signature) and to shut the >>>>>>> VM down >>>>>>> if >>>>>>> they get called. >>>>>>> >>>>>>> Thank you and best regards, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>> >>> From erik.helin at oracle.com Thu May 1 16:14:58 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 01 May 2014 18:14:58 +0200 Subject: RFR: 8034094: SA agent can't compile when jni_x86.h is used In-Reply-To: <5360E0D5.90203@oracle.com> References: <52F8EB7A.4080305@oracle.com> <5360DDC8.2000003@oracle.com> <5360E0D5.90203@oracle.com> Message-ID: <3005492.Fkng45DX1Y@ehelin-laptop> Hi Erik, thanks for having a look at the patches! On Wednesday 30 April 2014 13:39:01 PM Erik Joelsson wrote: > I think option 2 seems best. Ok, that is my preferred option as well, thanks! Erik > /Erik > > On 2014-04-30 13:26, Erik Helin wrote: > > Anyone interested in this patch? I ran into this issue again yesterday... > > > > Thanks, > > Erik > > > > On 2014-02-10 16:21, Erik Helin wrote: > >> Sigh, I forgot the subject... > >> > >> "RFR: 8034094: SA agent can't compile when jni_x86.h is used" > >> > >> Thanks, > >> Erik > >> > >> On 2014-02-10 16:08, Erik Helin wrote: > >>> Hi all, > >>> > >>> this patch fixes an issue with HotSpot's makefiles, IMPORT_JDK and > >>> jni_md.h. > >>> > >>> The bug manifests itself when using an IMPORT_JDK which > >>> include/linux/jni_md.h has a timestamp that is older than > >>> hotspot/src/cpu/x86/jni_x86.h. When this happens, the Makefiles will > >>> copy hotspot/src/cpu/x86/jni_x86.h to > >>> hotspot/build/jdk-linux-amd64/fastdebug/include/linux/jni_md.h. > >>> > >>> The issue is that hotspot/src/cpu/x86/jni_x86.h differs slightly from > >>> jdk/include/jni.h, since it is used for all operating systems: > >>> > >>> #if defined(SOLARIS) || defined(LINUX) || defined(_ALLBSD_SOURCE) > >>> ... // common stuff > >>> #else > >>> ... // windows stuff > >>> #endif > >>> > >>> We compile the SA agent, see make/linux/makefiles/saproc.make, without > >>> defining LINUX (LINUX is hotspot's own define, gcc uses __linux__). > >>> > >>> In my opinion, there are two ways to solve this: > >>> 1. Add -DLINUX to make/linux/makefiles/saproc.make (and corresponding > >>> > >>> defines for Solaris and BSD). > >>> > >>> 2. Rewrite the #if check in jni_x86.h to use platform specific "native" > >>> > >>> defines. > >>> > >>> I've created a patch for each alternative: > >>> 1: http://cr.openjdk.java.net/~ehelin/8034094/webrev.1/ > >>> 2: http://cr.openjdk.java.net/~ehelin/8034094/webrev.2/ > >>> > >>> For the second patch, note that I've inverted the #if check so that it > >>> checks for _WIN32 is defined and treat all others operating systems as > >>> "#else". > >>> > >>> Bug: > >>> https://bugs.openjdk.java.net/browse/JDK-8034094 > >>> > >>> Testing: > >>> - Compiled both version locally and made sure it worked > >>> - JPRT > >>> > >>> Thanks, > >>> Erik From erik.helin at oracle.com Thu May 1 16:16:31 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 01 May 2014 18:16:31 +0200 Subject: RFR: 8034094: SA agent can't compile when jni_x86.h is used In-Reply-To: <53614C90.2080701@oracle.com> References: <52F8EB7A.4080305@oracle.com> <52F8EE89.6040808@oracle.com> <53614C90.2080701@oracle.com> Message-ID: <1542127.onZt1XuZ2J@ehelin-laptop> On Wednesday 30 April 2014 23:18:40 PM Dmitry Samersoff wrote: > Erik, > > Sorry, missed the thread. No problem, thanks for having a look! On Wednesday 30 April 2014 23:18:40 PM Dmitry Samersoff wrote: > Changes (option 2) looks good for me. Thanks! Erik On Wednesday 30 April 2014 23:18:40 PM Dmitry Samersoff wrote: > -Dmitry > > On 2014-02-10 19:21, Erik Helin wrote: > > Sigh, I forgot the subject... > > > > "RFR: 8034094: SA agent can't compile when jni_x86.h is used" > > > > Thanks, > > Erik > > > > On 2014-02-10 16:08, Erik Helin wrote: > >> Hi all, > >> > >> this patch fixes an issue with HotSpot's makefiles, IMPORT_JDK and > >> jni_md.h. > >> > >> The bug manifests itself when using an IMPORT_JDK which > >> include/linux/jni_md.h has a timestamp that is older than > >> hotspot/src/cpu/x86/jni_x86.h. When this happens, the Makefiles will > >> copy hotspot/src/cpu/x86/jni_x86.h to > >> hotspot/build/jdk-linux-amd64/fastdebug/include/linux/jni_md.h. > >> > >> The issue is that hotspot/src/cpu/x86/jni_x86.h differs slightly from > >> jdk/include/jni.h, since it is used for all operating systems: > >> > >> #if defined(SOLARIS) || defined(LINUX) || defined(_ALLBSD_SOURCE) > >> ... // common stuff > >> #else > >> ... // windows stuff > >> #endif > >> > >> We compile the SA agent, see make/linux/makefiles/saproc.make, without > >> defining LINUX (LINUX is hotspot's own define, gcc uses __linux__). > >> > >> In my opinion, there are two ways to solve this: > >> 1. Add -DLINUX to make/linux/makefiles/saproc.make (and corresponding > >> > >> defines for Solaris and BSD). > >> > >> 2. Rewrite the #if check in jni_x86.h to use platform specific "native" > >> > >> defines. > >> > >> I've created a patch for each alternative: > >> 1: http://cr.openjdk.java.net/~ehelin/8034094/webrev.1/ > >> 2: http://cr.openjdk.java.net/~ehelin/8034094/webrev.2/ > >> > >> For the second patch, note that I've inverted the #if check so that it > >> checks for _WIN32 is defined and treat all others operating systems as > >> "#else". > >> > >> Bug: > >> https://bugs.openjdk.java.net/browse/JDK-8034094 > >> > >> Testing: > >> - Compiled both version locally and made sure it worked > >> - JPRT > >> > >> Thanks, > >> Erik From jon.masamitsu at oracle.com Thu May 1 17:34:54 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 01 May 2014 10:34:54 -0700 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <5360DBF1.3050807@oracle.com> References: <5360DBF1.3050807@oracle.com> Message-ID: <536285BE.2060303@oracle.com> Erik, Change looks good. Please add a comment explaining why we're using committed now and why capacity did not work. Basically an abbreviated version of your explanation below. Thanks. Jon On 4/30/2014 4:18 AM, Erik Helin wrote: > Hi all, > > this patch solves a rather tricky problem with the sizing of Metaspace. > > The issue happens when the GC threshold for Metaspace (called > "capacity_until_GC" in the code) becomes less than the committed > memory for Metaspace. Any calls to Metaspace::allocate that requires > committing more memory will then fail in > MetaspaceGC::allowed_expansion, because capacity_until_GC() < > MetaspaceAux::committed_memory(). The effect will be a > full GC and after the GC we try to expand and allocate. After the > expansion and before the allocation, one of two things can happen: > 1. capacity_until_GC is larger than the committed memory after the > expansion. The allocation will now succeed, but the next allocation > requiring a new chunk will *again* trigger a full GC. This pattern > will repeat itself for each new allocation request requiring a new > chunk. > 2. capacity_until_GC is still less than the committed memory even > after the expansion. We throw a Java OOME (incorrectly). > > How can the GC threshold for Metaspace be less than the committed > memory? The problem is that MetaspaceGC::compute_new_size uses the field > _allocated_capacity for describing the amount of memory in Metaspace > that is "in use". _allocated_capacity does not consider the memory in > the chunk free lists to be "in use", since memory in the chunk free > lists are supposed to be available for new allocations. The problem is > that the chunk free lists can become fragmented, and then the memory > is not available for all kinds of allocations. > > This patch change MetaspaceGC::compute_new_size to use > MetaspaceAux::committed_memory for describing how much memory that is > "in use". The effect will be that memory in the chunk free lists will no > longer be considered "in use" (but will of course be used for future > allocations where possible). This will prevent capacity_until_GC from > shrinking below the committed memory "by definiton", since > capacity_until_GC can't be lower than the memory that is "in use". > > Based on the results from the perf testing (see below), this change > has no performance impact. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ > > Testing: > - JPRT > - Ad-hoc testing: > - Kitchensink > - Dacapo > - Medrec > - runThese > - Parallel Class Loading testlist > - Metaspace testlist > - GC nightly testlist > - Perf testing: > - SPECjbb2005 > - SPECjbb2013 > - Derby > - Derby regression tests > > Thanks, > Erik From goetz.lindenmaier at sap.com Fri May 2 12:23:41 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 2 May 2014 12:23:41 +0000 Subject: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. Message-ID: <4295855A5C1DE049A61835A1887419CC2CEBEA0E@DEWDFEMB12A.global.corp.sap> Hi, Please review and test this change. I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8042195 This change introduces an umbrella header for orderAccess__.inline.hpp files: runtime/orderAccess.inline.hpp. This follows the scheme applied to atomic.inline.hpp. Further this change adds includes of orderAccess.inline.hpp in all .cpp and .inline.hpp files where a method of OrderAccess declared 'inline' is called. Finally it moves methods calling inline methods of OrderAccess from g1CollectedHeap.hpp and thread.hpp to the corresponding .inline.hpp files and adds the necessary includes in files using the moved functions. Still a lot of calls to inline methods of OrderAccess in .hpp files arn't preceded by an according include. But for these files an appropriate .inline.hpp file is missing: src/cpu/ppc/vm/javaFrameAnchor_ppc.hpp src/cpu/sparc/vm/javaFrameAnchor_sparc.hpp src/share/vm/classfile/classLoader.hpp src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp src/share/vm/memory/cardTableModRefBS.hpp src/share/vm/oops/constantPool.hpp src/share/vm/oops/cpCache.hpp src/share/vm/oops/instanceKlass.hpp src/share/vm/oops/method.hpp src/share/vm/oops/methodData.hpp src/share/vm/oops/typeArrayOop.hpp src/share/vm/runtime/interfaceSupport.hpp src/share/vm/utilities/array.hpp src/share/vm/utilities/taskqueue.hpp Testing: I ran builds on linuxx86_64, sun_32, sun_64, mac, windowsx86_64, linuxppc64 and aix; dbg, fastdbg and product without precompiled headers. Thanks and best regards, Goetz From vladimir.kozlov at oracle.com Fri May 2 19:36:08 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 02 May 2014 12:36:08 -0700 Subject: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEBEA0E@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEBEA0E@DEWDFEMB12A.global.corp.sap> Message-ID: <5363F3A8.5040208@oracle.com> Hi Goetz, In general it is good but sponsor from Oracle have to check and fix our closed sources too. I see in metaspace.cpp, precompiled.hpp you kept orderAccess.hpp include which is not needed: # include "runtime/orderAccess.hpp" +# include "runtime/orderAccess.inline.hpp" Typo in oops/klass.hpp?: - // Ordered list of all primary supertypes + // Orderd list of all primary supertypes In javaFrameAnchor.hpp preserve original order - globalDefinitions.hpp first. Thanks, Vladimir On 5/2/14 5:23 AM, Lindenmaier, Goetz wrote: > Hi, > > Please review and test this change. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8042195 > > This change introduces an umbrella header for orderAccess__.inline.hpp > files: runtime/orderAccess.inline.hpp. This follows the scheme applied to > atomic.inline.hpp. > > Further this change adds includes of orderAccess.inline.hpp in all .cpp > and .inline.hpp files where a method of OrderAccess declared 'inline' is > called. > > Finally it moves methods calling inline methods of OrderAccess from > g1CollectedHeap.hpp and thread.hpp to the corresponding .inline.hpp > files and adds the necessary includes in files using the moved > functions. > > Still a lot of calls to inline methods of OrderAccess in .hpp > files arn't preceded by an according include. But for these > files an appropriate .inline.hpp file is missing: > > src/cpu/ppc/vm/javaFrameAnchor_ppc.hpp > src/cpu/sparc/vm/javaFrameAnchor_sparc.hpp > src/share/vm/classfile/classLoader.hpp > src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp > src/share/vm/memory/cardTableModRefBS.hpp > src/share/vm/oops/constantPool.hpp > src/share/vm/oops/cpCache.hpp > src/share/vm/oops/instanceKlass.hpp > src/share/vm/oops/method.hpp > src/share/vm/oops/methodData.hpp > src/share/vm/oops/typeArrayOop.hpp > src/share/vm/runtime/interfaceSupport.hpp > src/share/vm/utilities/array.hpp > src/share/vm/utilities/taskqueue.hpp > > Testing: > I ran builds on linuxx86_64, sun_32, sun_64, mac, windowsx86_64, linuxppc64 and aix; > dbg, fastdbg and product without precompiled headers. > > Thanks and best regards, > Goetz > From goetz.lindenmaier at sap.com Fri May 2 21:28:53 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 2 May 2014 21:28:53 +0000 Subject: Introduce umbrella header orderAccess.inline.hpp In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC2CEB3B34@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEBECAA@DEWDFEMB12A.global.corp.sap> Thanks Chris! Yes, I can do prefetch, too. After this one. Best regards, Goetz. -----Original Message----- From: Christian Thalinger [mailto:christian.thalinger at oracle.com] Sent: Tuesday, April 29, 2014 9:44 PM To: Lindenmaier, Goetz Cc: hotspot-dev at openjdk.java.net Subject: Re: Introduce umbrella header orderAccess.inline.hpp Yes, please. Are you going to do the same for prefetch_*.inline.hpp? ;-) On Apr 29, 2014, at 4:08 AM, Lindenmaier, Goetz wrote: > Hi, > > I would like to contribute a change that introduces an umbrella header > orderAccess.inline.hpp to avoid the long #ifdef TARGET_OS_ARCH cascades. > There aren't too much of these, but that's because most files that > require that include automagically include the files with the > implementation. > With the umbrella header the includes can be listed explicitly without > having to add the #ifdef platform cascades everywhere. > > This follows the scheme that was applied to atomic.inline.hpp. > > Please tell me whether this change is acceptable, then I will open > an appropriate bug and send a RFR. The webrev below isn't tested > sufficiently yet. > > Preliminary webrev: > http://cr.openjdk.java.net/~goetz/webrevs/orderAcc_proposal/ > > The change would introduce the new file runtime/orderAccess.inline.hpp. > > Further this change adds includes of orderAccess.inline.hpp in all .cpp > and .inline.hpp files where a method of OrderAccess declared 'inline' is > called. > > Finally it moves methods calling inline methods of OrderAccess from > g1CollectedHeap.hpp and thread.hpp to the corresponding .inline.hpp > files and adds the necessary includes in files using the moved > functions. > > Still a lot of calls to inline methods of OrderAccess in .hpp > files aren't preceded by an according include. But for these > files an appropriate .inline.hpp file is missing: > > src/cpu/ppc/vm/javaFrameAnchor_ppc.hpp > src/cpu/sparc/vm/javaFrameAnchor_sparc.hpp > src/share/vm/classfile/classLoader.hpp > src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp > src/share/vm/memory/cardTableModRefBS.hpp > src/share/vm/oops/constantPool.hpp > src/share/vm/oops/cpCache.hpp > src/share/vm/oops/instanceKlass.hpp > src/share/vm/oops/method.hpp > src/share/vm/oops/methodData.hpp > src/share/vm/oops/typeArrayOop.hpp > src/share/vm/runtime/interfaceSupport.hpp > src/share/vm/utilities/array.hpp > src/share/vm/utilities/taskqueue.hpp > > Best regards, > Goetz. From goetz.lindenmaier at sap.com Fri May 2 21:42:04 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 2 May 2014 21:42:04 +0000 Subject: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. Message-ID: <4295855A5C1DE049A61835A1887419CC2CEBECE6@DEWDFEMB12A.global.corp.sap> Hi Vladimir, I fixed what you proposed except for precompiled.hpp: There always the .hpp and .inline.hpp are mentioned, so I'd like to do it for this file the same way. Webrev is updated: http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ No problem with closed sources, there is no hurry with this one. Thanks and best regards, Goetz Vladimir, sorry you get this twice, I forgot the mailing list first. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Kozlov Sent: Friday, May 02, 2014 9:36 PM To: hotspot-dev at openjdk.java.net Subject: Re: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. Hi Goetz, In general it is good but sponsor from Oracle have to check and fix our closed sources too. I see in metaspace.cpp, precompiled.hpp you kept orderAccess.hpp include which is not needed: # include "runtime/orderAccess.hpp" +# include "runtime/orderAccess.inline.hpp" Typo in oops/klass.hpp?: - // Ordered list of all primary supertypes + // Orderd list of all primary supertypes In javaFrameAnchor.hpp preserve original order - globalDefinitions.hpp first. Thanks, Vladimir On 5/2/14 5:23 AM, Lindenmaier, Goetz wrote: > Hi, > > Please review and test this change. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8042195 > > This change introduces an umbrella header for orderAccess__.inline.hpp > files: runtime/orderAccess.inline.hpp. This follows the scheme applied to > atomic.inline.hpp. > > Further this change adds includes of orderAccess.inline.hpp in all .cpp > and .inline.hpp files where a method of OrderAccess declared 'inline' is > called. > > Finally it moves methods calling inline methods of OrderAccess from > g1CollectedHeap.hpp and thread.hpp to the corresponding .inline.hpp > files and adds the necessary includes in files using the moved > functions. > > Still a lot of calls to inline methods of OrderAccess in .hpp > files arn't preceded by an according include. But for these > files an appropriate .inline.hpp file is missing: > > src/cpu/ppc/vm/javaFrameAnchor_ppc.hpp > src/cpu/sparc/vm/javaFrameAnchor_sparc.hpp > src/share/vm/classfile/classLoader.hpp > src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp > src/share/vm/memory/cardTableModRefBS.hpp > src/share/vm/oops/constantPool.hpp > src/share/vm/oops/cpCache.hpp > src/share/vm/oops/instanceKlass.hpp > src/share/vm/oops/method.hpp > src/share/vm/oops/methodData.hpp > src/share/vm/oops/typeArrayOop.hpp > src/share/vm/runtime/interfaceSupport.hpp > src/share/vm/utilities/array.hpp > src/share/vm/utilities/taskqueue.hpp > > Testing: > I ran builds on linuxx86_64, sun_32, sun_64, mac, windowsx86_64, linuxppc64 and aix; > dbg, fastdbg and product without precompiled headers. > > Thanks and best regards, > Goetz > From mike.duigou at oracle.com Sun May 4 03:24:06 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Sat, 3 May 2014 20:24:06 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <531F05D3.6020008@oracle.com> References: <531F05D3.6020008@oracle.com> Message-ID: Hello; Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. https://bugs.openjdk.java.net/browse/JDK-8032045 http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ Mike On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: > On 2014-03-11 00:49, Mike Duigou wrote: >> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >> >> https://bugs.openjdk.java.net/browse/JDK-8032045 >> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >> >> This version is, hopefully, almost ready to be pushed. > > I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. > > The top-level build changes looks fine. Thank you Mike for cleaning things up! > > /Magnus > >> >> Mike >> >> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >> >>> Hello all; >>> >>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>> >>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>> >>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>> >>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>> >>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>> >>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>> >>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>> >>> Thanks, >>> >>> Mike > From aph at redhat.com Sun May 4 12:05:13 2014 From: aph at redhat.com (Andrew Haley) Date: Sun, 04 May 2014 13:05:13 +0100 Subject: RFR [XS] 8041658: Use of -fdevirtualize on macroAssembler_x86.o (via -O2) with gcc 4.9.0 creates broken VM In-Reply-To: References: <1699862096.9102793.1398308854043.JavaMail.zimbra@redhat.com> <1742674290.9102964.1398309025077.JavaMail.zimbra@redhat.com> <5358FFC1.7020709@redhat.com> <535924D9.2070404@redhat.com> <53593EA1.2060308@redhat.com> <3645314C-08FB-467D-92A4-C59775FC7093@oracle.com> <535A3980.9020606@redhat.com> <535A6547.3070906@redhat.com> Message-ID: <53662CF9.4010103@redhat.com> On 04/25/2014 03:19 PM, Volker Simonis wrote: > Thanks for opening the bug! > > They first opened http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60963 > but categorized it as 'ubsan' (i.e. sanitizer bug) which it isn't. > > Hopefully they'll fix it soon:) There's a GCC patch at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60965#c6 Andrew. From per.liden at oracle.com Mon May 5 14:02:19 2014 From: per.liden at oracle.com (Per Liden) Date: Mon, 05 May 2014 16:02:19 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <5360DBF1.3050807@oracle.com> References: <5360DBF1.3050807@oracle.com> Message-ID: <536799EB.3040503@oracle.com> Looks good Erik! One comment in-line below. On 04/30/2014 01:18 PM, Erik Helin wrote: > Hi all, > > this patch solves a rather tricky problem with the sizing of Metaspace. > > The issue happens when the GC threshold for Metaspace (called > "capacity_until_GC" in the code) becomes less than the committed memory > for Metaspace. Any calls to Metaspace::allocate that requires committing > more memory will then fail in MetaspaceGC::allowed_expansion, because > capacity_until_GC() < MetaspaceAux::committed_memory(). The effect will > be a > full GC and after the GC we try to expand and allocate. After the > expansion and before the allocation, one of two things can happen: > 1. capacity_until_GC is larger than the committed memory after the > expansion. The allocation will now succeed, but the next allocation > requiring a new chunk will *again* trigger a full GC. This pattern > will repeat itself for each new allocation request requiring a new > chunk. > 2. capacity_until_GC is still less than the committed memory even > after the expansion. We throw a Java OOME (incorrectly). > > How can the GC threshold for Metaspace be less than the committed > memory? The problem is that MetaspaceGC::compute_new_size uses the field > _allocated_capacity for describing the amount of memory in Metaspace > that is "in use". _allocated_capacity does not consider the memory in > the chunk free lists to be "in use", since memory in the chunk free > lists are supposed to be available for new allocations. The problem is > that the chunk free lists can become fragmented, and then the memory is > not available for all kinds of allocations. > > This patch change MetaspaceGC::compute_new_size to use > MetaspaceAux::committed_memory for describing how much memory that is > "in use". The effect will be that memory in the chunk free lists will no > longer be considered "in use" (but will of course be used for future (I already mentioned this to Erik in private, but just in case someone else was confused about this) ... will no longer be considered "in use"... should be: ... will now be considered "in use"... cheers, /Per > allocations where possible). This will prevent capacity_until_GC from > shrinking below the committed memory "by definiton", since > capacity_until_GC can't be lower than the memory that is "in use". > > Based on the results from the perf testing (see below), this change has > no performance impact. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ > > Testing: > - JPRT > - Ad-hoc testing: > - Kitchensink > - Dacapo > - Medrec > - runThese > - Parallel Class Loading testlist > - Metaspace testlist > - GC nightly testlist > - Perf testing: > - SPECjbb2005 > - SPECjbb2013 > - Derby > - Derby regression tests > > Thanks, > Erik From erik.helin at oracle.com Mon May 5 14:06:52 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 05 May 2014 16:06:52 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <536285BE.2060303@oracle.com> References: <5360DBF1.3050807@oracle.com> <536285BE.2060303@oracle.com> Message-ID: <53679AFC.6010907@oracle.com> Hi Jon, thanks for having a look at the patch! On 2014-05-01 19:34, Jon Masamitsu wrote: > Erik, > > Change looks good. Thanks! On 2014-05-01 19:34, Jon Masamitsu wrote: > Please add a comment explaining why we're using committed now and why > capacity did not work. Basically an abbreviated version of your > explanation > below. The old code in compute_new_size was the last code in metaspace making use of "capacity" as a concept. My idea was to remove "capacity" completely from the metaspace code after this change. Do you think that we still should add a comment discussing a concept (capacity) that the code no longer will use? Is there a possibility that people that are new to the code will become confused? Thanks, Erik On 2014-05-01 19:34, Jon Masamitsu wrote: > Thanks. > > Jon > > On 4/30/2014 4:18 AM, Erik Helin wrote: >> Hi all, >> >> this patch solves a rather tricky problem with the sizing of Metaspace. >> >> The issue happens when the GC threshold for Metaspace (called >> "capacity_until_GC" in the code) becomes less than the committed >> memory for Metaspace. Any calls to Metaspace::allocate that requires >> committing more memory will then fail in >> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >> MetaspaceAux::committed_memory(). The effect will be a >> full GC and after the GC we try to expand and allocate. After the >> expansion and before the allocation, one of two things can happen: >> 1. capacity_until_GC is larger than the committed memory after the >> expansion. The allocation will now succeed, but the next allocation >> requiring a new chunk will *again* trigger a full GC. This pattern >> will repeat itself for each new allocation request requiring a new >> chunk. >> 2. capacity_until_GC is still less than the committed memory even >> after the expansion. We throw a Java OOME (incorrectly). >> >> How can the GC threshold for Metaspace be less than the committed >> memory? The problem is that MetaspaceGC::compute_new_size uses the field >> _allocated_capacity for describing the amount of memory in Metaspace >> that is "in use". _allocated_capacity does not consider the memory in >> the chunk free lists to be "in use", since memory in the chunk free >> lists are supposed to be available for new allocations. The problem is >> that the chunk free lists can become fragmented, and then the memory >> is not available for all kinds of allocations. >> >> This patch change MetaspaceGC::compute_new_size to use >> MetaspaceAux::committed_memory for describing how much memory that is >> "in use". The effect will be that memory in the chunk free lists will no >> longer be considered "in use" (but will of course be used for future >> allocations where possible). This will prevent capacity_until_GC from >> shrinking below the committed memory "by definiton", since >> capacity_until_GC can't be lower than the memory that is "in use". >> >> Based on the results from the perf testing (see below), this change >> has no performance impact. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >> >> Testing: >> - JPRT >> - Ad-hoc testing: >> - Kitchensink >> - Dacapo >> - Medrec >> - runThese >> - Parallel Class Loading testlist >> - Metaspace testlist >> - GC nightly testlist >> - Perf testing: >> - SPECjbb2005 >> - SPECjbb2013 >> - Derby >> - Derby regression tests >> >> Thanks, >> Erik > From erik.helin at oracle.com Mon May 5 14:08:25 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 05 May 2014 16:08:25 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <536799EB.3040503@oracle.com> References: <5360DBF1.3050807@oracle.com> <536799EB.3040503@oracle.com> Message-ID: <53679B59.8050409@oracle.com> On 2014-05-05 16:02, Per Liden wrote: > Looks good Erik! Thanks! > One comment in-line below. > > On 04/30/2014 01:18 PM, Erik Helin wrote: >> Hi all, >> >> this patch solves a rather tricky problem with the sizing of Metaspace. >> >> The issue happens when the GC threshold for Metaspace (called >> "capacity_until_GC" in the code) becomes less than the committed memory >> for Metaspace. Any calls to Metaspace::allocate that requires committing >> more memory will then fail in MetaspaceGC::allowed_expansion, because >> capacity_until_GC() < MetaspaceAux::committed_memory(). The effect will >> be a >> full GC and after the GC we try to expand and allocate. After the >> expansion and before the allocation, one of two things can happen: >> 1. capacity_until_GC is larger than the committed memory after the >> expansion. The allocation will now succeed, but the next allocation >> requiring a new chunk will *again* trigger a full GC. This pattern >> will repeat itself for each new allocation request requiring a new >> chunk. >> 2. capacity_until_GC is still less than the committed memory even >> after the expansion. We throw a Java OOME (incorrectly). >> >> How can the GC threshold for Metaspace be less than the committed >> memory? The problem is that MetaspaceGC::compute_new_size uses the field >> _allocated_capacity for describing the amount of memory in Metaspace >> that is "in use". _allocated_capacity does not consider the memory in >> the chunk free lists to be "in use", since memory in the chunk free >> lists are supposed to be available for new allocations. The problem is >> that the chunk free lists can become fragmented, and then the memory is >> not available for all kinds of allocations. >> >> This patch change MetaspaceGC::compute_new_size to use >> MetaspaceAux::committed_memory for describing how much memory that is >> "in use". The effect will be that memory in the chunk free lists will no >> longer be considered "in use" (but will of course be used for future > > (I already mentioned this to Erik in private, but just in case someone > else was confused about this) > > ... will no longer be considered "in use"... > > should be: > > ... will now be considered "in use"... Yes, that is correct, thanks for finding this! Erik > cheers, > /Per > >> allocations where possible). This will prevent capacity_until_GC from >> shrinking below the committed memory "by definiton", since >> capacity_until_GC can't be lower than the memory that is "in use". >> >> Based on the results from the perf testing (see below), this change has >> no performance impact. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >> >> Testing: >> - JPRT >> - Ad-hoc testing: >> - Kitchensink >> - Dacapo >> - Medrec >> - runThese >> - Parallel Class Loading testlist >> - Metaspace testlist >> - GC nightly testlist >> - Perf testing: >> - SPECjbb2005 >> - SPECjbb2013 >> - Derby >> - Derby regression tests >> >> Thanks, >> Erik > From jon.masamitsu at oracle.com Tue May 6 03:31:32 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 05 May 2014 20:31:32 -0700 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <53679AFC.6010907@oracle.com> References: <5360DBF1.3050807@oracle.com> <536285BE.2060303@oracle.com> <53679AFC.6010907@oracle.com> Message-ID: <53685794.3070205@oracle.com> On 5/5/2014 7:06 AM, Erik Helin wrote: > Hi Jon, > > thanks for having a look at the patch! > > On 2014-05-01 19:34, Jon Masamitsu wrote: >> Erik, >> >> Change looks good. > > Thanks! > > On 2014-05-01 19:34, Jon Masamitsu wrote: >> Please add a comment explaining why we're using committed now and why >> capacity did not work. Basically an abbreviated version of your >> explanation >> below. > > The old code in compute_new_size was the last code in metaspace making > use of "capacity" as a concept. My idea was to remove "capacity" > completely from the metaspace code after this change. > > Do you think that we still should add a comment discussing a concept > (capacity) that the code no longer will use? Is there a possibility > that people that are new to the code will become confused? Capacity is a concept that is present in the GC code. I really, really don't want to see capacity come back into the metaspace code and adding a comment is the only way I can think of to prevent it. Jon > > Thanks, > Erik > > On 2014-05-01 19:34, Jon Masamitsu wrote: >> Thanks. >> >> Jon >> >> On 4/30/2014 4:18 AM, Erik Helin wrote: >>> Hi all, >>> >>> this patch solves a rather tricky problem with the sizing of Metaspace. >>> >>> The issue happens when the GC threshold for Metaspace (called >>> "capacity_until_GC" in the code) becomes less than the committed >>> memory for Metaspace. Any calls to Metaspace::allocate that requires >>> committing more memory will then fail in >>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >>> MetaspaceAux::committed_memory(). The effect will be a >>> full GC and after the GC we try to expand and allocate. After the >>> expansion and before the allocation, one of two things can happen: >>> 1. capacity_until_GC is larger than the committed memory after the >>> expansion. The allocation will now succeed, but the next allocation >>> requiring a new chunk will *again* trigger a full GC. This pattern >>> will repeat itself for each new allocation request requiring a new >>> chunk. >>> 2. capacity_until_GC is still less than the committed memory even >>> after the expansion. We throw a Java OOME (incorrectly). >>> >>> How can the GC threshold for Metaspace be less than the committed >>> memory? The problem is that MetaspaceGC::compute_new_size uses the >>> field >>> _allocated_capacity for describing the amount of memory in Metaspace >>> that is "in use". _allocated_capacity does not consider the memory in >>> the chunk free lists to be "in use", since memory in the chunk free >>> lists are supposed to be available for new allocations. The problem is >>> that the chunk free lists can become fragmented, and then the memory >>> is not available for all kinds of allocations. >>> >>> This patch change MetaspaceGC::compute_new_size to use >>> MetaspaceAux::committed_memory for describing how much memory that is >>> "in use". The effect will be that memory in the chunk free lists >>> will no >>> longer be considered "in use" (but will of course be used for future >>> allocations where possible). This will prevent capacity_until_GC from >>> shrinking below the committed memory "by definiton", since >>> capacity_until_GC can't be lower than the memory that is "in use". >>> >>> Based on the results from the perf testing (see below), this change >>> has no performance impact. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >>> >>> Testing: >>> - JPRT >>> - Ad-hoc testing: >>> - Kitchensink >>> - Dacapo >>> - Medrec >>> - runThese >>> - Parallel Class Loading testlist >>> - Metaspace testlist >>> - GC nightly testlist >>> - Perf testing: >>> - SPECjbb2005 >>> - SPECjbb2013 >>> - Derby >>> - Derby regression tests >>> >>> Thanks, >>> Erik >> From albert.noll at oracle.com Tue May 6 05:52:55 2014 From: albert.noll at oracle.com (Albert) Date: Tue, 06 May 2014 07:52:55 +0200 Subject: [8u20] RFR(S): 8040085: dtrace/jsdt tests crash on solaris. found an unadvertised bad scavengable oop in the code cache Message-ID: <536878B7.2020001@oracle.com> Hi, I would like to request a 8u20 backport of JDK-8040085. The changes were pushed to 9 on 2014-04-28 and nightly testing showed no problems. The changes applied to 8u20 without conflicts. Master bug: https://bugs.openjdk.java.net/browse/JDK-8040085 Webrev: http://cr.openjdk.java.net/~anoll/8040085/webrev.02/ Changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ccbe4f8119b6 Thanks, Albert From vladimir.kozlov at oracle.com Tue May 6 06:33:12 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 05 May 2014 23:33:12 -0700 Subject: [8u20] RFR(S): 8040085: dtrace/jsdt tests crash on solaris. found an unadvertised bad scavengable oop in the code cache In-Reply-To: <536878B7.2020001@oracle.com> References: <536878B7.2020001@oracle.com> Message-ID: <53688228.5060403@oracle.com> Looks fine. Vladimir On 5/5/14 10:52 PM, Albert wrote: > Hi, > > I would like to request a 8u20 backport of JDK-8040085. > > The changes were pushed to 9 on 2014-04-28 and nightly testing showed no problems. > The changes applied to 8u20 without conflicts. > > Master bug: https://bugs.openjdk.java.net/browse/JDK-8040085 > Webrev: http://cr.openjdk.java.net/~anoll/8040085/webrev.02/ > Changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ccbe4f8119b6 > > Thanks, > Albert From albert.noll at oracle.com Tue May 6 06:43:21 2014 From: albert.noll at oracle.com (Albert) Date: Tue, 06 May 2014 08:43:21 +0200 Subject: [8u20] RFR(S): 8040085: dtrace/jsdt tests crash on solaris. found an unadvertised bad scavengable oop in the code cache In-Reply-To: <53688228.5060403@oracle.com> References: <536878B7.2020001@oracle.com> <53688228.5060403@oracle.com> Message-ID: <53688489.8020502@oracle.com> Thank you, Vladimir. Best, Albert On 05/06/2014 08:33 AM, Vladimir Kozlov wrote: > Looks fine. > > Vladimir > > On 5/5/14 10:52 PM, Albert wrote: >> Hi, >> >> I would like to request a 8u20 backport of JDK-8040085. >> >> The changes were pushed to 9 on 2014-04-28 and nightly testing showed >> no problems. >> The changes applied to 8u20 without conflicts. >> >> Master bug: https://bugs.openjdk.java.net/browse/JDK-8040085 >> Webrev: http://cr.openjdk.java.net/~anoll/8040085/webrev.02/ >> Changeset: >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ccbe4f8119b6 >> >> Thanks, >> Albert From vozerov at gridgain.com Tue May 6 06:54:19 2014 From: vozerov at gridgain.com (Vladimir Ozerov) Date: Tue, 6 May 2014 10:54:19 +0400 Subject: Possible C2 compiler bug (Review ID: JI-9011841) Message-ID: Hi, Recently we revealed a bug causing read of incorrect memory location when C2 compiles particular method in JDK7u55. I submitted it to Oracle and currently it is under review with ID JI-9011841. As this bug affects our customers and appears to be a security issue as well, can I ask HotSpot team to look at it with priority? The bug can be easily reproduced with the following code: import sun.misc.*; import java.lang.reflect.*; import java.util.*; public class JVMFailure { private static final Unsafe UNSAFE = unsafe(); private static final long BYTE_ARR_OFF = UNSAFE.arrayBaseOffset(byte[].class); private static Unsafe unsafe() { try { return Unsafe.getUnsafe(); } catch (Exception e) { try { Field field = Unsafe.class.getDeclaredField("theUnsafe"); field.setAccessible(true); return (Unsafe)field.get(null); } catch (Exception e1) { throw new RuntimeException(e1); } } } private static int readInt(long ptr) { return UNSAFE.getInt(ptr); } private static byte[] readBytes(long ptr, byte[] arr) { UNSAFE.copyMemory(null, ptr, arr, BYTE_ARR_OFF, arr.length); return arr; } private Deque ptrs = new ArrayDeque<>(); private void put(byte[] keyBytes, byte[] valBytes) { long ptr = UNSAFE.allocateMemory(12 + keyBytes.length + valBytes.length); UNSAFE.putInt(ptr + 4, keyBytes.length); UNSAFE.putInt(ptr + 8, valBytes.length); UNSAFE.copyMemory(keyBytes, BYTE_ARR_OFF, null, ptr + 12, keyBytes.length); UNSAFE.copyMemory(valBytes, BYTE_ARR_OFF, null, ptr + 12 + keyBytes.length, valBytes.length); ptrs.addLast(ptr); if (ptrs.size() > 100) { long ptr0 = ptrs.pollFirst(); byte[] valBytes0 = faultyMethod(ptr0); if (valBytes0[0] != 0) System.out.println(Arrays.toString(valBytes0)); int keySize = UNSAFE.getInt(ptr + 4); int valSize = UNSAFE.getInt(ptr + 8); UNSAFE.setMemory(ptr, 12 + keySize + valSize, (byte)0xAB); UNSAFE.freeMemory(ptr); } } private byte[] faultyMethod(long ptr) { int keySize = readInt(ptr + 4); int valSize = readInt(ptr + 8); return readBytes(ptr + 12 + keySize, new byte[valSize]); } public static void main(String[] args) { JVMFailure fail = new JVMFailure(); byte[] keyBytes = new byte[] { 127, 127, 127, 127 }; byte[] valBytes = new byte[74]; for (int i = 0; i < 74; i++) valBytes[i] = (byte)i; for (;;) fail.put(keyBytes, valBytes); } } Set the following JVM arguments: "-XX:+UnlockDiagnosticVMOptions -ea -XX:+PrintCompilation -XX:+LogCompilation -XX:CompileCommand=exclude,JVMFailure,put" and run it. As soon as "faultyMethod" is compiled the program begin printing read byte array revealing that it was read with incorrect offset. Root cause of this problem is invalid code generated by C2 which mistakenly changes value of particular register thus shifting memory address to be read (see bug report for more details). The problem is reporducible in JDK7u55 on both Windows and Linux. Not reproducible in JDK8 starting from early-access version. Regards, Vladimir Ozerov. From erik.helin at oracle.com Tue May 6 08:44:06 2014 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 06 May 2014 10:44:06 +0200 Subject: RFR: 8027915: TestParallelHeapSizeFlags fails with unexpected heap size on sparcv9 Message-ID: <5368A0D6.5050609@oracle.com> Hi all, this patch changes os::page_size_for_region(min_size, max_size, min_pages) to always guarantee that the returned page size <= max_size / min_pages. Prior to this patch, page_size_for_region could return a page size larger than max_size / min_pages. This can cause some unexpected behavior for code using page_size_for_region, such as the following regression tests: http://cr.openjdk.java.net/~ehelin/8027915/regression-test/ As the test shows, if you have a 2 MB region and you must use at least 2 pages, you would still get back 2 MB as the page size on a Linux machine that supports 2 MB large pages. The reason for this is explained in a comment above the function in os.hpp: // The current implementation ignores min_pages if a larger page // size is an exact multiple of both region_min_size and // region_max_size. This allows larger pages to be used when doing // so would not cause fragmentation; in particular, a single page can // be used when region_min_size == region_max_size == a supported page // size. Reducing fragmentation of large pages is good but the varying return value makes it very hard to depend on os::page_size_for_region in calculations (for example when sizing the heap). This patch removes the fragmentation prevention mechanism, simplifies the function and adds a regression tests: http://cr.openjdk.java.net/~ehelin/8027915/webrev.00/ Testing: - JPRT Thanks, Erik From volker.simonis at gmail.com Tue May 6 09:14:55 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 6 May 2014 11:14:55 +0200 Subject: RFR(XXS): 8042443: Fix C++-Interpreter after "8036956: remove EnableInvokeDynamic flag" Message-ID: Hi, could somebody please review and sponsor this one-line change which fixes the C++-Interpreter build after change "8036956: remove EnableInvokeDynamic flag" introduced a bogus "}" in bytecodeInterpreter.cpp: http://cr.openjdk.java.net/~simonis/webrevs/8042443/ https://bugs.openjdk.java.net/browse/JDK-8042443 Change "8036956: remove EnableInvokeDynamic flag" introduced a bogus "}" which prevents the C++-Interpreter from building. The fix is trivial: diff -r 5f1e8fe1a4af src/share/vm/interpreter/bytecodeInterpreter.cpp --- a/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 09:13:36 2014 +0200 +++ b/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 10:50:32 2014 +0200 @@ -508,7 +508,6 @@ #ifdef ASSERT if (istate->_msg != initialize) { assert(labs(istate->_stack_base - istate->_stack_limit) == (istate->_method->max_stack() + 1), "bad stack limit"); - } #ifndef SHARK IA32_ONLY(assert(istate->_stack_limit == istate->_thread->last_Java_sp() + 1, "wrong")); #endif // !SHARK Thank you and best regards, Volker From staffan.larsen at oracle.com Tue May 6 09:17:29 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 6 May 2014 11:17:29 +0200 Subject: RFR(XXS): 8042443: Fix C++-Interpreter after "8036956: remove EnableInvokeDynamic flag" In-Reply-To: References: Message-ID: <45AF51E1-91A3-4AE6-A695-CA158EBBBFAF@oracle.com> Looks good to me. I guess this should go into hs-comp? /Staffan On 6 maj 2014, at 11:14, Volker Simonis wrote: > Hi, > > could somebody please review and sponsor this one-line change which > fixes the C++-Interpreter build after change "8036956: remove > EnableInvokeDynamic flag" introduced a bogus "}" in > bytecodeInterpreter.cpp: > > http://cr.openjdk.java.net/~simonis/webrevs/8042443/ > https://bugs.openjdk.java.net/browse/JDK-8042443 > > Change "8036956: remove EnableInvokeDynamic flag" introduced a bogus > "}" which prevents the C++-Interpreter from building. The fix is > trivial: > > diff -r 5f1e8fe1a4af src/share/vm/interpreter/bytecodeInterpreter.cpp > --- a/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 > 09:13:36 2014 +0200 > +++ b/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 > 10:50:32 2014 +0200 > @@ -508,7 +508,6 @@ > #ifdef ASSERT > if (istate->_msg != initialize) { > assert(labs(istate->_stack_base - istate->_stack_limit) == > (istate->_method->max_stack() + 1), "bad stack limit"); > - } > #ifndef SHARK > IA32_ONLY(assert(istate->_stack_limit == > istate->_thread->last_Java_sp() + 1, "wrong")); > #endif // !SHARK > > > Thank you and best regards, > Volker From volker.simonis at gmail.com Tue May 6 09:23:21 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 6 May 2014 11:23:21 +0200 Subject: RFR(XXS): 8042443: Fix C++-Interpreter after "8036956: remove EnableInvokeDynamic flag" In-Reply-To: <45AF51E1-91A3-4AE6-A695-CA158EBBBFAF@oracle.com> References: <45AF51E1-91A3-4AE6-A695-CA158EBBBFAF@oracle.com> Message-ID: Right. As far as I can see, hs-comp is the the only repository where the offending change "8036956: remove EnableInvokeDynamic flag" has been applied so far. It would be good if this fix could be pushed to hs-comp before hs-comp will be pushed down to jdk9/dev to avoid build failures of the C++-Interpreter in the subsequent forests. Thanks, Volker On Tue, May 6, 2014 at 11:17 AM, Staffan Larsen wrote: > Looks good to me. I guess this should go into hs-comp? > > /Staffan > > On 6 maj 2014, at 11:14, Volker Simonis wrote: > >> Hi, >> >> could somebody please review and sponsor this one-line change which >> fixes the C++-Interpreter build after change "8036956: remove >> EnableInvokeDynamic flag" introduced a bogus "}" in >> bytecodeInterpreter.cpp: >> >> http://cr.openjdk.java.net/~simonis/webrevs/8042443/ >> https://bugs.openjdk.java.net/browse/JDK-8042443 >> >> Change "8036956: remove EnableInvokeDynamic flag" introduced a bogus >> "}" which prevents the C++-Interpreter from building. The fix is >> trivial: >> >> diff -r 5f1e8fe1a4af src/share/vm/interpreter/bytecodeInterpreter.cpp >> --- a/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 >> 09:13:36 2014 +0200 >> +++ b/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 >> 10:50:32 2014 +0200 >> @@ -508,7 +508,6 @@ >> #ifdef ASSERT >> if (istate->_msg != initialize) { >> assert(labs(istate->_stack_base - istate->_stack_limit) == >> (istate->_method->max_stack() + 1), "bad stack limit"); >> - } >> #ifndef SHARK >> IA32_ONLY(assert(istate->_stack_limit == >> istate->_thread->last_Java_sp() + 1, "wrong")); >> #endif // !SHARK >> >> >> Thank you and best regards, >> Volker > From albert.noll at oracle.com Tue May 6 09:31:12 2014 From: albert.noll at oracle.com (Albert) Date: Tue, 06 May 2014 11:31:12 +0200 Subject: RFR(XXS): 8042443: Fix C++-Interpreter after "8036956: remove EnableInvokeDynamic flag" In-Reply-To: References: <45AF51E1-91A3-4AE6-A695-CA158EBBBFAF@oracle.com> Message-ID: <5368ABE0.2000504@oracle.com> Hi Volker, I can push your changes. I think one review should do it. Best, Albert On 05/06/2014 11:23 AM, Volker Simonis wrote: > Right. As far as I can see, hs-comp is the the only repository where > the offending change "8036956: remove EnableInvokeDynamic flag" has > been applied so far. > > It would be good if this fix could be pushed to hs-comp before hs-comp > will be pushed down to jdk9/dev to avoid build failures of the > C++-Interpreter in the subsequent forests. > > Thanks, > Volker > > On Tue, May 6, 2014 at 11:17 AM, Staffan Larsen > wrote: >> Looks good to me. I guess this should go into hs-comp? >> >> /Staffan >> >> On 6 maj 2014, at 11:14, Volker Simonis wrote: >> >>> Hi, >>> >>> could somebody please review and sponsor this one-line change which >>> fixes the C++-Interpreter build after change "8036956: remove >>> EnableInvokeDynamic flag" introduced a bogus "}" in >>> bytecodeInterpreter.cpp: >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/8042443/ >>> https://bugs.openjdk.java.net/browse/JDK-8042443 >>> >>> Change "8036956: remove EnableInvokeDynamic flag" introduced a bogus >>> "}" which prevents the C++-Interpreter from building. The fix is >>> trivial: >>> >>> diff -r 5f1e8fe1a4af src/share/vm/interpreter/bytecodeInterpreter.cpp >>> --- a/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 >>> 09:13:36 2014 +0200 >>> +++ b/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 >>> 10:50:32 2014 +0200 >>> @@ -508,7 +508,6 @@ >>> #ifdef ASSERT >>> if (istate->_msg != initialize) { >>> assert(labs(istate->_stack_base - istate->_stack_limit) == >>> (istate->_method->max_stack() + 1), "bad stack limit"); >>> - } >>> #ifndef SHARK >>> IA32_ONLY(assert(istate->_stack_limit == >>> istate->_thread->last_Java_sp() + 1, "wrong")); >>> #endif // !SHARK >>> >>> >>> Thank you and best regards, >>> Volker From volker.simonis at gmail.com Tue May 6 09:38:48 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 6 May 2014 11:38:48 +0200 Subject: RFR(XXS): 8042443: Fix C++-Interpreter after "8036956: remove EnableInvokeDynamic flag" In-Reply-To: <5368ABE0.2000504@oracle.com> References: <45AF51E1-91A3-4AE6-A695-CA158EBBBFAF@oracle.com> <5368ABE0.2000504@oracle.com> Message-ID: Thanks a lot Albert, Volker On Tue, May 6, 2014 at 11:31 AM, Albert wrote: > Hi Volker, > > I can push your changes. I think one review should do it. > > Best, > Albert > > > On 05/06/2014 11:23 AM, Volker Simonis wrote: >> >> Right. As far as I can see, hs-comp is the the only repository where >> the offending change "8036956: remove EnableInvokeDynamic flag" has >> been applied so far. >> >> It would be good if this fix could be pushed to hs-comp before hs-comp >> will be pushed down to jdk9/dev to avoid build failures of the >> C++-Interpreter in the subsequent forests. >> >> Thanks, >> Volker >> >> On Tue, May 6, 2014 at 11:17 AM, Staffan Larsen >> wrote: >>> >>> Looks good to me. I guess this should go into hs-comp? >>> >>> /Staffan >>> >>> On 6 maj 2014, at 11:14, Volker Simonis wrote: >>> >>>> Hi, >>>> >>>> could somebody please review and sponsor this one-line change which >>>> fixes the C++-Interpreter build after change "8036956: remove >>>> EnableInvokeDynamic flag" introduced a bogus "}" in >>>> bytecodeInterpreter.cpp: >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/8042443/ >>>> https://bugs.openjdk.java.net/browse/JDK-8042443 >>>> >>>> Change "8036956: remove EnableInvokeDynamic flag" introduced a bogus >>>> "}" which prevents the C++-Interpreter from building. The fix is >>>> trivial: >>>> >>>> diff -r 5f1e8fe1a4af src/share/vm/interpreter/bytecodeInterpreter.cpp >>>> --- a/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 >>>> 09:13:36 2014 +0200 >>>> +++ b/src/share/vm/interpreter/bytecodeInterpreter.cpp Tue May 06 >>>> 10:50:32 2014 +0200 >>>> @@ -508,7 +508,6 @@ >>>> #ifdef ASSERT >>>> if (istate->_msg != initialize) { >>>> assert(labs(istate->_stack_base - istate->_stack_limit) == >>>> (istate->_method->max_stack() + 1), "bad stack limit"); >>>> - } >>>> #ifndef SHARK >>>> IA32_ONLY(assert(istate->_stack_limit == >>>> istate->_thread->last_Java_sp() + 1, "wrong")); >>>> #endif // !SHARK >>>> >>>> >>>> Thank you and best regards, >>>> Volker > > From goetz.lindenmaier at sap.com Tue May 6 10:17:34 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 6 May 2014 10:17:34 +0000 Subject: RFR(M): 8042309: Some bugfixes for the ppc64 port. Message-ID: <4295855A5C1DE049A61835A1887419CC2CEBFC0A@DEWDFEMB12A.global.corp.sap> Hi, please review this change. I also please need a sponsor. It should go to jdk8u20 at some point. It only touches files needed for the ppc64 port. http://cr.openjdk.java.net/~goetz/webrevs/8042309/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8042309 This change fixes a few bugs in the ppc64 port: * Expand of max/min nodes in .ad file relied on 64-bit sign extend but placed the values in 32-bit registers. After SpillCopies the upper bits were lost. * Expand of decodeNKlass used wrong node. * Implement missing support for method handles in generate_throw_exception. * Some small optimizations and cleanups. Thanks and best regards, Goetz. From stefan.johansson at oracle.com Tue May 6 11:12:22 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 06 May 2014 13:12:22 +0200 Subject: RFR: 8027915: TestParallelHeapSizeFlags fails with unexpected heap size on sparcv9 In-Reply-To: <5368A0D6.5050609@oracle.com> References: <5368A0D6.5050609@oracle.com> Message-ID: <5368C396.4020206@oracle.com> Hi Erik, Thanks for making page_size_for_region more strict. A few comments: src/share/vm/runtime/os.cpp With this fix page_size_for_region doesn't care about alignment at all (old version wasn't much better) and to me that feels wrong. I would expect to get a page size that the given region sizes is aligned to, but I might be missing some use case where that isn't needed. src/share/vm/gc_implementation/parallelScavenge/generationSizer.cpp For the heap we want both min and max to be a multiple of the page size so we shoud call page_size_for_region for both _min_heap_byte_size and _max_heap_byte_size and use the smallest of the two returned page sizes. src/share/vm/memory/heap.cpp The current patch will not change the default behavior so I'm fine with just calling page_size_for_region once using reserved_size, but someone with more insight might know if we should handle it the same way as the generationSizer and use both committed and reserved. Thanks, Stefan On 2014-05-06 10:44, Erik Helin wrote: > Hi all, > > this patch changes > os::page_size_for_region(min_size, max_size, min_pages) to always > guarantee that the returned page size <= max_size / min_pages. Prior > to this patch, page_size_for_region could return a page size larger > than max_size / min_pages. This can cause some unexpected behavior > for code using page_size_for_region, such as the following regression > tests: > > http://cr.openjdk.java.net/~ehelin/8027915/regression-test/ > > As the test shows, if you have a 2 MB region and you must use at least > 2 pages, you would still get back 2 MB as the page size on a Linux > machine that supports 2 MB large pages. The reason for this is > explained in a comment above the function in os.hpp: > > // The current implementation ignores min_pages if a larger page > // size is an exact multiple of both region_min_size and > // region_max_size. This allows larger pages to be used when doing > // so would not cause fragmentation; in particular, a single page can > // be used when region_min_size == region_max_size == a supported page > // size. > > Reducing fragmentation of large pages is good but the varying return > value makes it very hard to depend on os::page_size_for_region in > calculations (for example when sizing the heap). > > This patch removes the fragmentation prevention mechanism, simplifies > the function and adds a regression tests: > > http://cr.openjdk.java.net/~ehelin/8027915/webrev.00/ > > Testing: > - JPRT > > Thanks, > Erik From dmitry.fazunenko at oracle.com Tue May 6 10:58:48 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 06 May 2014 14:58:48 +0400 Subject: test/.../testlibrary/DynamicVMOption.java - common lib for dynamic VM options Message-ID: <5368C068.8000400@oracle.com> Hi, Would please look at the new version of a common library for testing dynamic VM options: http://cr.openjdk.java.net/~iignatyev/dfazunenko/8039489/webrev.00/ Updated tests using this library will look like: http://cr.openjdk.java.net/~iignatyev/dfazunenko/8041358/webrev.00/ Any your comments will be much appreciated. Thanks, Dima From vladimir.kozlov at oracle.com Tue May 6 18:55:13 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 06 May 2014 11:55:13 -0700 Subject: RFR(M): 8042309: Some bugfixes for the ppc64 port. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEBFC0A@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEBFC0A@DEWDFEMB12A.global.corp.sap> Message-ID: <53693011.2080100@oracle.com> Looks fine to me. Only minor issues in ppc.ad: New vtable_index is not used. It is strange to see predicate(false) in expanded instructions. Thanks, Vladimir On 5/6/14 3:17 AM, Lindenmaier, Goetz wrote: > Hi, > > please review this change. I also please need a sponsor. > It should go to jdk8u20 at some point. It only touches files > needed for the ppc64 port. > http://cr.openjdk.java.net/~goetz/webrevs/8042309/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8042309 > > This change fixes a few bugs in the ppc64 port: > > * Expand of max/min nodes in .ad file relied on 64-bit sign extend > but placed the values in 32-bit registers. After SpillCopies > the upper bits were lost. > * Expand of decodeNKlass used wrong node. > * Implement missing support for method handles in generate_throw_exception. > * Some small optimizations and cleanups. > > Thanks and best regards, > Goetz. > From goetz.lindenmaier at sap.com Tue May 6 19:22:19 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 6 May 2014 19:22:19 +0000 Subject: RFR(M): 8042309: Some bugfixes for the ppc64 port. In-Reply-To: <53693011.2080100@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEBFC0A@DEWDFEMB12A.global.corp.sap> <53693011.2080100@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEBFF70@DEWDFEMB12A.global.corp.sap> Hi Vladimir, I removed the dead variable and updated the webrev. I also removed a debug print. In the expanded nodes we need mach rules so they get the poper funcionaltiy generated by adlc, but we don't want them to match, as they not fully implement the required functionality. We use that quite often. Thanks for the review, Goetz. -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Tuesday, May 06, 2014 8:55 PM To: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net; 'ppc-aix-port-dev at openjdk.java.net' Subject: Re: RFR(M): 8042309: Some bugfixes for the ppc64 port. Looks fine to me. Only minor issues in ppc.ad: New vtable_index is not used. It is strange to see predicate(false) in expanded instructions. Thanks, Vladimir On 5/6/14 3:17 AM, Lindenmaier, Goetz wrote: > Hi, > > please review this change. I also please need a sponsor. > It should go to jdk8u20 at some point. It only touches files > needed for the ppc64 port. > http://cr.openjdk.java.net/~goetz/webrevs/8042309/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8042309 > > This change fixes a few bugs in the ppc64 port: > > * Expand of max/min nodes in .ad file relied on 64-bit sign extend > but placed the values in 32-bit registers. After SpillCopies > the upper bits were lost. > * Expand of decodeNKlass used wrong node. > * Implement missing support for method handles in generate_throw_exception. > * Some small optimizations and cleanups. > > Thanks and best regards, > Goetz. > From vladimir.kozlov at oracle.com Tue May 6 19:25:06 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 06 May 2014 12:25:06 -0700 Subject: [8u20] RFR for batch of fixes: 8035936, 8042281, 8041959, 8041957, 8039298 Message-ID: <53693712.6040504@oracle.com> I would like to backport next fixes into 8u20. They are applied cleanly to 8u sources. Next fix was pushed into jdk9 6 days ago: 8035936: SIGBUS in StubRoutines::aesencryptBlock, solaris-sparc http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/7c68337bb685 https://bugs.openjdk.java.net/browse/JDK-8035936 Next is the post fix of test after 8035936: 8042281: compiler/7184394/TestAESMain.java fails with: javax.crypto.IllegalBlockSizeException: Input length not multiple of 16 bytes http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ae70708e34c5 https://bugs.openjdk.java.net/browse/JDK-8042281 Next 2 fixes were pushed into jdk9 7 days ago: 8041959: Skip replay parsing errors with ReplayIgnoreInitErrors http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/4f0ae81062fd https://bugs.openjdk.java.net/browse/JDK-8041959 8041957: -XX:UseAVX=0 cause assert(UseAVX) failed http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/9883c95f17f6 https://bugs.openjdk.java.net/browse/JDK-8041957 Next fix was pushed into jdk9 4 days ago: 8039298: assert(base == NULL || t_adr->isa_rawptr() || !phase->type(base)->higher_equal(TypePtr::NULL_PTR)) http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/bd1a985905ed https://bugs.openjdk.java.net/browse/JDK-8039298 Thanks, Vladimir From zhengyu.gu at oracle.com Tue May 6 19:44:40 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Tue, 06 May 2014 15:44:40 -0400 Subject: Update: RFR(XS) 8041415: remove port.{cpp,hpp} files In-Reply-To: <5367F65A.6090906@oracle.com> References: <5367F2D5.4080606@oracle.com> <5367F65A.6090906@oracle.com> Message-ID: <53693BA8.50109@oracle.com> Hi, I updated the webrev to reflect Lois' change and Vladimir's comments. Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 Webrev: http://cr.openjdk.java.net/~zgu/8041415/webrev.01/ Thanks, -Zhengyu On 5/5/2014 4:36 PM, Vladimir Kozlov wrote: > CC to compiler alias since you changing our code. > > Copyright year change should be 2014. > > FileBuffRegion removal is done under separate bug 8041620 by Lois > Foltan, the review was sent today today. > > domgraph.cpp, output.cpp: use max_juint instead of (uint) -1. > > Thanks, > Vladimir > > On 5/5/14 1:21 PM, Zhengyu Gu wrote: >> This is a code cleanup based on attached patch from Christian Thalinger. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >> Webrev: > > http://cr.openjdk.java.net/~zgu/8041415/webrev.00/ > >> >> Test: >> JPRT >> >> Thanks, >> From vladimir.kozlov at oracle.com Tue May 6 19:44:45 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 06 May 2014 12:44:45 -0700 Subject: RFR(M): 8042309: Some bugfixes for the ppc64 port. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEBFF70@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEBFC0A@DEWDFEMB12A.global.corp.sap> <53693011.2080100@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEBFF70@DEWDFEMB12A.global.corp.sap> Message-ID: <53693BAD.8080707@oracle.com> Good. The push job is in JPRT. Thanks, Vladimir On 5/6/14 12:22 PM, Lindenmaier, Goetz wrote: > Hi Vladimir, > > I removed the dead variable and updated the webrev. > I also removed a debug print. > > In the expanded nodes we need mach rules so they get > the poper funcionaltiy generated by adlc, but we don't > want them to match, as they not fully implement the > required functionality. We use that quite often. > > Thanks for the review, > Goetz. > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Tuesday, May 06, 2014 8:55 PM > To: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net; 'ppc-aix-port-dev at openjdk.java.net' > Subject: Re: RFR(M): 8042309: Some bugfixes for the ppc64 port. > > Looks fine to me. Only minor issues in ppc.ad: > New vtable_index is not used. > It is strange to see predicate(false) in expanded instructions. > > Thanks, > Vladimir > > On 5/6/14 3:17 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> please review this change. I also please need a sponsor. >> It should go to jdk8u20 at some point. It only touches files >> needed for the ppc64 port. >> http://cr.openjdk.java.net/~goetz/webrevs/8042309/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8042309 >> >> This change fixes a few bugs in the ppc64 port: >> >> * Expand of max/min nodes in .ad file relied on 64-bit sign extend >> but placed the values in 32-bit registers. After SpillCopies >> the upper bits were lost. >> * Expand of decodeNKlass used wrong node. >> * Implement missing support for method handles in generate_throw_exception. >> * Some small optimizations and cleanups. >> >> Thanks and best regards, >> Goetz. >> From vladimir.kozlov at oracle.com Tue May 6 19:50:09 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 06 May 2014 12:50:09 -0700 Subject: Update: RFR(XS) 8041415: remove port.{cpp,hpp} files In-Reply-To: <53693BA8.50109@oracle.com> References: <5367F2D5.4080606@oracle.com> <5367F65A.6090906@oracle.com> <53693BA8.50109@oracle.com> Message-ID: <53693CF1.9010506@oracle.com> output.cpp change still use (uint) -1 Vladimir On 5/6/14 12:44 PM, Zhengyu Gu wrote: > Hi, > > I updated the webrev to reflect Lois' change and Vladimir's comments. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 > Webrev: http://cr.openjdk.java.net/~zgu/8041415/webrev.01/ > > > > Thanks, > > -Zhengyu > > On 5/5/2014 4:36 PM, Vladimir Kozlov wrote: >> CC to compiler alias since you changing our code. >> >> Copyright year change should be 2014. >> >> FileBuffRegion removal is done under separate bug 8041620 by Lois >> Foltan, the review was sent today today. >> >> domgraph.cpp, output.cpp: use max_juint instead of (uint) -1. >> >> Thanks, >> Vladimir >> >> On 5/5/14 1:21 PM, Zhengyu Gu wrote: >>> This is a code cleanup based on attached patch from Christian Thalinger. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>> Webrev: >> >> http://cr.openjdk.java.net/~zgu/8041415/webrev.00/ >> >>> >>> Test: >>> JPRT >>> >>> Thanks, >>> > From zhengyu.gu at oracle.com Tue May 6 20:33:54 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Tue, 06 May 2014 16:33:54 -0400 Subject: Update: RFR(XS) 8041415: remove port.{cpp,hpp} files In-Reply-To: <53693CF1.9010506@oracle.com> References: <5367F2D5.4080606@oracle.com> <5367F65A.6090906@oracle.com> <53693BA8.50109@oracle.com> <53693CF1.9010506@oracle.com> Message-ID: <53694732.5090303@oracle.com> Sorry, missed this one. One more try: http://cr.openjdk.java.net/~zgu/8041415/webrev.02/ Thanks, -Zhengyu On 5/6/2014 3:50 PM, Vladimir Kozlov wrote: > output.cpp change still use (uint) -1 > > Vladimir > > On 5/6/14 12:44 PM, Zhengyu Gu wrote: >> Hi, >> >> I updated the webrev to reflect Lois' change and Vladimir's comments. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >> Webrev: http://cr.openjdk.java.net/~zgu/8041415/webrev.01/ >> >> >> >> Thanks, >> >> -Zhengyu >> >> On 5/5/2014 4:36 PM, Vladimir Kozlov wrote: >>> CC to compiler alias since you changing our code. >>> >>> Copyright year change should be 2014. >>> >>> FileBuffRegion removal is done under separate bug 8041620 by Lois >>> Foltan, the review was sent today today. >>> >>> domgraph.cpp, output.cpp: use max_juint instead of (uint) -1. >>> >>> Thanks, >>> Vladimir >>> >>> On 5/5/14 1:21 PM, Zhengyu Gu wrote: >>>> This is a code cleanup based on attached patch from Christian >>>> Thalinger. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>>> Webrev: >>> >>> http://cr.openjdk.java.net/~zgu/8041415/webrev.00/ >>> >>>> >>>> Test: >>>> JPRT >>>> >>>> Thanks, >>>> >> From vladimir.kozlov at oracle.com Tue May 6 20:46:16 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 06 May 2014 13:46:16 -0700 Subject: Update: RFR(XS) 8041415: remove port.{cpp,hpp} files In-Reply-To: <53694732.5090303@oracle.com> References: <5367F2D5.4080606@oracle.com> <5367F65A.6090906@oracle.com> <53693BA8.50109@oracle.com> <53693CF1.9010506@oracle.com> <53694732.5090303@oracle.com> Message-ID: <53694A18.2000902@oracle.com> This looks good. Thanks, Vladimir On 5/6/14 1:33 PM, Zhengyu Gu wrote: > Sorry, missed this one. > > One more try: http://cr.openjdk.java.net/~zgu/8041415/webrev.02/ > > > Thanks, > > -Zhengyu > > On 5/6/2014 3:50 PM, Vladimir Kozlov wrote: >> output.cpp change still use (uint) -1 >> >> Vladimir >> >> On 5/6/14 12:44 PM, Zhengyu Gu wrote: >>> Hi, >>> >>> I updated the webrev to reflect Lois' change and Vladimir's comments. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>> Webrev: http://cr.openjdk.java.net/~zgu/8041415/webrev.01/ >>> >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >>> On 5/5/2014 4:36 PM, Vladimir Kozlov wrote: >>>> CC to compiler alias since you changing our code. >>>> >>>> Copyright year change should be 2014. >>>> >>>> FileBuffRegion removal is done under separate bug 8041620 by Lois >>>> Foltan, the review was sent today today. >>>> >>>> domgraph.cpp, output.cpp: use max_juint instead of (uint) -1. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 5/5/14 1:21 PM, Zhengyu Gu wrote: >>>>> This is a code cleanup based on attached patch from Christian >>>>> Thalinger. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~zgu/8041415/webrev.00/ >>>> >>>>> >>>>> Test: >>>>> JPRT >>>>> >>>>> Thanks, >>>>> >>> > From coleen.phillimore at oracle.com Tue May 6 20:47:38 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 May 2014 16:47:38 -0400 Subject: Update: RFR(XS) 8041415: remove port.{cpp,hpp} files In-Reply-To: <53694732.5090303@oracle.com> References: <5367F2D5.4080606@oracle.com> <5367F65A.6090906@oracle.com> <53693BA8.50109@oracle.com> <53693CF1.9010506@oracle.com> <53694732.5090303@oracle.com> Message-ID: <53694A6A.8090808@oracle.com> This looks good to me, but I think you need someone from the compiler group to review also because it's all in the compiler code. Thanks! Coleen On 5/6/14, 4:33 PM, Zhengyu Gu wrote: > Sorry, missed this one. > > One more try: http://cr.openjdk.java.net/~zgu/8041415/webrev.02/ > > > Thanks, > > -Zhengyu > > On 5/6/2014 3:50 PM, Vladimir Kozlov wrote: >> output.cpp change still use (uint) -1 >> >> Vladimir >> >> On 5/6/14 12:44 PM, Zhengyu Gu wrote: >>> Hi, >>> >>> I updated the webrev to reflect Lois' change and Vladimir's comments. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>> Webrev: http://cr.openjdk.java.net/~zgu/8041415/webrev.01/ >>> >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >>> On 5/5/2014 4:36 PM, Vladimir Kozlov wrote: >>>> CC to compiler alias since you changing our code. >>>> >>>> Copyright year change should be 2014. >>>> >>>> FileBuffRegion removal is done under separate bug 8041620 by Lois >>>> Foltan, the review was sent today today. >>>> >>>> domgraph.cpp, output.cpp: use max_juint instead of (uint) -1. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 5/5/14 1:21 PM, Zhengyu Gu wrote: >>>>> This is a code cleanup based on attached patch from Christian >>>>> Thalinger. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~zgu/8041415/webrev.00/ >>>> >>>>> >>>>> Test: >>>>> JPRT >>>>> >>>>> Thanks, >>>>> >>> > From christian.thalinger at oracle.com Wed May 7 00:19:14 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 6 May 2014 14:19:14 -1000 Subject: [8u20] RFR for batch of fixes: 8035936, 8042281, 8041959, 8041957, 8039298 In-Reply-To: <53693712.6040504@oracle.com> References: <53693712.6040504@oracle.com> Message-ID: <06C4D72B-84A3-4618-AB06-CCCE1264AAE2@oracle.com> All of them are good for 8u. On May 6, 2014, at 9:25 AM, Vladimir Kozlov wrote: > I would like to backport next fixes into 8u20. They are applied cleanly to 8u sources. > > Next fix was pushed into jdk9 6 days ago: > > 8035936: SIGBUS in StubRoutines::aesencryptBlock, solaris-sparc > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/7c68337bb685 > https://bugs.openjdk.java.net/browse/JDK-8035936 > > Next is the post fix of test after 8035936: > > 8042281: compiler/7184394/TestAESMain.java fails with: javax.crypto.IllegalBlockSizeException: Input length not multiple of 16 bytes > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ae70708e34c5 > https://bugs.openjdk.java.net/browse/JDK-8042281 > > Next 2 fixes were pushed into jdk9 7 days ago: > > 8041959: Skip replay parsing errors with ReplayIgnoreInitErrors > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/4f0ae81062fd > https://bugs.openjdk.java.net/browse/JDK-8041959 > > 8041957: -XX:UseAVX=0 cause assert(UseAVX) failed > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/9883c95f17f6 > https://bugs.openjdk.java.net/browse/JDK-8041957 > > Next fix was pushed into jdk9 4 days ago: > > 8039298: assert(base == NULL || t_adr->isa_rawptr() || !phase->type(base)->higher_equal(TypePtr::NULL_PTR)) > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/bd1a985905ed > https://bugs.openjdk.java.net/browse/JDK-8039298 > > Thanks, > Vladimir > From vladimir.kozlov at oracle.com Wed May 7 00:30:51 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 06 May 2014 17:30:51 -0700 Subject: [8u20] RFR for batch of fixes: 8035936, 8042281, 8041959, 8041957, 8039298 In-Reply-To: <06C4D72B-84A3-4618-AB06-CCCE1264AAE2@oracle.com> References: <53693712.6040504@oracle.com> <06C4D72B-84A3-4618-AB06-CCCE1264AAE2@oracle.com> Message-ID: <53697EBB.7040107@oracle.com> Thank you, Christian Vladimir On 5/6/14 5:19 PM, Christian Thalinger wrote: > All of them are good for 8u. > > On May 6, 2014, at 9:25 AM, Vladimir Kozlov wrote: > >> I would like to backport next fixes into 8u20. They are applied cleanly to 8u sources. >> >> Next fix was pushed into jdk9 6 days ago: >> >> 8035936: SIGBUS in StubRoutines::aesencryptBlock, solaris-sparc >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/7c68337bb685 >> https://bugs.openjdk.java.net/browse/JDK-8035936 >> >> Next is the post fix of test after 8035936: >> >> 8042281: compiler/7184394/TestAESMain.java fails with: javax.crypto.IllegalBlockSizeException: Input length not multiple of 16 bytes >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ae70708e34c5 >> https://bugs.openjdk.java.net/browse/JDK-8042281 >> >> Next 2 fixes were pushed into jdk9 7 days ago: >> >> 8041959: Skip replay parsing errors with ReplayIgnoreInitErrors >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/4f0ae81062fd >> https://bugs.openjdk.java.net/browse/JDK-8041959 >> >> 8041957: -XX:UseAVX=0 cause assert(UseAVX) failed >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/9883c95f17f6 >> https://bugs.openjdk.java.net/browse/JDK-8041957 >> >> Next fix was pushed into jdk9 4 days ago: >> >> 8039298: assert(base == NULL || t_adr->isa_rawptr() || !phase->type(base)->higher_equal(TypePtr::NULL_PTR)) >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/bd1a985905ed >> https://bugs.openjdk.java.net/browse/JDK-8039298 >> >> Thanks, >> Vladimir >> > From albert.noll at oracle.com Wed May 7 07:17:19 2014 From: albert.noll at oracle.com (Albert) Date: Wed, 07 May 2014 09:17:19 +0200 Subject: [8u20] RFR(S): 8040085: dtrace/jsdt tests crash on solaris. found an unadvertised bad scavengable oop in the code cache In-Reply-To: <53688489.8020502@oracle.com> References: <536878B7.2020001@oracle.com> <53688228.5060403@oracle.com> <53688489.8020502@oracle.com> Message-ID: <5369DDFF.8050400@oracle.com> Can I push this? I am sometimes unsure if there is only 1 approval. Thanks, Albert On 05/06/2014 08:43 AM, Albert wrote: > Thank you, Vladimir. > > Best, > Albert > > On 05/06/2014 08:33 AM, Vladimir Kozlov wrote: >> Looks fine. >> >> Vladimir >> >> On 5/5/14 10:52 PM, Albert wrote: >>> Hi, >>> >>> I would like to request a 8u20 backport of JDK-8040085. >>> >>> The changes were pushed to 9 on 2014-04-28 and nightly testing >>> showed no problems. >>> The changes applied to 8u20 without conflicts. >>> >>> Master bug: https://bugs.openjdk.java.net/browse/JDK-8040085 >>> Webrev: http://cr.openjdk.java.net/~anoll/8040085/webrev.02/ >>> Changeset: >>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ccbe4f8119b6 >>> >>> Thanks, >>> Albert > From goetz.lindenmaier at sap.com Wed May 7 07:29:54 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 May 2014 07:29:54 +0000 Subject: RFR(M): 8042309: Some bugfixes for the ppc64 port. In-Reply-To: <53693BAD.8080707@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEBFC0A@DEWDFEMB12A.global.corp.sap> <53693011.2080100@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEBFF70@DEWDFEMB12A.global.corp.sap> <53693BAD.8080707@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC0078@DEWDFEMB12A.global.corp.sap> Thanks for pushing this, Vladimir! Best regards, Goetz -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Tuesday, May 06, 2014 9:45 PM To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net'; 'ppc-aix-port-dev at openjdk.java.net' Subject: Re: RFR(M): 8042309: Some bugfixes for the ppc64 port. Good. The push job is in JPRT. Thanks, Vladimir On 5/6/14 12:22 PM, Lindenmaier, Goetz wrote: > Hi Vladimir, > > I removed the dead variable and updated the webrev. > I also removed a debug print. > > In the expanded nodes we need mach rules so they get > the poper funcionaltiy generated by adlc, but we don't > want them to match, as they not fully implement the > required functionality. We use that quite often. > > Thanks for the review, > Goetz. > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Tuesday, May 06, 2014 8:55 PM > To: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net; 'ppc-aix-port-dev at openjdk.java.net' > Subject: Re: RFR(M): 8042309: Some bugfixes for the ppc64 port. > > Looks fine to me. Only minor issues in ppc.ad: > New vtable_index is not used. > It is strange to see predicate(false) in expanded instructions. > > Thanks, > Vladimir > > On 5/6/14 3:17 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> please review this change. I also please need a sponsor. >> It should go to jdk8u20 at some point. It only touches files >> needed for the ppc64 port. >> http://cr.openjdk.java.net/~goetz/webrevs/8042309/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8042309 >> >> This change fixes a few bugs in the ppc64 port: >> >> * Expand of max/min nodes in .ad file relied on 64-bit sign extend >> but placed the values in 32-bit registers. After SpillCopies >> the upper bits were lost. >> * Expand of decodeNKlass used wrong node. >> * Implement missing support for method handles in generate_throw_exception. >> * Some small optimizations and cleanups. >> >> Thanks and best regards, >> Goetz. >> From albert.noll at oracle.com Wed May 7 11:34:47 2014 From: albert.noll at oracle.com (Albert) Date: Wed, 07 May 2014 13:34:47 +0200 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 Message-ID: <536A1A57.4090707@oracle.com> Hi, could I get reviews for this patch? Bug: https://bugs.openjdk.java.net/browse/JDK-8042570 Problem: The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) prior to accessing the compilation queues of C1 and C2 in CompileBroker::mark_on_stack(). CompileBroker::mark_on_stack() is executed at a safepoint. The locks that I introduced can cause the deadlock, since the MethodCompileQueue_lock lock is held at a safepoint that is reached from ciEnv::register_method(). This causes the deadlock. I verified that deadlock happens as described above using the Redefine_v52_strict_invoke_redefine test. Solution: Remove locks in CompileBroker::mark_on_stack(). Testing: Failing test cases. webrev: http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ Many thanks in advance and sorry having introduced this bug, Albert From zhengyu.gu at oracle.com Wed May 7 12:59:17 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Wed, 07 May 2014 08:59:17 -0400 Subject: Update: RFR(XS) 8041415: remove port.{cpp,hpp} files In-Reply-To: <53694A18.2000902@oracle.com> References: <5367F2D5.4080606@oracle.com> <5367F65A.6090906@oracle.com> <53693BA8.50109@oracle.com> <53693CF1.9010506@oracle.com> <53694732.5090303@oracle.com> <53694A18.2000902@oracle.com> Message-ID: <536A2E25.5050105@oracle.com> Thanks for the review. -Zhengyu On 5/6/2014 4:46 PM, Vladimir Kozlov wrote: > This looks good. > > Thanks, > Vladimir > > On 5/6/14 1:33 PM, Zhengyu Gu wrote: >> Sorry, missed this one. >> >> One more try: http://cr.openjdk.java.net/~zgu/8041415/webrev.02/ >> >> >> Thanks, >> >> -Zhengyu >> >> On 5/6/2014 3:50 PM, Vladimir Kozlov wrote: >>> output.cpp change still use (uint) -1 >>> >>> Vladimir >>> >>> On 5/6/14 12:44 PM, Zhengyu Gu wrote: >>>> Hi, >>>> >>>> I updated the webrev to reflect Lois' change and Vladimir's comments. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8041415/webrev.01/ >>>> >>>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>> On 5/5/2014 4:36 PM, Vladimir Kozlov wrote: >>>>> CC to compiler alias since you changing our code. >>>>> >>>>> Copyright year change should be 2014. >>>>> >>>>> FileBuffRegion removal is done under separate bug 8041620 by Lois >>>>> Foltan, the review was sent today today. >>>>> >>>>> domgraph.cpp, output.cpp: use max_juint instead of (uint) -1. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 5/5/14 1:21 PM, Zhengyu Gu wrote: >>>>>> This is a code cleanup based on attached patch from Christian >>>>>> Thalinger. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>>>>> Webrev: >>>>> >>>>> http://cr.openjdk.java.net/~zgu/8041415/webrev.00/ >>>>> >>>>>> >>>>>> Test: >>>>>> JPRT >>>>>> >>>>>> Thanks, >>>>>> >>>> >> From zhengyu.gu at oracle.com Wed May 7 13:00:06 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Wed, 07 May 2014 09:00:06 -0400 Subject: Update: RFR(XS) 8041415: remove port.{cpp,hpp} files In-Reply-To: <53694A6A.8090808@oracle.com> References: <5367F2D5.4080606@oracle.com> <5367F65A.6090906@oracle.com> <53693BA8.50109@oracle.com> <53693CF1.9010506@oracle.com> <53694732.5090303@oracle.com> <53694A6A.8090808@oracle.com> Message-ID: <536A2E56.5050309@oracle.com> Thanks for the review. Vladimir Kozlov also reviewed. -Zhengyu On 5/6/2014 4:47 PM, Coleen Phillimore wrote: > > This looks good to me, but I think you need someone from the compiler > group to review also because it's all in the compiler code. > > Thanks! > Coleen > > On 5/6/14, 4:33 PM, Zhengyu Gu wrote: >> Sorry, missed this one. >> >> One more try: http://cr.openjdk.java.net/~zgu/8041415/webrev.02/ >> >> >> Thanks, >> >> -Zhengyu >> >> On 5/6/2014 3:50 PM, Vladimir Kozlov wrote: >>> output.cpp change still use (uint) -1 >>> >>> Vladimir >>> >>> On 5/6/14 12:44 PM, Zhengyu Gu wrote: >>>> Hi, >>>> >>>> I updated the webrev to reflect Lois' change and Vladimir's comments. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8041415/webrev.01/ >>>> >>>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>> On 5/5/2014 4:36 PM, Vladimir Kozlov wrote: >>>>> CC to compiler alias since you changing our code. >>>>> >>>>> Copyright year change should be 2014. >>>>> >>>>> FileBuffRegion removal is done under separate bug 8041620 by Lois >>>>> Foltan, the review was sent today today. >>>>> >>>>> domgraph.cpp, output.cpp: use max_juint instead of (uint) -1. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 5/5/14 1:21 PM, Zhengyu Gu wrote: >>>>>> This is a code cleanup based on attached patch from Christian >>>>>> Thalinger. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8041415 >>>>>> Webrev: >>>>> >>>>> http://cr.openjdk.java.net/~zgu/8041415/webrev.00/ >>>>> >>>>>> >>>>>> Test: >>>>>> JPRT >>>>>> >>>>>> Thanks, >>>>>> >>>> >> > From daniel.daugherty at oracle.com Wed May 7 13:46:24 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 07 May 2014 07:46:24 -0600 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 In-Reply-To: <536A1A57.4090707@oracle.com> References: <536A1A57.4090707@oracle.com> Message-ID: <536A3930.50300@oracle.com> > http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ src/share/vm/compiler/compileBroker.cpp No comments. Thumbs up! Dan On 5/7/14 5:34 AM, Albert wrote: > Hi, > > could I get reviews for this patch? > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8042570 > > Problem: > The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) > prior to accessing the compilation queues > of C1 and C2 in CompileBroker::mark_on_stack(). > CompileBroker::mark_on_stack() is executed at a safepoint. > The locks that I introduced can cause the deadlock, since the > MethodCompileQueue_lock lock is held at a safepoint > that is reached from ciEnv::register_method(). This causes the deadlock. > > I verified that deadlock happens as described above using the > Redefine_v52_strict_invoke_redefine test. > > Solution: > Remove locks in CompileBroker::mark_on_stack(). > > Testing: > Failing test cases. > > webrev: > http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ > > Many thanks in advance and sorry having introduced this bug, > Albert From albert.noll at oracle.com Wed May 7 14:46:28 2014 From: albert.noll at oracle.com (Albert) Date: Wed, 07 May 2014 16:46:28 +0200 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 In-Reply-To: <536A3930.50300@oracle.com> References: <536A1A57.4090707@oracle.com> <536A3930.50300@oracle.com> Message-ID: <536A4744.4050205@oracle.com> Dan, thanks for the review. Best, Albert On 05/07/2014 03:46 PM, Daniel D. Daugherty wrote: > > http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ > > src/share/vm/compiler/compileBroker.cpp > No comments. > > Thumbs up! > > Dan > > > On 5/7/14 5:34 AM, Albert wrote: >> Hi, >> >> could I get reviews for this patch? >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8042570 >> >> Problem: >> The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) >> prior to accessing the compilation queues >> of C1 and C2 in CompileBroker::mark_on_stack(). >> CompileBroker::mark_on_stack() is executed at a safepoint. >> The locks that I introduced can cause the deadlock, since the >> MethodCompileQueue_lock lock is held at a safepoint >> that is reached from ciEnv::register_method(). This causes the deadlock. >> >> I verified that deadlock happens as described above using the >> Redefine_v52_strict_invoke_redefine test. >> >> Solution: >> Remove locks in CompileBroker::mark_on_stack(). >> >> Testing: >> Failing test cases. >> >> webrev: >> http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ >> >> Many thanks in advance and sorry having introduced this bug, >> Albert > From vladimir.kozlov at oracle.com Wed May 7 15:59:43 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 07 May 2014 08:59:43 -0700 Subject: [8u20] RFR(S): 8040085: dtrace/jsdt tests crash on solaris. found an unadvertised bad scavengable oop in the code cache In-Reply-To: <5369DDFF.8050400@oracle.com> References: <536878B7.2020001@oracle.com> <53688228.5060403@oracle.com> <53688489.8020502@oracle.com> <5369DDFF.8050400@oracle.com> Message-ID: <536A586F.6010400@oracle.com> Yes, you can push it. Vladimir On 5/7/14 12:17 AM, Albert wrote: > Can I push this? I am sometimes unsure if there is only 1 approval. > > Thanks, > Albert > > On 05/06/2014 08:43 AM, Albert wrote: >> Thank you, Vladimir. >> >> Best, >> Albert >> >> On 05/06/2014 08:33 AM, Vladimir Kozlov wrote: >>> Looks fine. >>> >>> Vladimir >>> >>> On 5/5/14 10:52 PM, Albert wrote: >>>> Hi, >>>> >>>> I would like to request a 8u20 backport of JDK-8040085. >>>> >>>> The changes were pushed to 9 on 2014-04-28 and nightly testing showed no problems. >>>> The changes applied to 8u20 without conflicts. >>>> >>>> Master bug: https://bugs.openjdk.java.net/browse/JDK-8040085 >>>> Webrev: http://cr.openjdk.java.net/~anoll/8040085/webrev.02/ >>>> Changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/ccbe4f8119b6 >>>> >>>> Thanks, >>>> Albert >> > From vladimir.kozlov at oracle.com Wed May 7 16:02:30 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 07 May 2014 09:02:30 -0700 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 In-Reply-To: <536A1A57.4090707@oracle.com> References: <536A1A57.4090707@oracle.com> Message-ID: <536A5916.60204@oracle.com> Looks good. Thanks, Vladimir On 5/7/14 4:34 AM, Albert wrote: > Hi, > > could I get reviews for this patch? > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8042570 > > Problem: > The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) prior to accessing the compilation queues > of C1 and C2 in CompileBroker::mark_on_stack(). CompileBroker::mark_on_stack() is executed at a safepoint. > The locks that I introduced can cause the deadlock, since the MethodCompileQueue_lock lock is held at a safepoint > that is reached from ciEnv::register_method(). This causes the deadlock. > > I verified that deadlock happens as described above using the Redefine_v52_strict_invoke_redefine test. > > Solution: > Remove locks in CompileBroker::mark_on_stack(). > > Testing: > Failing test cases. > > webrev: > http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ > > Many thanks in advance and sorry having introduced this bug, > Albert From ebourg at apache.org Wed May 7 07:36:42 2014 From: ebourg at apache.org (Emmanuel Bourg) Date: Wed, 07 May 2014 09:36:42 +0200 Subject: Hotspot build broken with GNU make 4.0 Message-ID: <5369E28A.1010301@apache.org> Hi, GNU make has been upgraded to the version 4.0 in Debian and it broke the hotspot build. The issue is caused by the adjust-mflags.sh script in hotspot/make/linux/makefiles, the sed regexp corrupts the first argument containing the character 'j'. For example invoking this: sh ./hotspot/make/linux/makefiles/adjust-mflags.sh '-rRs -I/home/ebourg/openjdk8/make/common' 2 generates: -rRs -I/home/ebourg/open -j2 -dk8/make/common make 4.0 complains about the invalid parameters and returns an error. I guess make 3.x is more lenient and ignores unknown the parameters. Emmanuel Bourg From fuudtorrentsru at gmail.com Wed May 7 17:38:37 2014 From: fuudtorrentsru at gmail.com (Fedor Bobin) Date: Wed, 7 May 2014 21:38:37 +0400 Subject: JDK-4834738 NullPointerException: Better info In-Reply-To: <534CE15A.9030909@oracle.com> References: <534CD269.2060900@oracle.com> <534CE15A.9030909@oracle.com> Message-ID: Hello. Any review? Currently I work on adding such functionality to compiled code. I understand how to obtain method name and signature (for NPE inside compiled method). You can see my solution in this gist https://gist.github.com/Fuud/7b401c0a408c82f09c45 It prints method info to stdout. But I do not understand how to pass this information to implicit exception handler. Any suggestions? Thanks. Fedor Bobin. 2014-04-15 11:35 GMT+04:00 David Holmes : > On 15/04/2014 5:16 PM, Fedor Bobin wrote: > >> Hello. >> >> What is the best way to submit patch? >> Currently you can see it at https://gist.github.com/Fuud/10709048 >> > > If you become a regular contributor to OpenJDK you can get access to > cr.openjdk.java.net to upload patches. Otherwise inline attachments or a > link to something like github is okay. > > > Can you explain what information you add? >>>> >>> >> This java code will produce NPE; >> >> ((String)null).length(); >> >> Before patch exception stack trace looks like this: >> >> java.lang.NullPointerException >> at Test.testNPE(Test.java:14) >> at Test.main(Test.java:8) >> >> After patch if code is executing by interpreter NPE will have additional >> information >> >> java.lang.NullPointerException: can not invoke method >> java.lang.String.length()I on null object >> at Test.testNPE(Test.java:14) >> at Test.main(Test.java:8) >> >> This additional information can help to understand cause of NPE in case >> of call chains. For example, let we have java code >> >> a.b().c(); >> >> if we get NPE without message we can not understand is a==null or >> a.b()==null. After patch we will get enough information: >> >> java.lang.NullPointerException: can not invoke method A.b() on null >> object >> >> So we can understand that a is null. >> > > There have been a couple of RFEs on this in the past, one very recently, > and they have been closed as "will not fix" - mainly because except in some > limited circumstances the VM does not have much additional information to > add. That said your patch is the "inverse" of the other requests - they > wanted to know which "variable" was null, whereas you are showing which > method invocation was attempted on a null reference. > > Hopefully some folks from the runtime team will be able to take a look at > this and evaluate it. But it would be nice to be able to get similar > information from compiled code too - though that may not be feasible. > > Thanks, > David > > >> Thanks. >> Fedor Bobin >> >> >> >> 2014-04-15 10:32 GMT+04:00 David Holmes > >: >> >> >> Hi Fedor, >> >> >> On 14/04/2014 1:09 AM, Fedor Bobin wrote: >> >> Can you please review patch for JDK-4834738. >> >> >> Patches are stripped when sent as attachments. >> >> >> Currently I added more information to NPE if it is handled by >> interpreter >> or template interpreter on x86-64. On other platforms and after >> compilation >> jdk will handle NPE as usual. >> >> >> Can you explain what information you add? >> >> Thanks, >> David >> >> >> From henry.jen at oracle.com Wed May 7 17:59:00 2014 From: henry.jen at oracle.com (Henry Jen) Date: Wed, 07 May 2014 10:59:00 -0700 Subject: Hotspot build broken with GNU make 4.0 In-Reply-To: <5369E28A.1010301@apache.org> References: <5369E28A.1010301@apache.org> Message-ID: <536A7464.3090404@oracle.com> I think you hit on this bug, https://bugs.openjdk.java.net/browse/JDK-8028407 Cheers, Henry On 05/07/2014 12:36 AM, Emmanuel Bourg wrote: > Hi, > > GNU make has been upgraded to the version 4.0 in Debian and it broke the > hotspot build. > > The issue is caused by the adjust-mflags.sh script in > hotspot/make/linux/makefiles, the sed regexp corrupts the first argument > containing the character 'j'. > > For example invoking this: > > sh ./hotspot/make/linux/makefiles/adjust-mflags.sh '-rRs > -I/home/ebourg/openjdk8/make/common' 2 > > generates: > > -rRs -I/home/ebourg/open -j2 -dk8/make/common > > make 4.0 complains about the invalid parameters and returns an error. I > guess make 3.x is more lenient and ignores unknown the parameters. > > Emmanuel Bourg > From david.r.chase at oracle.com Wed May 7 18:35:06 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 7 May 2014 14:35:06 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <5361C68E.9040002@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> Message-ID: <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> Re-revised patch. Main changes from previous: 1) 'print_raw(' introductions replaced with 'print("%s", ' with the exception that the autogenerated code in adlc still uses print_raw because that still seemed slightly less annoying than dealing with the quoted and unquoted format strings appearing in the same fprintf format. Print_raw was generally judged to be kinda ugly, and it also turned out that there were other stream-like classes with no "raw" equivalent so the "%s", idiom had to be used elsewhere anyway. I didn't replace cr() with print_cr("%s", "") because there were opposing opinions (and not just mine) as to which choice was uglier. 2) PRIxPTR introductions replaced with INTPTR_FORMAT. 3) PTR_FORMAT introductions replaced with INTPTR_FORMAT -- though they are the same, we are printing intptr_t, so the format should be INTPTR_FORMAT. Note that neither PRIxPTR nor PTR_FORMAT are banished -- I merely did not make the confusion worse. If we want to normalize those out altogether, that is another cleanup. 4) I followed Henry Jen's suggestions for sprinkling in more format attributes to reduce the need for disabled checking, and removed the check-disables that were no longer necessary (and I think I got them all). 5) To help ensure against future portability bit-rot, I tracked down more functions/methods that were eligible for format attributes. This lead to another round of format-use corrections under both gcc and clang (different sets of corrections, I took their union, obviously). 6) I also prepared a backport to 8u to look for any glitches there (there were glitches) and see if they had any influence on the patches to 9 (they didn't). Webrevs: jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-07a/ jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-07b/ Testing: Built "by hand" fastdebug on Mountain Lion + XCode 4.6.3 Mavericks + XCode 5.1.1 Ubuntu 32-bit + gcc 4.8 JPRT of hotspot building on all platforms. JTREG of hotspot/{compiler,gc,runtime} on Mountain Lion 8u has been build-tested on Mountain Lion and Ubuntu-32-gcc-4.8 -- out-of-the-box new build still fails to go on Mavericks, though I understand it can be persuaded with enough flags. I'm told that a patch of this size might need careful timing both because it splatters across all of hotspot, and because it will complicate any backports of bug fixes to 8, which is why I prepared a backport patch in addition to the regular one, both to accelerate that process when we need it to happen and to give a feel for what the patch would look like and touch. From albert.noll at oracle.com Wed May 7 20:01:41 2014 From: albert.noll at oracle.com (Albert) Date: Wed, 07 May 2014 22:01:41 +0200 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 In-Reply-To: <536A5916.60204@oracle.com> References: <536A1A57.4090707@oracle.com> <536A5916.60204@oracle.com> Message-ID: <536A9125.3080107@oracle.com> Thank you for the review, Vladimir. Best, Albert On 05/07/2014 06:02 PM, Vladimir Kozlov wrote: > Looks good. > > Thanks, > Vladimir > > On 5/7/14 4:34 AM, Albert wrote: >> Hi, >> >> could I get reviews for this patch? >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8042570 >> >> Problem: >> The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) >> prior to accessing the compilation queues >> of C1 and C2 in CompileBroker::mark_on_stack(). >> CompileBroker::mark_on_stack() is executed at a safepoint. >> The locks that I introduced can cause the deadlock, since the >> MethodCompileQueue_lock lock is held at a safepoint >> that is reached from ciEnv::register_method(). This causes the deadlock. >> >> I verified that deadlock happens as described above using the >> Redefine_v52_strict_invoke_redefine test. >> >> Solution: >> Remove locks in CompileBroker::mark_on_stack(). >> >> Testing: >> Failing test cases. >> >> webrev: >> http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ >> >> Many thanks in advance and sorry having introduced this bug, >> Albert From goetz.lindenmaier at sap.com Wed May 7 21:53:34 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 May 2014 21:53:34 +0000 Subject: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEBECE6@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEBECE6@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC0359@DEWDFEMB12A.global.corp.sap> Hi, could somebody please sponsor this change? Christian, Stefan, David, you appreciated this. Maybe one of you could find the time to sponsor it? Sorry for urging, there is no hurry, but I don't want it to be forgotten. Thanks and best regards, Goetz -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Freitag, 2. Mai 2014 23:42 To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' Subject: RE: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. Hi Vladimir, I fixed what you proposed except for precompiled.hpp: There always the .hpp and .inline.hpp are mentioned, so I'd like to do it for this file the same way. Webrev is updated: http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ No problem with closed sources, there is no hurry with this one. Thanks and best regards, Goetz Vladimir, sorry you get this twice, I forgot the mailing list first. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Kozlov Sent: Friday, May 02, 2014 9:36 PM To: hotspot-dev at openjdk.java.net Subject: Re: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. Hi Goetz, In general it is good but sponsor from Oracle have to check and fix our closed sources too. I see in metaspace.cpp, precompiled.hpp you kept orderAccess.hpp include which is not needed: # include "runtime/orderAccess.hpp" +# include "runtime/orderAccess.inline.hpp" Typo in oops/klass.hpp?: - // Ordered list of all primary supertypes + // Orderd list of all primary supertypes In javaFrameAnchor.hpp preserve original order - globalDefinitions.hpp first. Thanks, Vladimir On 5/2/14 5:23 AM, Lindenmaier, Goetz wrote: > Hi, > > Please review and test this change. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8042195 > > This change introduces an umbrella header for orderAccess__.inline.hpp > files: runtime/orderAccess.inline.hpp. This follows the scheme applied to > atomic.inline.hpp. > > Further this change adds includes of orderAccess.inline.hpp in all .cpp > and .inline.hpp files where a method of OrderAccess declared 'inline' is > called. > > Finally it moves methods calling inline methods of OrderAccess from > g1CollectedHeap.hpp and thread.hpp to the corresponding .inline.hpp > files and adds the necessary includes in files using the moved > functions. > > Still a lot of calls to inline methods of OrderAccess in .hpp > files arn't preceded by an according include. But for these > files an appropriate .inline.hpp file is missing: > > src/cpu/ppc/vm/javaFrameAnchor_ppc.hpp > src/cpu/sparc/vm/javaFrameAnchor_sparc.hpp > src/share/vm/classfile/classLoader.hpp > src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp > src/share/vm/memory/cardTableModRefBS.hpp > src/share/vm/oops/constantPool.hpp > src/share/vm/oops/cpCache.hpp > src/share/vm/oops/instanceKlass.hpp > src/share/vm/oops/method.hpp > src/share/vm/oops/methodData.hpp > src/share/vm/oops/typeArrayOop.hpp > src/share/vm/runtime/interfaceSupport.hpp > src/share/vm/utilities/array.hpp > src/share/vm/utilities/taskqueue.hpp > > Testing: > I ran builds on linuxx86_64, sun_32, sun_64, mac, windowsx86_64, linuxppc64 and aix; > dbg, fastdbg and product without precompiled headers. > > Thanks and best regards, > Goetz > From vladimir.kozlov at oracle.com Wed May 7 22:38:40 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 07 May 2014 15:38:40 -0700 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> Message-ID: <536AB5F0.2080706@oracle.com> On 5/7/14 11:35 AM, David Chase wrote: > Re-revised patch. > > Main changes from previous: > > 1) 'print_raw(' introductions replaced with 'print("%s", ' with the exception that the autogenerated code in adlc still uses print_raw because that still seemed slightly less annoying than dealing with the quoted and unquoted format strings appearing in the same fprintf format. Print_raw was generally judged to be kinda ugly, and it also turned out that there were other stream-like classes with no "raw" equivalent so the "%s", idiom had to be used elsewhere anyway. I didn't replace cr() with print_cr("%s", "") because there were opposing opinions (and not just mine) as to which choice was uglier. I am one of who are against using print_cr("%s", "") instead of cr(). The cr() is well know and used method. And usually it is invoked as tty->cr() or st->cr() so its meaning is clear. I see several cases of new print_raw_cr(). Why not print_cr("%s", msg)? It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. In ciEnv.cpp it can be merged with previous line: tty->print("# Compiler replay data is saved as: %s", buffer); tty->print("# Compiler inline data is saved as: %s", buffer); In ageTable.cpp why you changed type of 'age' from uint to uintx? The used format for it is "%3u". Did you changes some spacing in generateOopMap.cpp? I see only Copyright year update. In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. Thanks, Vladimir > > 2) PRIxPTR introductions replaced with INTPTR_FORMAT. > > 3) PTR_FORMAT introductions replaced with INTPTR_FORMAT -- though they are the same, we are printing intptr_t, so the format should be INTPTR_FORMAT. > > Note that neither PRIxPTR nor PTR_FORMAT are banished -- I merely did not make the confusion worse. > If we want to normalize those out altogether, that is another cleanup. > > 4) I followed Henry Jen's suggestions for sprinkling in more format attributes to reduce the need for disabled checking, and removed the check-disables that were no longer necessary (and I think I got them all). > > 5) To help ensure against future portability bit-rot, I tracked down more functions/methods that were eligible for format attributes. This lead to another round of format-use corrections under both gcc and clang (different sets of corrections, I took their union, obviously). > > 6) I also prepared a backport to 8u to look for any glitches there (there were glitches) and see if they had any influence on the patches to 9 (they didn't). > > Webrevs: > jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-07a/ > jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-07b/ > > Testing: > Built "by hand" fastdebug on > Mountain Lion + XCode 4.6.3 > Mavericks + XCode 5.1.1 > Ubuntu 32-bit + gcc 4.8 > > JPRT of hotspot building on all platforms. > > JTREG of hotspot/{compiler,gc,runtime} on Mountain Lion > > 8u has been build-tested on Mountain Lion and Ubuntu-32-gcc-4.8 -- out-of-the-box new build still fails to go on Mavericks, though I understand it can be persuaded with enough flags. > > I'm told that a patch of this size might need careful timing both because it splatters across all of hotspot, and because it will complicate any backports of bug fixes to 8, which is why I prepared a backport patch in addition to the regular one, both to accelerate that process when we need it to happen and to give a feel for what the patch would look like and touch. > From vladimir.kozlov at oracle.com Wed May 7 23:28:54 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 07 May 2014 16:28:54 -0700 Subject: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEC0359@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEBECE6@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC0359@DEWDFEMB12A.global.corp.sap> Message-ID: <536AC1B6.4010300@oracle.com> I sent RFR for closed changes and I am testing it in JPRT. Vladimir On 5/7/14 2:53 PM, Lindenmaier, Goetz wrote: > Hi, > > could somebody please sponsor this change? > Christian, Stefan, David, you appreciated this. Maybe one of you > could find the time to sponsor it? > > Sorry for urging, there is no hurry, but I don't want it to be forgotten. > > Thanks and best regards, > Goetz > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz > Sent: Freitag, 2. Mai 2014 23:42 > To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' > Subject: RE: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. > > Hi Vladimir, > > I fixed what you proposed except for precompiled.hpp: > There always the .hpp and .inline.hpp are mentioned, so I'd like to > do it for this file the same way. > > Webrev is updated: > http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ > > No problem with closed sources, there is no hurry with this one. > > Thanks and best regards, > Goetz > > > Vladimir, sorry you get this twice, I forgot the mailing list first. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Kozlov > Sent: Friday, May 02, 2014 9:36 PM > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. > > Hi Goetz, > > In general it is good but sponsor from Oracle have to check and fix our > closed sources too. > > I see in metaspace.cpp, precompiled.hpp you kept orderAccess.hpp include > which is not needed: > > # include "runtime/orderAccess.hpp" > +# include "runtime/orderAccess.inline.hpp" > > Typo in oops/klass.hpp?: > > - // Ordered list of all primary supertypes > + // Orderd list of all primary supertypes > > In javaFrameAnchor.hpp preserve original order - globalDefinitions.hpp > first. > > Thanks, > Vladimir > > On 5/2/14 5:23 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> Please review and test this change. I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8042195 >> >> This change introduces an umbrella header for orderAccess__.inline.hpp >> files: runtime/orderAccess.inline.hpp. This follows the scheme applied to >> atomic.inline.hpp. >> >> Further this change adds includes of orderAccess.inline.hpp in all .cpp >> and .inline.hpp files where a method of OrderAccess declared 'inline' is >> called. >> >> Finally it moves methods calling inline methods of OrderAccess from >> g1CollectedHeap.hpp and thread.hpp to the corresponding .inline.hpp >> files and adds the necessary includes in files using the moved >> functions. >> >> Still a lot of calls to inline methods of OrderAccess in .hpp >> files arn't preceded by an according include. But for these >> files an appropriate .inline.hpp file is missing: >> >> src/cpu/ppc/vm/javaFrameAnchor_ppc.hpp >> src/cpu/sparc/vm/javaFrameAnchor_sparc.hpp >> src/share/vm/classfile/classLoader.hpp >> src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp >> src/share/vm/memory/cardTableModRefBS.hpp >> src/share/vm/oops/constantPool.hpp >> src/share/vm/oops/cpCache.hpp >> src/share/vm/oops/instanceKlass.hpp >> src/share/vm/oops/method.hpp >> src/share/vm/oops/methodData.hpp >> src/share/vm/oops/typeArrayOop.hpp >> src/share/vm/runtime/interfaceSupport.hpp >> src/share/vm/utilities/array.hpp >> src/share/vm/utilities/taskqueue.hpp >> >> Testing: >> I ran builds on linuxx86_64, sun_32, sun_64, mac, windowsx86_64, linuxppc64 and aix; >> dbg, fastdbg and product without precompiled headers. >> >> Thanks and best regards, >> Goetz >> From coleen.phillimore at oracle.com Wed May 7 23:59:20 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 07 May 2014 19:59:20 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536AB5F0.2080706@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: <536AC8D8.6060804@oracle.com> On 5/7/14, 6:38 PM, Vladimir Kozlov wrote: > On 5/7/14 11:35 AM, David Chase wrote: >> Re-revised patch. >> >> Main changes from previous: >> >> 1) 'print_raw(' introductions replaced with 'print("%s", ' with the >> exception that the autogenerated code in adlc still uses print_raw >> because that still seemed slightly less annoying than dealing with >> the quoted and unquoted format strings appearing in the same fprintf >> format. Print_raw was generally judged to be kinda ugly, and it also >> turned out that there were other stream-like classes with no "raw" >> equivalent so the "%s", idiom had to be used elsewhere anyway. I >> didn't replace cr() with print_cr("%s", "") because there were >> opposing opinions (and not just mine) as to which choice was uglier. > > I am one of who are against using print_cr("%s", "") instead of cr(). > The cr() is well know and used method. And usually it is invoked as > tty->cr() or st->cr() so its meaning is clear. Me too. So this has been a lot of work and a lot of information over email. I had one question - why isn't there a FORMAT type that can be used for void* ? Using these p2i functions is going to get really annoying. Does this also address bug https://bugs.openjdk.java.net/browse/JDK-8029996 ? thanks, Coleen > > > I see several cases of new print_raw_cr(). Why not print_cr("%s", > msg)? It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, > compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, > metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, > instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, > os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, > diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. > > In ciEnv.cpp it can be merged with previous line: > > tty->print("# Compiler replay data is saved as: %s", buffer); > tty->print("# Compiler inline data is saved as: %s", buffer); > > In ageTable.cpp why you changed type of 'age' from uint to uintx? The > used format for it is "%3u". > > Did you changes some spacing in generateOopMap.cpp? I see only > Copyright year update. > > In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. > > Thanks, > Vladimir > >> >> 2) PRIxPTR introductions replaced with INTPTR_FORMAT. >> >> 3) PTR_FORMAT introductions replaced with INTPTR_FORMAT -- though >> they are the same, we are printing intptr_t, so the format should be >> INTPTR_FORMAT. >> >> Note that neither PRIxPTR nor PTR_FORMAT are banished -- I merely did >> not make the confusion worse. >> If we want to normalize those out altogether, that is another cleanup. >> >> 4) I followed Henry Jen's suggestions for sprinkling in more format >> attributes to reduce the need for disabled checking, and removed the >> check-disables that were no longer necessary (and I think I got them >> all). >> >> 5) To help ensure against future portability bit-rot, I tracked down >> more functions/methods that were eligible for format attributes. >> This lead to another round of format-use corrections under both gcc >> and clang (different sets of corrections, I took their union, >> obviously). >> >> 6) I also prepared a backport to 8u to look for any glitches there >> (there were glitches) and see if they had any influence on the >> patches to 9 (they didn't). >> >> Webrevs: >> jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-07a/ >> jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-07b/ >> >> Testing: >> Built "by hand" fastdebug on >> Mountain Lion + XCode 4.6.3 >> Mavericks + XCode 5.1.1 >> Ubuntu 32-bit + gcc 4.8 >> >> JPRT of hotspot building on all platforms. >> >> JTREG of hotspot/{compiler,gc,runtime} on Mountain Lion >> >> 8u has been build-tested on Mountain Lion and Ubuntu-32-gcc-4.8 -- >> out-of-the-box new build still fails to go on Mavericks, though I >> understand it can be persuaded with enough flags. >> >> I'm told that a patch of this size might need careful timing both >> because it splatters across all of hotspot, and because it will >> complicate any backports of bug fixes to 8, which is why I prepared a >> backport patch in addition to the regular one, both to accelerate >> that process when we need it to happen and to give a feel for what >> the patch would look like and touch. >> From vladimir.kozlov at oracle.com Thu May 8 00:11:54 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 07 May 2014 17:11:54 -0700 Subject: RFR (M): 8042195: Introduce umbrella header orderAccess.inline.hpp. In-Reply-To: <536AC1B6.4010300@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEBECE6@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC0359@DEWDFEMB12A.global.corp.sap> <536AC1B6.4010300@oracle.com> Message-ID: <536ACBCA.2070205@oracle.com> Closed changes were reviewed and the testing almost done (no problems). I am pushing this into hs-comp. Thanks, Vladimir On 5/7/14 4:28 PM, Vladimir Kozlov wrote: > I sent RFR for closed changes and I am testing it in JPRT. > > Vladimir > > On 5/7/14 2:53 PM, Lindenmaier, Goetz wrote: >> Hi, >> >> could somebody please sponsor this change? >> Christian, Stefan, David, you appreciated this. Maybe one of you >> could find the time to sponsor it? >> >> Sorry for urging, there is no hurry, but I don't want it to be forgotten. >> >> Thanks and best regards, >> Goetz >> >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> Behalf Of Lindenmaier, Goetz >> Sent: Freitag, 2. Mai 2014 23:42 >> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >> Subject: RE: RFR (M): 8042195: Introduce umbrella header >> orderAccess.inline.hpp. >> >> Hi Vladimir, >> >> I fixed what you proposed except for precompiled.hpp: >> There always the .hpp and .inline.hpp are mentioned, so I'd like to >> do it for this file the same way. >> >> Webrev is updated: >> http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ >> >> No problem with closed sources, there is no hurry with this one. >> >> Thanks and best regards, >> Goetz >> >> >> Vladimir, sorry you get this twice, I forgot the mailing list first. >> >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> Behalf Of Vladimir Kozlov >> Sent: Friday, May 02, 2014 9:36 PM >> To: hotspot-dev at openjdk.java.net >> Subject: Re: RFR (M): 8042195: Introduce umbrella header >> orderAccess.inline.hpp. >> >> Hi Goetz, >> >> In general it is good but sponsor from Oracle have to check and fix our >> closed sources too. >> >> I see in metaspace.cpp, precompiled.hpp you kept orderAccess.hpp include >> which is not needed: >> >> # include "runtime/orderAccess.hpp" >> +# include "runtime/orderAccess.inline.hpp" >> >> Typo in oops/klass.hpp?: >> >> - // Ordered list of all primary supertypes >> + // Orderd list of all primary supertypes >> >> In javaFrameAnchor.hpp preserve original order - globalDefinitions.hpp >> first. >> >> Thanks, >> Vladimir >> >> On 5/2/14 5:23 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> Please review and test this change. I please need a sponsor. >>> http://cr.openjdk.java.net/~goetz/webrevs/8042195-oaIncl/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8042195 >>> >>> This change introduces an umbrella header for >>> orderAccess__.inline.hpp >>> files: runtime/orderAccess.inline.hpp. This follows the scheme >>> applied to >>> atomic.inline.hpp. >>> >>> Further this change adds includes of orderAccess.inline.hpp in all .cpp >>> and .inline.hpp files where a method of OrderAccess declared 'inline' is >>> called. >>> >>> Finally it moves methods calling inline methods of OrderAccess from >>> g1CollectedHeap.hpp and thread.hpp to the corresponding .inline.hpp >>> files and adds the necessary includes in files using the moved >>> functions. >>> >>> Still a lot of calls to inline methods of OrderAccess in .hpp >>> files arn't preceded by an according include. But for these >>> files an appropriate .inline.hpp file is missing: >>> >>> src/cpu/ppc/vm/javaFrameAnchor_ppc.hpp >>> src/cpu/sparc/vm/javaFrameAnchor_sparc.hpp >>> src/share/vm/classfile/classLoader.hpp >>> src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp >>> src/share/vm/memory/cardTableModRefBS.hpp >>> src/share/vm/oops/constantPool.hpp >>> src/share/vm/oops/cpCache.hpp >>> src/share/vm/oops/instanceKlass.hpp >>> src/share/vm/oops/method.hpp >>> src/share/vm/oops/methodData.hpp >>> src/share/vm/oops/typeArrayOop.hpp >>> src/share/vm/runtime/interfaceSupport.hpp >>> src/share/vm/utilities/array.hpp >>> src/share/vm/utilities/taskqueue.hpp >>> >>> Testing: >>> I ran builds on linuxx86_64, sun_32, sun_64, mac, windowsx86_64, >>> linuxppc64 and aix; >>> dbg, fastdbg and product without precompiled headers. >>> >>> Thanks and best regards, >>> Goetz >>> From david.r.chase at oracle.com Thu May 8 00:59:51 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 7 May 2014 20:59:51 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536AB5F0.2080706@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: <1E245D19-10C9-4B14-855D-2F4C6562790E@oracle.com> On 2014-05-07, at 6:38 PM, Vladimir Kozlov wrote: > I see several cases of new print_raw_cr(). Why not print_cr("%s", msg)? It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. Thank you, I missed print_raw_cr (because I was so thrilled at how efficiently I dispatched print_raw). > In ciEnv.cpp it can be merged with previous line: > > tty->print("# Compiler replay data is saved as: %s", buffer); > tty->print("# Compiler inline data is saved as: %s", buffer); > > In ageTable.cpp why you changed type of 'age' from uint to uintx? The used format for it is "%3u". I am not sure; I will check. It appeared from the repeated use of UINTX_FORMAT that uintx was intended, so I changed it. I can (will, unless someone objects) change that back to uint with consistent formats. > Did you changes some spacing in generateOopMap.cpp? I see only Copyright year update. This is probably an artifact of incorporating Henry's better use of annotations, which then caused the backout of some earlier changes. I wasn't supposed to have touched any copyrights yet; that I have an automated script to handle, along with trailing spaces. > In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. Will do. Thanks very much for your attention to detail on this. David From vladimir.kozlov at oracle.com Thu May 8 01:27:59 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 07 May 2014 18:27:59 -0700 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <1E245D19-10C9-4B14-855D-2F4C6562790E@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <1E245D19-10C9-4B14-855D-2F4C6562790E@oracle.com> Message-ID: <536ADD9F.1040403@oracle.com> On 5/7/14 5:59 PM, David Chase wrote: > > On 2014-05-07, at 6:38 PM, Vladimir Kozlov wrote: >> I see several cases of new print_raw_cr(). Why not print_cr("%s", msg)? It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. > > Thank you, I missed print_raw_cr (because I was so thrilled at how efficiently I dispatched print_raw). > >> In ciEnv.cpp it can be merged with previous line: >> >> tty->print("# Compiler replay data is saved as: %s", buffer); >> tty->print("# Compiler inline data is saved as: %s", buffer); Correction: the above should be print_cr(). The original code was: tty->print("# Compiler replay data is saved as: "); tty->print_cr(buffer); Thanks, Vladimir >> >> In ageTable.cpp why you changed type of 'age' from uint to uintx? The used format for it is "%3u". > > I am not sure; I will check. It appeared from the repeated use of UINTX_FORMAT that uintx was intended, > so I changed it. I can (will, unless someone objects) change that back to uint with consistent formats. It is printed only in one place and you kept %3u for it: ! gclog_or_tty->print_cr("- age %3u: " SIZE_FORMAT_W(10) " bytes, " SIZE_FORMAT_W(10) " total", age, sizes[age]*oopSize, total*oopSize); Note, age <= table_size and table_size is 16. Vladimir > >> Did you changes some spacing in generateOopMap.cpp? I see only Copyright year update. > > This is probably an artifact of incorporating Henry's better use of annotations, which then caused > the backout of some earlier changes. I wasn't supposed to have touched any copyrights yet; that > I have an automated script to handle, along with trailing spaces. > >> In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. > > Will do. > > Thanks very much for your attention to detail on this. > > David > From david.holmes at oracle.com Thu May 8 02:45:42 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 May 2014 12:45:42 +1000 Subject: Hotspot build broken with GNU make 4.0 In-Reply-To: <536A7464.3090404@oracle.com> References: <5369E28A.1010301@apache.org> <536A7464.3090404@oracle.com> Message-ID: <536AEFD6.7080208@oracle.com> On 8/05/2014 3:59 AM, Henry Jen wrote: > I think you hit on this bug, > > https://bugs.openjdk.java.net/browse/JDK-8028407 Which is fixed, so that begs the question as to which forest Emmanuel was using? David > Cheers, > Henry > > > On 05/07/2014 12:36 AM, Emmanuel Bourg wrote: >> Hi, >> >> GNU make has been upgraded to the version 4.0 in Debian and it broke the >> hotspot build. >> >> The issue is caused by the adjust-mflags.sh script in >> hotspot/make/linux/makefiles, the sed regexp corrupts the first argument >> containing the character 'j'. >> >> For example invoking this: >> >> sh ./hotspot/make/linux/makefiles/adjust-mflags.sh '-rRs >> -I/home/ebourg/openjdk8/make/common' 2 >> >> generates: >> >> -rRs -I/home/ebourg/open -j2 -dk8/make/common >> >> make 4.0 complains about the invalid parameters and returns an error. I >> guess make 3.x is more lenient and ignores unknown the parameters. >> >> Emmanuel Bourg >> From david.holmes at oracle.com Thu May 8 03:02:35 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 May 2014 13:02:35 +1000 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536AB5F0.2080706@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: <536AF3CB.60000@oracle.com> On 8/05/2014 8:38 AM, Vladimir Kozlov wrote: > On 5/7/14 11:35 AM, David Chase wrote: >> Re-revised patch. >> >> Main changes from previous: >> >> 1) 'print_raw(' introductions replaced with 'print("%s", ' with the >> exception that the autogenerated code in adlc still uses print_raw >> because that still seemed slightly less annoying than dealing with the >> quoted and unquoted format strings appearing in the same fprintf >> format. Print_raw was generally judged to be kinda ugly, and it also >> turned out that there were other stream-like classes with no "raw" >> equivalent so the "%s", idiom had to be used elsewhere anyway. I >> didn't replace cr() with print_cr("%s", "") because there were >> opposing opinions (and not just mine) as to which choice was uglier. > > I am one of who are against using print_cr("%s", "") instead of cr(). > The cr() is well know and used method. And usually it is invoked as > tty->cr() or st->cr() so its meaning is clear. The 70 occurrences of print_cr("") versus 800+ for cr() suggests it was not universally well known. I would have preferred it to have been called print_cr() but that ship sailed long ago. I certainly would never suggest using print_cr("%s", "") ! Cheers, David H. ------- > > I see several cases of new print_raw_cr(). Why not print_cr("%s", msg)? > It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, > compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, > metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, > instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, > os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, > diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. > > In ciEnv.cpp it can be merged with previous line: > > tty->print("# Compiler replay data is saved as: %s", buffer); > tty->print("# Compiler inline data is saved as: %s", buffer); > > In ageTable.cpp why you changed type of 'age' from uint to uintx? The > used format for it is "%3u". > > Did you changes some spacing in generateOopMap.cpp? I see only Copyright > year update. > > In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. > > Thanks, > Vladimir > >> >> 2) PRIxPTR introductions replaced with INTPTR_FORMAT. >> >> 3) PTR_FORMAT introductions replaced with INTPTR_FORMAT -- though they >> are the same, we are printing intptr_t, so the format should be >> INTPTR_FORMAT. >> >> Note that neither PRIxPTR nor PTR_FORMAT are banished -- I merely did >> not make the confusion worse. >> If we want to normalize those out altogether, that is another cleanup. >> >> 4) I followed Henry Jen's suggestions for sprinkling in more format >> attributes to reduce the need for disabled checking, and removed the >> check-disables that were no longer necessary (and I think I got them >> all). >> >> 5) To help ensure against future portability bit-rot, I tracked down >> more functions/methods that were eligible for format attributes. This >> lead to another round of format-use corrections under both gcc and >> clang (different sets of corrections, I took their union, obviously). >> >> 6) I also prepared a backport to 8u to look for any glitches there >> (there were glitches) and see if they had any influence on the patches >> to 9 (they didn't). >> >> Webrevs: >> jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-07a/ >> jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-07b/ >> >> Testing: >> Built "by hand" fastdebug on >> Mountain Lion + XCode 4.6.3 >> Mavericks + XCode 5.1.1 >> Ubuntu 32-bit + gcc 4.8 >> >> JPRT of hotspot building on all platforms. >> >> JTREG of hotspot/{compiler,gc,runtime} on Mountain Lion >> >> 8u has been build-tested on Mountain Lion and Ubuntu-32-gcc-4.8 -- >> out-of-the-box new build still fails to go on Mavericks, though I >> understand it can be persuaded with enough flags. >> >> I'm told that a patch of this size might need careful timing both >> because it splatters across all of hotspot, and because it will >> complicate any backports of bug fixes to 8, which is why I prepared a >> backport patch in addition to the regular one, both to accelerate that >> process when we need it to happen and to give a feel for what the >> patch would look like and touch. >> From david.holmes at oracle.com Thu May 8 03:05:16 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 May 2014 13:05:16 +1000 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 In-Reply-To: <536A1A57.4090707@oracle.com> References: <536A1A57.4090707@oracle.com> Message-ID: <536AF46C.7000507@oracle.com> On 7/05/2014 9:34 PM, Albert wrote: > Hi, > > could I get reviews for this patch? > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8042570 > > Problem: > The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) prior > to accessing the compilation queues > of C1 and C2 in CompileBroker::mark_on_stack(). > CompileBroker::mark_on_stack() is executed at a safepoint. > The locks that I introduced can cause the deadlock, since the > MethodCompileQueue_lock lock is held at a safepoint > that is reached from ciEnv::register_method(). This causes the deadlock. This begs the question why you thought locking was necessary then and you no longer think it necessary now? David ----- > I verified that deadlock happens as described above using the > Redefine_v52_strict_invoke_redefine test. > > Solution: > Remove locks in CompileBroker::mark_on_stack(). > > Testing: > Failing test cases. > > webrev: > http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ > > Many thanks in advance and sorry having introduced this bug, > Albert From david.r.chase at oracle.com Thu May 8 01:13:08 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 7 May 2014 21:13:08 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536AC8D8.6060804@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536AC8D8.6060804@oracle.com> Message-ID: <6CEDDC71-6274-409F-8AF0-F7712E40C87F@oracle.com> On 2014-05-07, at 7:59 PM, Coleen Phillimore wrote: > So this has been a lot of work and a lot of information over email. I had one question - why isn't there a FORMAT type that can be used for void* ? Using these p2i functions is going to get really annoying. What I understand, from reading stackoverflow and poking around, is that %p does not have a uniform behavior across platforms; some of them prefix a 0x, others do not. I think there is variance in how the width can be specified, not sure. If we can sort out the %p variation, perhaps by defining PTR_FORMAT to be either 0x%p or %p depending on the platform, and if we can sort out width issues also, then we could get rid of INTPTR_FORMAT (convert to PTR_FORMAT) and remove the p2i casts. > Does this also address bug https://bugs.openjdk.java.net/browse/JDK-8029996 ? Yes. However, it hides the attribute in a macro, since I am not sure whether the attribute is supported across all compilers (it is currently activated for gcc and clang) and there's one or two files where the formats are so borked that turning off the attribute macros before file inclusion (this will need fixing) From an earlier email (I should check if this got fixed recently by gc people): > For two files, > src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp > src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp > the macro ATTRIBUTE_PRINTF(x,y) > is defined to be empty above the header inclusions; the format strings are wrong in peculiar ways, and they are buried in macros. gotta go, town meeting accidentally got interesting. David From albert.noll at oracle.com Thu May 8 05:12:51 2014 From: albert.noll at oracle.com (Albert) Date: Thu, 08 May 2014 07:12:51 +0200 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 In-Reply-To: <536AF46C.7000507@oracle.com> References: <536A1A57.4090707@oracle.com> <536AF46C.7000507@oracle.com> Message-ID: <536B1253.2010204@oracle.com> On 05/08/2014 05:05 AM, David Holmes wrote: > On 7/05/2014 9:34 PM, Albert wrote: >> Hi, >> >> could I get reviews for this patch? >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8042570 >> >> Problem: >> The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) prior >> to accessing the compilation queues >> of C1 and C2 in CompileBroker::mark_on_stack(). >> CompileBroker::mark_on_stack() is executed at a safepoint. >> The locks that I introduced can cause the deadlock, since the >> MethodCompileQueue_lock lock is held at a safepoint >> that is reached from ciEnv::register_method(). This causes the deadlock. > > This begs the question why you thought locking was necessary then and > you no longer think it necessary now? > I introduced the lock in the first place, since compilation queues are accessed by multiple threads and CompileQueue::mark_on_stack() modifies elements of the queue without acquiring a lock. However, I did not realize that CompileQueue::mark_on_stack() is executed at a safepoint. Best, Albert > David > ----- > >> I verified that deadlock happens as described above using the >> Redefine_v52_strict_invoke_redefine test. >> >> Solution: >> Remove locks in CompileBroker::mark_on_stack(). >> >> Testing: >> Failing test cases. >> >> webrev: >> http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ >> >> Many thanks in advance and sorry having introduced this bug, >> Albert From erik.helin at oracle.com Thu May 8 08:09:58 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 08 May 2014 10:09:58 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <53685794.3070205@oracle.com> References: <5360DBF1.3050807@oracle.com> <53679AFC.6010907@oracle.com> <53685794.3070205@oracle.com> Message-ID: <2119064.xStVBG7Sy1@ehelin-laptop> On Monday 05 May 2014 20:31:32 PM Jon Masamitsu wrote: > On 5/5/2014 7:06 AM, Erik Helin wrote: > > Hi Jon, > > > > thanks for having a look at the patch! > > > > On 2014-05-01 19:34, Jon Masamitsu wrote: > >> Erik, > >> > >> Change looks good. > > > > Thanks! > > > > On 2014-05-01 19:34, Jon Masamitsu wrote: > >> Please add a comment explaining why we're using committed now and why > >> capacity did not work. Basically an abbreviated version of your > >> explanation > >> below. > > > > The old code in compute_new_size was the last code in metaspace making > > use of "capacity" as a concept. My idea was to remove "capacity" > > completely from the metaspace code after this change. > > > > Do you think that we still should add a comment discussing a concept > > (capacity) that the code no longer will use? Is there a possibility > > that people that are new to the code will become confused? > > Capacity is a concept that is present in the GC code. > I really, really don't want to see capacity come back > into the metaspace code and adding a comment is > the only way I can think of to prevent it. I've uploaded a new patch with a comment: http://cr.openjdk.java.net/~ehelin/8034852/webrev.01/ I've also added the description of the problem in this reveiew request as a comment in the bug: https://bugs.openjdk.java.net/browse/JDK-8034852 Hopefully, if someone wants to change used_after_gc, they will read the comment, maybe do a "hg annotate" to see the change that caused the comment, look up the bug and see the full description of the problem (since the comment talks about serious bugs in the past). What do you think? Thanks, Erik > Jon > > > Thanks, > > Erik > > > > On 2014-05-01 19:34, Jon Masamitsu wrote: > >> Thanks. > >> > >> Jon > >> > >> On 4/30/2014 4:18 AM, Erik Helin wrote: > >>> Hi all, > >>> > >>> this patch solves a rather tricky problem with the sizing of Metaspace. > >>> > >>> The issue happens when the GC threshold for Metaspace (called > >>> "capacity_until_GC" in the code) becomes less than the committed > >>> memory for Metaspace. Any calls to Metaspace::allocate that requires > >>> committing more memory will then fail in > >>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < > >>> MetaspaceAux::committed_memory(). The effect will be a > >>> full GC and after the GC we try to expand and allocate. After the > >>> > >>> expansion and before the allocation, one of two things can happen: > >>> 1. capacity_until_GC is larger than the committed memory after the > >>> > >>> expansion. The allocation will now succeed, but the next allocation > >>> requiring a new chunk will *again* trigger a full GC. This pattern > >>> will repeat itself for each new allocation request requiring a new > >>> chunk. > >>> > >>> 2. capacity_until_GC is still less than the committed memory even > >>> > >>> after the expansion. We throw a Java OOME (incorrectly). > >>> > >>> How can the GC threshold for Metaspace be less than the committed > >>> memory? The problem is that MetaspaceGC::compute_new_size uses the > >>> field > >>> _allocated_capacity for describing the amount of memory in Metaspace > >>> that is "in use". _allocated_capacity does not consider the memory in > >>> the chunk free lists to be "in use", since memory in the chunk free > >>> lists are supposed to be available for new allocations. The problem is > >>> that the chunk free lists can become fragmented, and then the memory > >>> is not available for all kinds of allocations. > >>> > >>> This patch change MetaspaceGC::compute_new_size to use > >>> MetaspaceAux::committed_memory for describing how much memory that is > >>> "in use". The effect will be that memory in the chunk free lists > >>> will no > >>> longer be considered "in use" (but will of course be used for future > >>> allocations where possible). This will prevent capacity_until_GC from > >>> shrinking below the committed memory "by definiton", since > >>> capacity_until_GC can't be lower than the memory that is "in use". > >>> > >>> Based on the results from the perf testing (see below), this change > >>> has no performance impact. > >>> > >>> Webrev: > >>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ > >>> > >>> Testing: > >>> - JPRT > >>> > >>> - Ad-hoc testing: > >>> - Kitchensink > >>> - Dacapo > >>> - Medrec > >>> - runThese > >>> - Parallel Class Loading testlist > >>> - Metaspace testlist > >>> - GC nightly testlist > >>> > >>> - Perf testing: > >>> - SPECjbb2005 > >>> - SPECjbb2013 > >>> - Derby > >>> > >>> - Derby regression tests > >>> > >>> Thanks, > >>> Erik From stefan.karlsson at oracle.com Thu May 8 08:15:24 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 08 May 2014 10:15:24 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <2119064.xStVBG7Sy1@ehelin-laptop> References: <5360DBF1.3050807@oracle.com> <53679AFC.6010907@oracle.com> <53685794.3070205@oracle.com> <2119064.xStVBG7Sy1@ehelin-laptop> Message-ID: <536B3D1C.3000809@oracle.com> On 2014-05-08 10:09, Erik Helin wrote: > On Monday 05 May 2014 20:31:32 PM Jon Masamitsu wrote: >> On 5/5/2014 7:06 AM, Erik Helin wrote: >>> Hi Jon, >>> >>> thanks for having a look at the patch! >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Erik, >>>> >>>> Change looks good. >>> Thanks! >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Please add a comment explaining why we're using committed now and why >>>> capacity did not work. Basically an abbreviated version of your >>>> explanation >>>> below. >>> The old code in compute_new_size was the last code in metaspace making >>> use of "capacity" as a concept. My idea was to remove "capacity" >>> completely from the metaspace code after this change. >>> >>> Do you think that we still should add a comment discussing a concept >>> (capacity) that the code no longer will use? Is there a possibility >>> that people that are new to the code will become confused? >> Capacity is a concept that is present in the GC code. >> I really, really don't want to see capacity come back >> into the metaspace code and adding a comment is >> the only way I can think of to prevent it. > I've uploaded a new patch with a comment: > http://cr.openjdk.java.net/~ehelin/8034852/webrev.01/ > > I've also added the description of the problem in this reveiew request as a > comment in the bug: > https://bugs.openjdk.java.net/browse/JDK-8034852 > > Hopefully, if someone wants to change used_after_gc, they will read the > comment, maybe do a "hg annotate" to see the change that caused the comment, > look up the bug and see the full description of the problem > (since the comment talks about serious bugs in the past). > > What do you think? Looks good to me. thanks, StefanK > > Thanks, > Erik > >> Jon >> >>> Thanks, >>> Erik >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Thanks. >>>> >>>> Jon >>>> >>>> On 4/30/2014 4:18 AM, Erik Helin wrote: >>>>> Hi all, >>>>> >>>>> this patch solves a rather tricky problem with the sizing of Metaspace. >>>>> >>>>> The issue happens when the GC threshold for Metaspace (called >>>>> "capacity_until_GC" in the code) becomes less than the committed >>>>> memory for Metaspace. Any calls to Metaspace::allocate that requires >>>>> committing more memory will then fail in >>>>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >>>>> MetaspaceAux::committed_memory(). The effect will be a >>>>> full GC and after the GC we try to expand and allocate. After the >>>>> >>>>> expansion and before the allocation, one of two things can happen: >>>>> 1. capacity_until_GC is larger than the committed memory after the >>>>> >>>>> expansion. The allocation will now succeed, but the next allocation >>>>> requiring a new chunk will *again* trigger a full GC. This pattern >>>>> will repeat itself for each new allocation request requiring a new >>>>> chunk. >>>>> >>>>> 2. capacity_until_GC is still less than the committed memory even >>>>> >>>>> after the expansion. We throw a Java OOME (incorrectly). >>>>> >>>>> How can the GC threshold for Metaspace be less than the committed >>>>> memory? The problem is that MetaspaceGC::compute_new_size uses the >>>>> field >>>>> _allocated_capacity for describing the amount of memory in Metaspace >>>>> that is "in use". _allocated_capacity does not consider the memory in >>>>> the chunk free lists to be "in use", since memory in the chunk free >>>>> lists are supposed to be available for new allocations. The problem is >>>>> that the chunk free lists can become fragmented, and then the memory >>>>> is not available for all kinds of allocations. >>>>> >>>>> This patch change MetaspaceGC::compute_new_size to use >>>>> MetaspaceAux::committed_memory for describing how much memory that is >>>>> "in use". The effect will be that memory in the chunk free lists >>>>> will no >>>>> longer be considered "in use" (but will of course be used for future >>>>> allocations where possible). This will prevent capacity_until_GC from >>>>> shrinking below the committed memory "by definiton", since >>>>> capacity_until_GC can't be lower than the memory that is "in use". >>>>> >>>>> Based on the results from the perf testing (see below), this change >>>>> has no performance impact. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >>>>> >>>>> Testing: >>>>> - JPRT >>>>> >>>>> - Ad-hoc testing: >>>>> - Kitchensink >>>>> - Dacapo >>>>> - Medrec >>>>> - runThese >>>>> - Parallel Class Loading testlist >>>>> - Metaspace testlist >>>>> - GC nightly testlist >>>>> >>>>> - Perf testing: >>>>> - SPECjbb2005 >>>>> - SPECjbb2013 >>>>> - Derby >>>>> >>>>> - Derby regression tests >>>>> >>>>> Thanks, >>>>> Erik From david.holmes at oracle.com Thu May 8 07:37:12 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 May 2014 17:37:12 +1000 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <6CEDDC71-6274-409F-8AF0-F7712E40C87F@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536AC8D8.6060804@oracle.com> <6CEDDC71-6274-409F-8AF0-F7712E40C87F@oracle.com> Message-ID: <536B3428.5010304@oracle.com> On 8/05/2014 11:13 AM, David Chase wrote: > > On 2014-05-07, at 7:59 PM, Coleen Phillimore wrote: >> So this has been a lot of work and a lot of information over email. I had one question - why isn't there a FORMAT type that can be used for void* ? Using these p2i functions is going to get really annoying. > > What I understand, from reading stackoverflow and poking around, is that %p does not have a uniform behavior across platforms; some of them prefix a 0x, others do not. I think there is variance in how the width can be specified, not sure. > > If we can sort out the %p variation, perhaps by defining PTR_FORMAT to be either 0x%p or %p depending on the platform, and if we can sort out width issues also, then we could get rid of INTPTR_FORMAT (convert to PTR_FORMAT) and remove the p2i casts. I hate to say this, but figuring out the compiler-specific behaviour for %p would seem to involve far less effort and change, than converting all pointers using p2i and changing from PTR_FORMAT to INTPTR_FORMAT. It may even be something that configure can work out for us at configure time ?? David H. -------- >> Does this also address bug https://bugs.openjdk.java.net/browse/JDK-8029996 ? > > Yes. However, it hides the attribute in a macro, since I am not sure whether the attribute is supported across all compilers (it is currently activated for gcc and clang) and there's one or two files where the formats are so borked that turning off the attribute macros before file inclusion (this will need fixing) > > From an earlier email (I should check if this got fixed recently by gc people): >> For two files, >> src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp >> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp >> the macro ATTRIBUTE_PRINTF(x,y) >> is defined to be empty above the header inclusions; the format strings are wrong in peculiar ways, and they are buried in macros. > > gotta go, town meeting accidentally got interesting. > > David > From david.r.chase at oracle.com Thu May 8 11:06:54 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 8 May 2014 07:06:54 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536B3428.5010304@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536AC8D8.6060804@oracle.com> <6CEDDC71-6274-409F-8AF0-F7712E40C87F@oracle.com> <536B3428.5010304@oracle.com> Message-ID: <6C6B4645-3E7D-4C31-9C4E-202076C78B43@oracle.com> On 2014-05-08, at 3:37 AM, David Holmes wrote: > On 8/05/2014 11:13 AM, David Chase wrote: >> >> On 2014-05-07, at 7:59 PM, Coleen Phillimore wrote: >>> So this has been a lot of work and a lot of information over email. I had one question - why isn't there a FORMAT type that can be used for void* ? Using these p2i functions is going to get really annoying. >> >> What I understand, from reading stackoverflow and poking around, is that %p does not have a uniform behavior across platforms; some of them prefix a 0x, others do not. I think there is variance in how the width can be specified, not sure. >> >> If we can sort out the %p variation, perhaps by defining PTR_FORMAT to be either 0x%p or %p depending on the platform, and if we can sort out width issues also, then we could get rid of INTPTR_FORMAT (convert to PTR_FORMAT) and remove the p2i casts. > > I hate to say this, but figuring out the compiler-specific behaviour for %p would seem to involve far less effort and change, than converting all pointers using p2i and changing from PTR_FORMAT to INTPTR_FORMAT. It may even be something that configure can work out for us at configure time ?? I have this nagging suspicion that you are right. I should at least make the effort to see what happens if I use a %08p format on all platforms of alleged interest. David From erik.helin at oracle.com Thu May 8 11:40:31 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 08 May 2014 13:40:31 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <536B3D1C.3000809@oracle.com> References: <5360DBF1.3050807@oracle.com> <53679AFC.6010907@oracle.com> <53685794.3070205@oracle.com> <2119064.xStVBG7Sy1@ehelin-laptop> <536B3D1C.3000809@oracle.com> Message-ID: <536B6D2F.8010608@oracle.com> On 2014-05-08 10:15, Stefan Karlsson wrote: > On 2014-05-08 10:09, Erik Helin wrote: >> On Monday 05 May 2014 20:31:32 PM Jon Masamitsu wrote: >>> On 5/5/2014 7:06 AM, Erik Helin wrote: >>>> Hi Jon, >>>> >>>> thanks for having a look at the patch! >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Erik, >>>>> >>>>> Change looks good. >>>> Thanks! >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Please add a comment explaining why we're using committed now and why >>>>> capacity did not work. Basically an abbreviated version of your >>>>> explanation >>>>> below. >>>> The old code in compute_new_size was the last code in metaspace making >>>> use of "capacity" as a concept. My idea was to remove "capacity" >>>> completely from the metaspace code after this change. >>>> >>>> Do you think that we still should add a comment discussing a concept >>>> (capacity) that the code no longer will use? Is there a possibility >>>> that people that are new to the code will become confused? >>> Capacity is a concept that is present in the GC code. >>> I really, really don't want to see capacity come back >>> into the metaspace code and adding a comment is >>> the only way I can think of to prevent it. >> I've uploaded a new patch with a comment: >> http://cr.openjdk.java.net/~ehelin/8034852/webrev.01/ >> >> I've also added the description of the problem in this reveiew request >> as a >> comment in the bug: >> https://bugs.openjdk.java.net/browse/JDK-8034852 >> >> Hopefully, if someone wants to change used_after_gc, they will read the >> comment, maybe do a "hg annotate" to see the change that caused the >> comment, >> look up the bug and see the full description of the problem >> (since the comment talks about serious bugs in the past). >> >> What do you think? > > Looks good to me. Thanks! Erik > thanks, > StefanK > >> >> Thanks, >> Erik >> >>> Jon >>> >>>> Thanks, >>>> Erik >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Thanks. >>>>> >>>>> Jon >>>>> >>>>> On 4/30/2014 4:18 AM, Erik Helin wrote: >>>>>> Hi all, >>>>>> >>>>>> this patch solves a rather tricky problem with the sizing of >>>>>> Metaspace. >>>>>> >>>>>> The issue happens when the GC threshold for Metaspace (called >>>>>> "capacity_until_GC" in the code) becomes less than the committed >>>>>> memory for Metaspace. Any calls to Metaspace::allocate that requires >>>>>> committing more memory will then fail in >>>>>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >>>>>> MetaspaceAux::committed_memory(). The effect will be a >>>>>> full GC and after the GC we try to expand and allocate. After the >>>>>> >>>>>> expansion and before the allocation, one of two things can happen: >>>>>> 1. capacity_until_GC is larger than the committed memory after the >>>>>> expansion. The allocation will now succeed, but the next >>>>>> allocation >>>>>> requiring a new chunk will *again* trigger a full GC. This >>>>>> pattern >>>>>> will repeat itself for each new allocation request requiring >>>>>> a new >>>>>> chunk. >>>>>> 2. capacity_until_GC is still less than the committed memory even >>>>>> after the expansion. We throw a Java OOME (incorrectly). >>>>>> >>>>>> How can the GC threshold for Metaspace be less than the committed >>>>>> memory? The problem is that MetaspaceGC::compute_new_size uses the >>>>>> field >>>>>> _allocated_capacity for describing the amount of memory in Metaspace >>>>>> that is "in use". _allocated_capacity does not consider the memory in >>>>>> the chunk free lists to be "in use", since memory in the chunk free >>>>>> lists are supposed to be available for new allocations. The >>>>>> problem is >>>>>> that the chunk free lists can become fragmented, and then the memory >>>>>> is not available for all kinds of allocations. >>>>>> >>>>>> This patch change MetaspaceGC::compute_new_size to use >>>>>> MetaspaceAux::committed_memory for describing how much memory that is >>>>>> "in use". The effect will be that memory in the chunk free lists >>>>>> will no >>>>>> longer be considered "in use" (but will of course be used for future >>>>>> allocations where possible). This will prevent capacity_until_GC from >>>>>> shrinking below the committed memory "by definiton", since >>>>>> capacity_until_GC can't be lower than the memory that is "in use". >>>>>> >>>>>> Based on the results from the perf testing (see below), this change >>>>>> has no performance impact. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >>>>>> >>>>>> Testing: >>>>>> - JPRT >>>>>> >>>>>> - Ad-hoc testing: >>>>>> - Kitchensink >>>>>> - Dacapo >>>>>> - Medrec >>>>>> - runThese >>>>>> - Parallel Class Loading testlist >>>>>> - Metaspace testlist >>>>>> - GC nightly testlist >>>>>> >>>>>> - Perf testing: >>>>>> - SPECjbb2005 >>>>>> - SPECjbb2013 >>>>>> - Derby >>>>>> >>>>>> - Derby regression tests >>>>>> >>>>>> Thanks, >>>>>> Erik > From coleen.phillimore at oracle.com Thu May 8 12:02:41 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 08 May 2014 08:02:41 -0400 Subject: [9] RFR(XS): 8042570: Excessive number of tests timing out on nightly testing due to fix for 8040798 In-Reply-To: <536B1253.2010204@oracle.com> References: <536A1A57.4090707@oracle.com> <536AF46C.7000507@oracle.com> <536B1253.2010204@oracle.com> Message-ID: <536B7261.7070602@oracle.com> On 5/8/14, 1:12 AM, Albert wrote: > On 05/08/2014 05:05 AM, David Holmes wrote: >> On 7/05/2014 9:34 PM, Albert wrote: >>> Hi, >>> >>> could I get reviews for this patch? >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8042570 >>> >>> Problem: >>> The fix of JDK-8040798 introduced a lock (MethodCompileQueue_lock) >>> prior >>> to accessing the compilation queues >>> of C1 and C2 in CompileBroker::mark_on_stack(). >>> CompileBroker::mark_on_stack() is executed at a safepoint. >>> The locks that I introduced can cause the deadlock, since the >>> MethodCompileQueue_lock lock is held at a safepoint >>> that is reached from ciEnv::register_method(). This causes the >>> deadlock. >> >> This begs the question why you thought locking was necessary then and >> you no longer think it necessary now? >> > I introduced the lock in the first place, since compilation queues are > accessed by multiple threads and CompileQueue::mark_on_stack() > modifies elements of the queue without acquiring a lock. However, I > did not realize that CompileQueue::mark_on_stack() is executed at > a safepoint. You might have to make sure that the compiler thread (that doesn't stop at safepoint if it doesn't have to), doesn't access these queues outside the safepoint though. I don't think it will because the compiler threads transition into VM to touch Method* and other metadata. I think. Coleen > > Best, > Albert >> David >> ----- >> >>> I verified that deadlock happens as described above using the >>> Redefine_v52_strict_invoke_redefine test. >>> >>> Solution: >>> Remove locks in CompileBroker::mark_on_stack(). >>> >>> Testing: >>> Failing test cases. >>> >>> webrev: >>> http://cr.openjdk.java.net/~anoll/8042570/webrev.00/ >>> >>> Many thanks in advance and sorry having introduced this bug, >>> Albert > From david.r.chase at oracle.com Thu May 8 12:44:51 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 8 May 2014 08:44:51 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <6C6B4645-3E7D-4C31-9C4E-202076C78B43@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536AC8D8.6060804@oracle.com> <6CEDDC71-6274-409F-8AF0-F7712E40C87F@oracle.com> <536B3428.5010304@oracle.com> <6C6B4645-3E7D-4C31-9C4E-202076C78B43@oracle.com> Message-ID: <72C9BFB3-6C58-490F-B8FB-4ACD45989E99@oracle.com> On 2014-05-08, at 7:06 AM, David Chase wrote: > On 2014-05-08, at 3:37 AM, David Holmes wrote: >> On 8/05/2014 11:13 AM, David Chase wrote: >>> >>> On 2014-05-07, at 7:59 PM, Coleen Phillimore wrote: >>>> So this has been a lot of work and a lot of information over email. I had one question - why isn't there a FORMAT type that can be used for void* ? Using these p2i functions is going to get really annoying. >>> >>> What I understand, from reading stackoverflow and poking around, is that %p does not have a uniform behavior across platforms; some of them prefix a 0x, others do not. I think there is variance in how the width can be specified, not sure. >>> >>> If we can sort out the %p variation, perhaps by defining PTR_FORMAT to be either 0x%p or %p depending on the platform, and if we can sort out width issues also, then we could get rid of INTPTR_FORMAT (convert to PTR_FORMAT) and remove the p2i casts. >> >> I hate to say this, but figuring out the compiler-specific behaviour for %p would seem to involve far less effort and change, than converting all pointers using p2i and changing from PTR_FORMAT to INTPTR_FORMAT. It may even be something that configure can work out for us at configure time ?? > > I have this nagging suspicion that you are right. I should at least make the effort to see what happens if I use a %08p format on all platforms of alleged interest. Here's my test program: #include int main(int argc, char ** argv) { printf("Hello world, %4p\n", argv); printf("Hello world, %16p\n", argv); printf("Hello world, %016p\n", argv); return 1; } On gcc-ish systems, this prints things like Hello world, 0x7fff573d9ca0 Hello world, 0x7fff573d9ca0 Hello world, 0x007fff5379e9e0 HOWEVER, I get inconsistent compilation warnings for %016p gcc-4.5 and gcc-4.8 (no warnings) XCode 5 (clang): hellop.c:5:26: warning: flag '0' results in undefined behavior with 'p' conversion specifier [-Wformat] clang-3.1: hellop.c:5:26: warning: flag '0' results in undefined behavior with 'p' conversion specifier [-Wformat] XCode 4.6 (gcc): hellop.c:5: warning: '0' flag used with ?%p? printf format (printed twice for a single line) Solaris studio (12.3, on Solaris 11 x86) prints Hello world, feffeac0 Hello world, feffeac0 Hello world, 00000000feffeac0 Visual Studio 2010 Hello world, 00E330D0 Hello world, 00E330D0 Hello world, 00E330D0 Note that Windows yells at you for pointers, but not for hex formatting. We could handle this mix of behaviors with a conditional definition (leading 0x or not) as long as we did not care too much about the width of the format. I.e., #if defined __GNUC__ #define PTR_FORMAT %p #else #define PTR_FORMAT 0x%p #endif What do we think of this? Can we get an adequate value of "we" before we inflict this on everyone else? David From igor.ignatyev at oracle.com Thu May 8 15:28:13 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 08 May 2014 19:28:13 +0400 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot In-Reply-To: <536B9E36.6090802@oracle.com> References: <536B7CF0.6010508@oracle.com> <536B9E36.6090802@oracle.com> Message-ID: <536BA28D.7030808@oracle.com> // cc'ing hotspot-dev instaed of compiler, runtime and gc lists. On 05/08/2014 07:09 PM, Filipp Zhinkin wrote: > Andrey, > > I've CC'ed compiler and runtime mailing list, because you're changes > affect test for other components as too. > > I don't like your solution (but I'm not a reviewer, so treat my words > just as suggestion), > because we'll have to write more meta information for each test and it > is very easy to > forget to install WhiteBoxPermission if you don't test your test with > some security manager. > > From my point of view, it will be better to extend ClassFileInstaller > so it will copy not only > a class whose name was passed as an arguments, but also all inner > classes of that class. > And if someone want copy only specified class without inner classes, > then some option > could be added to ClassFileInstaller to force such behaviour. > > Thanks, > Filipp. > > On 05/08/2014 04:47 PM, Andrey Zakharov wrote: >> Hi! >> Suggesting patch with fixes for >> https://bugs.openjdk.java.net/browse/JDK-8011397 >> >> webrev: https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz >> patch: >> https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPermission >> >> >> >> Thanks. > From erik.helin at oracle.com Thu May 8 15:34:56 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 08 May 2014 17:34:56 +0200 Subject: RFR: 8027915: TestParallelHeapSizeFlags fails with unexpected heap size on sparcv9 In-Reply-To: <5368C396.4020206@oracle.com> References: <5368A0D6.5050609@oracle.com> <5368C396.4020206@oracle.com> Message-ID: <536BA420.5000408@oracle.com> Hi Stefan, thanks for your thorough review! Please see new webrev at: http://cr.openjdk.java.net/~ehelin/8027915/webrev.01/ and incremental webrev at: http://cr.openjdk.java.net/~ehelin/8027915/webrev.00-01/ and comments inline! On 2014-05-06 13:12, Stefan Johansson wrote: > Hi Erik, > > Thanks for making page_size_for_region more strict. A few comments: > > src/share/vm/runtime/os.cpp > > With this fix page_size_for_region doesn't care about alignment at all > (old version wasn't much better) and to me that feels wrong. I would > expect to get a page size that the given region sizes is aligned to, but > I might be missing some use case where that isn't needed. Agree, page_size_for_region should check alignment for large pages. Added fix and regression tests. On 2014-05-06 13:12, Stefan Johansson wrote: > src/share/vm/gc_implementation/parallelScavenge/generationSizer.cpp > > For the heap we want both min and max to be a multiple of the page size > so we shoud call page_size_for_region for both _min_heap_byte_size and > _max_heap_byte_size and use the smallest of the two returned page sizes. Agree, fixed. On 2014-05-06 13:12, Stefan Johansson wrote: > src/share/vm/memory/heap.cpp > > The current patch will not change the default behavior so I'm fine with > just calling page_size_for_region once using reserved_size, but someone > with more insight might know if we should handle it the same way as the > generationSizer and use both committed and reserved. Since I'm not sure how the compiler guys want to handle this, I opted for the safe alternative and updated the patch to be backwards compatible with the old code (except slightly more strict). Thanks, Erik > Thanks, > Stefan > > > On 2014-05-06 10:44, Erik Helin wrote: >> Hi all, >> >> this patch changes >> os::page_size_for_region(min_size, max_size, min_pages) to always >> guarantee that the returned page size <= max_size / min_pages. Prior >> to this patch, page_size_for_region could return a page size larger >> than max_size / min_pages. This can cause some unexpected behavior >> for code using page_size_for_region, such as the following regression >> tests: >> >> http://cr.openjdk.java.net/~ehelin/8027915/regression-test/ >> >> As the test shows, if you have a 2 MB region and you must use at least >> 2 pages, you would still get back 2 MB as the page size on a Linux >> machine that supports 2 MB large pages. The reason for this is >> explained in a comment above the function in os.hpp: >> >> // The current implementation ignores min_pages if a larger page >> // size is an exact multiple of both region_min_size and >> // region_max_size. This allows larger pages to be used when doing >> // so would not cause fragmentation; in particular, a single page can >> // be used when region_min_size == region_max_size == a supported page >> // size. >> >> Reducing fragmentation of large pages is good but the varying return >> value makes it very hard to depend on os::page_size_for_region in >> calculations (for example when sizing the heap). >> >> This patch removes the fragmentation prevention mechanism, simplifies >> the function and adds a regression tests: >> >> http://cr.openjdk.java.net/~ehelin/8027915/webrev.00/ >> >> Testing: >> - JPRT >> >> Thanks, >> Erik > From jon.masamitsu at oracle.com Thu May 8 17:28:54 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 08 May 2014 10:28:54 -0700 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <2119064.xStVBG7Sy1@ehelin-laptop> References: <5360DBF1.3050807@oracle.com> <53679AFC.6010907@oracle.com> <53685794.3070205@oracle.com> <2119064.xStVBG7Sy1@ehelin-laptop> Message-ID: <536BBED6.4070702@oracle.com> On 05/08/2014 01:09 AM, Erik Helin wrote: > On Monday 05 May 2014 20:31:32 PM Jon Masamitsu wrote: >> On 5/5/2014 7:06 AM, Erik Helin wrote: >>> Hi Jon, >>> >>> thanks for having a look at the patch! >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Erik, >>>> >>>> Change looks good. >>> Thanks! >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Please add a comment explaining why we're using committed now and why >>>> capacity did not work. Basically an abbreviated version of your >>>> explanation >>>> below. >>> The old code in compute_new_size was the last code in metaspace making >>> use of "capacity" as a concept. My idea was to remove "capacity" >>> completely from the metaspace code after this change. >>> >>> Do you think that we still should add a comment discussing a concept >>> (capacity) that the code no longer will use? Is there a possibility >>> that people that are new to the code will become confused? >> Capacity is a concept that is present in the GC code. >> I really, really don't want to see capacity come back >> into the metaspace code and adding a comment is >> the only way I can think of to prevent it. > I've uploaded a new patch with a comment: > http://cr.openjdk.java.net/~ehelin/8034852/webrev.01/ > > I've also added the description of the problem in this reveiew request as a > comment in the bug: > https://bugs.openjdk.java.net/browse/JDK-8034852 > > Hopefully, if someone wants to change used_after_gc, they will read the > comment, maybe do a "hg annotate" to see the change that caused the comment, > look up the bug and see the full description of the problem > (since the comment talks about serious bugs in the past). > > What do you think? I like it. Thanks. Reviewed. Jon > Thanks, > Erik > >> Jon >> >>> Thanks, >>> Erik >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Thanks. >>>> >>>> Jon >>>> >>>> On 4/30/2014 4:18 AM, Erik Helin wrote: >>>>> Hi all, >>>>> >>>>> this patch solves a rather tricky problem with the sizing of Metaspace. >>>>> >>>>> The issue happens when the GC threshold for Metaspace (called >>>>> "capacity_until_GC" in the code) becomes less than the committed >>>>> memory for Metaspace. Any calls to Metaspace::allocate that requires >>>>> committing more memory will then fail in >>>>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >>>>> MetaspaceAux::committed_memory(). The effect will be a >>>>> full GC and after the GC we try to expand and allocate. After the >>>>> >>>>> expansion and before the allocation, one of two things can happen: >>>>> 1. capacity_until_GC is larger than the committed memory after the >>>>> >>>>> expansion. The allocation will now succeed, but the next allocation >>>>> requiring a new chunk will *again* trigger a full GC. This pattern >>>>> will repeat itself for each new allocation request requiring a new >>>>> chunk. >>>>> >>>>> 2. capacity_until_GC is still less than the committed memory even >>>>> >>>>> after the expansion. We throw a Java OOME (incorrectly). >>>>> >>>>> How can the GC threshold for Metaspace be less than the committed >>>>> memory? The problem is that MetaspaceGC::compute_new_size uses the >>>>> field >>>>> _allocated_capacity for describing the amount of memory in Metaspace >>>>> that is "in use". _allocated_capacity does not consider the memory in >>>>> the chunk free lists to be "in use", since memory in the chunk free >>>>> lists are supposed to be available for new allocations. The problem is >>>>> that the chunk free lists can become fragmented, and then the memory >>>>> is not available for all kinds of allocations. >>>>> >>>>> This patch change MetaspaceGC::compute_new_size to use >>>>> MetaspaceAux::committed_memory for describing how much memory that is >>>>> "in use". The effect will be that memory in the chunk free lists >>>>> will no >>>>> longer be considered "in use" (but will of course be used for future >>>>> allocations where possible). This will prevent capacity_until_GC from >>>>> shrinking below the committed memory "by definiton", since >>>>> capacity_until_GC can't be lower than the memory that is "in use". >>>>> >>>>> Based on the results from the perf testing (see below), this change >>>>> has no performance impact. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >>>>> >>>>> Testing: >>>>> - JPRT >>>>> >>>>> - Ad-hoc testing: >>>>> - Kitchensink >>>>> - Dacapo >>>>> - Medrec >>>>> - runThese >>>>> - Parallel Class Loading testlist >>>>> - Metaspace testlist >>>>> - GC nightly testlist >>>>> >>>>> - Perf testing: >>>>> - SPECjbb2005 >>>>> - SPECjbb2013 >>>>> - Derby >>>>> >>>>> - Derby regression tests >>>>> >>>>> Thanks, >>>>> Erik From vladimir.kozlov at oracle.com Thu May 8 17:33:52 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 08 May 2014 10:33:52 -0700 Subject: [8u20] backport 8042309: Some bugfixes for the ppc64 port. Message-ID: <536BC000.1080509@oracle.com> I am backporting these only ppc64 specific changes into 8u20. They are applied cleanly. https://bugs.openjdk.java.net/browse/JDK-8042309 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b5e1deb7279e Thanks, Vladimir From mike.duigou at oracle.com Thu May 8 19:13:14 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Thu, 8 May 2014 12:13:14 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: References: <531F05D3.6020008@oracle.com> Message-ID: Ping. Still needing reviewers on this issue. I plan to push this change through the hotspot-rt repo. Mike On May 3 2014, at 20:24 , Mike Duigou wrote: > Hello; > > Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. > > This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. > > https://bugs.openjdk.java.net/browse/JDK-8032045 > http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ > > Mike > > On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: > >> On 2014-03-11 00:49, Mike Duigou wrote: >>> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>> >>> This version is, hopefully, almost ready to be pushed. >> >> I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. >> >> The top-level build changes looks fine. Thank you Mike for cleaning things up! >> >> /Magnus >> >>> >>> Mike >>> >>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>> >>>> Hello all; >>>> >>>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>>> >>>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>>> >>>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>>> >>>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>>> >>>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>>> >>>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>> >>>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>> >>>> Thanks, >>>> >>>> Mike >> > From vladimir.kozlov at oracle.com Thu May 8 19:16:43 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 08 May 2014 12:16:43 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: References: <531F05D3.6020008@oracle.com> Message-ID: <536BD81B.7020709@oracle.com> Hotspot changes look good. Vladimir On 5/3/14 8:24 PM, Mike Duigou wrote: > Hello; > > Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. > > This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. > > https://bugs.openjdk.java.net/browse/JDK-8032045 > http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ > > Mike > > On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: > >> On 2014-03-11 00:49, Mike Duigou wrote: >>> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>> >>> This version is, hopefully, almost ready to be pushed. >> >> I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. >> >> The top-level build changes looks fine. Thank you Mike for cleaning things up! >> >> /Magnus >> >>> >>> Mike >>> >>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>> >>>> Hello all; >>>> >>>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>>> >>>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>>> >>>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>>> >>>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>>> >>>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>>> >>>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>> >>>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>> >>>> Thanks, >>>> >>>> Mike >> > From coleen.phillimore at oracle.com Thu May 8 19:49:18 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 08 May 2014 15:49:18 -0400 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: References: <531F05D3.6020008@oracle.com> Message-ID: <536BDFBE.1040201@oracle.com> Hi, What tests did you run with these flags on? I can't read makefiles - are they enabled by default? If there are new test failures because these flags are enabled, we need to know before they are checked in. Also, because I'm the gatekeeper. Thanks, Coleen On 5/8/14, 3:13 PM, Mike Duigou wrote: > Ping. Still needing reviewers on this issue. > > I plan to push this change through the hotspot-rt repo. > > Mike > > On May 3 2014, at 20:24 , Mike Duigou wrote: > >> Hello; >> >> Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. >> >> This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. >> >> https://bugs.openjdk.java.net/browse/JDK-8032045 >> http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ >> >> Mike >> >> On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: >> >>> On 2014-03-11 00:49, Mike Duigou wrote: >>>> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>>> >>>> This version is, hopefully, almost ready to be pushed. >>> I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. >>> >>> The top-level build changes looks fine. Thank you Mike for cleaning things up! >>> >>> /Magnus >>> >>>> Mike >>>> >>>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>>> >>>>> Hello all; >>>>> >>>>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>>>> >>>>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>>>> >>>>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>>>> >>>>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>>>> >>>>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>>>> >>>>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>>> >>>>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>>> >>>>> Thanks, >>>>> >>>>> Mike From goetz.lindenmaier at sap.com Thu May 8 20:28:24 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 8 May 2014 20:28:24 +0000 Subject: [8u20] backport 8042309: Some bugfixes for the ppc64 port. In-Reply-To: <536BC000.1080509@oracle.com> References: <536BC000.1080509@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC0724@DEWDFEMB12A.global.corp.sap> Hi Vladimir, thanks for backporting this! Best regards, Goetz -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Kozlov Sent: Donnerstag, 8. Mai 2014 19:34 To: hotspot-dev at openjdk.java.net Subject: [8u20] backport 8042309: Some bugfixes for the ppc64 port. I am backporting these only ppc64 specific changes into 8u20. They are applied cleanly. https://bugs.openjdk.java.net/browse/JDK-8042309 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b5e1deb7279e Thanks, Vladimir From david.r.chase at oracle.com Thu May 8 21:08:53 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 8 May 2014 17:08:53 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536AB5F0.2080706@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: On 2014-05-07, at 6:38 PM, Vladimir Kozlov wrote: > I see several cases of new print_raw_cr(). Why not print_cr("%s", msg)? It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. Fixed. > In ciEnv.cpp it can be merged with previous line: > > tty->print("# Compiler replay data is saved as: %s", buffer); > tty->print("# Compiler inline data is saved as: %s", buffer); Merged. > In ageTable.cpp why you changed type of 'age' from uint to uintx? The used format for it is "%3u". Reverted. > Did you changes some spacing in generateOopMap.cpp? I see only Copyright year update. Reverted. > In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. Fixed. I looked into the issue of changing PTR_FORMAT to be a platform-dependent variation on %p; however it appears to be not-possible to do this without changing with some formatted pointer outputs (I've tried hard not to change outputs at all, though there have been some changes at reviewer request), and because this may affect user-visible outputs (e.g., parsing GC logs) I'd rather not do it as part of this giant patch -- if there's a problem, this is a big revert. This bug fix also deals with https://bugs.openjdk.java.net/browse/JDK-8029996 -- that is, all printf-like methods are tagged as being printf-like, and all carn-inducing uses of those methods has been "fixed", either through actual correction of the format, or by selective muting of the gcc warnings when they were merely gcc being picky about different names for the same number of integer bytes (I think these should still be attended to incrementally). I think the follow-on bugs are: 1) investigate definition of PTR_FORMAT as some form of %p, together with a clean separation of PTR_FORMAT and INTPTR_FORMAT in formatting (we should be nearly there already). Do NOT do a wholesale replace of INTPTR_FORMAT. 2) incremental removal of annoying *and* unnecessary INTPTR_FORMAT/p2i where the output is not intended for external consumption. 3) Attention paid to the formatting macros in src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp which are so broken that format checking in those two files is completely disabled. 4) incremental removal of PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC (This should follow resolution of follow-on bug (1), so that the PTR_FORMAT option is available). New webrevs (alleged backport included for completeness): jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/ jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-08a/ (Note that copyrights will be inserted mechanically later; there is also a single instance of a .cpp file with an extra space on one line, that will also be fixed mechanically.) New testing: jdk9: Mavericks Mountain Lion Solaris 11 SPARC Ubuntu 32-bit, gcc-4.8 Ubuntu 64-bit, gcc-4.8 JPRT build, including closed jdk8u: Mountain Lion Ubuntu 32-bit, gcc-4.8 Ubuntu 64-bit, gcc-4.8 JPRT build, including closed Possible complications to backporting fixes from nine to 8u20 is an issue, thus I am also working on the 8u version of this concurrently. Is it soup yet? David From christian.tornqvist at oracle.com Thu May 8 21:38:40 2014 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Thu, 8 May 2014 17:38:40 -0400 Subject: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 Message-ID: <090301cf6b05$e1492730$a3db7590$@oracle.com> Hi everyone, Small fix for issues with generation of project files for Visual Studio. Recent changes in how Hotspot versioning works broke the Visual Studio build, the fix has been tested by me, Harold and Lois. Webrev can be found at: http://cr.openjdk.java.net/~ctornqvi/webrev/8042471/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8042471 Thanks, Christian From mike.duigou at oracle.com Thu May 8 21:47:27 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Thu, 8 May 2014 14:47:27 -0700 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: On May 8 2014, at 14:08 , David Chase wrote: > > 1) investigate definition of PTR_FORMAT as some form of %p, together with a clean separation > of PTR_FORMAT and INTPTR_FORMAT in formatting (we should be nearly there already). > Do NOT do a wholesale replace of INTPTR_FORMAT. This is something we should definitely move towards. Output form of pointers issue in the jdk repo as well. (Still many %x for pointers). Code or tests which depend upon something other than %p format should be changed/fixed. > 4) incremental removal of PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC > (This should follow resolution of follow-on bug (1), so that the PTR_FORMAT option is available). The PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC usage seemed like a big hammer on first reading but it is much better than -Wno-error=format-literal and I am glad to see that you have a plan for being able to reduce it's use. Is there any way we can ensure that it happens and doesn't end up as maintenance rot? Mike From david.r.chase at oracle.com Fri May 9 00:11:18 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 8 May 2014 20:11:18 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: <549F6E22-ABEA-4FD5-AD86-35A1BA08BC3D@oracle.com> On 2014-05-08, at 5:47 PM, Mike Duigou wrote: > On May 8 2014, at 14:08 , David Chase wrote: >> >> 1) investigate definition of PTR_FORMAT as some form of %p, together with a clean separation >> of PTR_FORMAT and INTPTR_FORMAT in formatting (we should be nearly there already). >> Do NOT do a wholesale replace of INTPTR_FORMAT. > > This is something we should definitely move towards. Output form of pointers issue in the jdk repo as well. (Still many %x for pointers). Code or tests which depend upon something other than %p format should be changed/fixed. I think we are free to move quickly for those cases where end-users have not written tools depending tightly on how we happen to already format pointers. It's not just our code and our tests, I think that is the problem. The two main differences I expect that will occur are changes in length, and on windows the hex output for pointers uses ABCDEF instead of abcdef. >> 4) incremental removal of PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC >> (This should follow resolution of follow-on bug (1), so that the PTR_FORMAT option is available). > > The PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC usage seemed like a big hammer on first reading but it is much better than -Wno-error=format-literal and I am glad to see that you have a plan for being able to reduce it's use. Is there any way we can ensure that it happens and doesn't end up as maintenance rot? mute-warnings-for-gcc had several purposes. The main one was actually to allow me to defer decisions where I did not think I knew what was intended. Anyplace someone was printing a pointer, that was easy, just wrap the value with p2i(), but in other cases there was (e.g.) %d used to print a size_t, and it was not clear whether the right choice was to replace the format or cast the value, and for me to make the change would make it look more intentional than it deserved (unless some future code-editor was cross-referencing portions of code to Hg histories, which seems tedious for them). So to make these go away, I think we have to lean on each team; just comment the line, see what goes blooie, fix it, repeat. I realized this part way through the GC code, so there are some touches there where I may have exceeded my authority, but we're talking about not more than half-a-dozen, then I got queasy and started looking for a way out. In other parts of the code (i.e., the compiler) I was l willing to change code a little more aggressively because I had a better feel for the territory. David From vladimir.kozlov at oracle.com Fri May 9 00:32:28 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 08 May 2014 17:32:28 -0700 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: <536C221C.4090309@oracle.com> Looks good to me. Thanks, Vladimir On 5/8/14 2:08 PM, David Chase wrote: > > On 2014-05-07, at 6:38 PM, Vladimir Kozlov wrote: > >> I see several cases of new print_raw_cr(). Why not print_cr("%s", msg)? It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. > > Fixed. > >> In ciEnv.cpp it can be merged with previous line: >> >> tty->print("# Compiler replay data is saved as: %s", buffer); >> tty->print("# Compiler inline data is saved as: %s", buffer); > > Merged. > >> In ageTable.cpp why you changed type of 'age' from uint to uintx? The used format for it is "%3u". > > Reverted. > >> Did you changes some spacing in generateOopMap.cpp? I see only Copyright year update. > > Reverted. > >> In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. > > Fixed. > > I looked into the issue of changing PTR_FORMAT to be a platform-dependent variation on %p; > however it appears to be not-possible to do this without changing with some formatted pointer > outputs (I've tried hard not to change outputs at all, though there have been some changes at > reviewer request), and because this may affect user-visible outputs (e.g., parsing GC logs) I'd > rather not do it as part of this giant patch -- if there's a problem, this is a big revert. > > This bug fix also deals with https://bugs.openjdk.java.net/browse/JDK-8029996 -- that is, > all printf-like methods are tagged as being printf-like, and all carn-inducing uses of those > methods has been "fixed", either through actual correction of the format, or by selective > muting of the gcc warnings when they were merely gcc being picky about different names > for the same number of integer bytes (I think these should still be attended to incrementally). > > I think the follow-on bugs are: > > 1) investigate definition of PTR_FORMAT as some form of %p, together with a clean separation > of PTR_FORMAT and INTPTR_FORMAT in formatting (we should be nearly there already). > Do NOT do a wholesale replace of INTPTR_FORMAT. > > 2) incremental removal of annoying *and* unnecessary INTPTR_FORMAT/p2i where the output > is not intended for external consumption. > > 3) Attention paid to the formatting macros in > > src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp > src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp > > which are so broken that format checking in those two files is completely disabled. > > 4) incremental removal of PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC > (This should follow resolution of follow-on bug (1), so that the PTR_FORMAT option is available). > > New webrevs (alleged backport included for completeness): > > jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/ > jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-08a/ > > (Note that copyrights will be inserted mechanically later; there is also a single instance > of a .cpp file with an extra space on one line, that will also be fixed mechanically.) > > New testing: > > jdk9: > Mavericks > Mountain Lion > Solaris 11 SPARC > Ubuntu 32-bit, gcc-4.8 > Ubuntu 64-bit, gcc-4.8 > JPRT build, including closed > > jdk8u: > Mountain Lion > Ubuntu 32-bit, gcc-4.8 > Ubuntu 64-bit, gcc-4.8 > JPRT build, including closed > > Possible complications to backporting fixes from nine to 8u20 is an issue, thus > I am also working on the 8u version of this concurrently. > > Is it soup yet? > > David > > From david.r.chase at oracle.com Fri May 9 02:54:40 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 8 May 2014 22:54:40 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536C221C.4090309@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536C221C.4090309@oracle.com> Message-ID: <1DB6B061-727E-4B94-A5A1-4FCC8974987C@oracle.com> FYI, both jdk9 and jdk8u versions of the patch passed hotspot testing on JPRT (in addition to passing for all builds). David On 2014-05-08, at 8:32 PM, Vladimir Kozlov wrote: > Looks good to me. > > Thanks, > Vladimir > > On 5/8/14 2:08 PM, David Chase wrote: >> >> On 2014-05-07, at 6:38 PM, Vladimir Kozlov wrote: >> >>> I see several cases of new print_raw_cr(). Why not print_cr("%s", msg)? It is in ciInstanceKlass.cpp, javaClasses.cpp, codeBlob.cpp, compilerOracle.cpp, g1CollectedHeap.cpp, psPromotionManager.cpp, metaspaceShared.cpp, constMethod.cpp, constantPool.cpp, cpCache.cpp, instanceKlass.cpp, method.cpp, loopnode.cpp, jniCheck.cpp, java.cpp, os.cpp, sweeper.cpp, thread.cpp, attachListener.cpp, diagnosticCommand.cpp, nmtDCmd.cpp, ostream.cpp, vmError.cpp. >> >> Fixed. >> >>> In ciEnv.cpp it can be merged with previous line: >>> >>> tty->print("# Compiler replay data is saved as: %s", buffer); >>> tty->print("# Compiler inline data is saved as: %s", buffer); >> >> Merged. >> >>> In ageTable.cpp why you changed type of 'age' from uint to uintx? The used format for it is "%3u". >> >> Reverted. >> >>> Did you changes some spacing in generateOopMap.cpp? I see only Copyright year update. >> >> Reverted. >> >>> In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. >> >> Fixed. >> >> I looked into the issue of changing PTR_FORMAT to be a platform-dependent variation on %p; >> however it appears to be not-possible to do this without changing with some formatted pointer >> outputs (I've tried hard not to change outputs at all, though there have been some changes at >> reviewer request), and because this may affect user-visible outputs (e.g., parsing GC logs) I'd >> rather not do it as part of this giant patch -- if there's a problem, this is a big revert. >> >> This bug fix also deals with https://bugs.openjdk.java.net/browse/JDK-8029996 -- that is, >> all printf-like methods are tagged as being printf-like, and all carn-inducing uses of those >> methods has been "fixed", either through actual correction of the format, or by selective >> muting of the gcc warnings when they were merely gcc being picky about different names >> for the same number of integer bytes (I think these should still be attended to incrementally). >> >> I think the follow-on bugs are: >> >> 1) investigate definition of PTR_FORMAT as some form of %p, together with a clean separation >> of PTR_FORMAT and INTPTR_FORMAT in formatting (we should be nearly there already). >> Do NOT do a wholesale replace of INTPTR_FORMAT. >> >> 2) incremental removal of annoying *and* unnecessary INTPTR_FORMAT/p2i where the output >> is not intended for external consumption. >> >> 3) Attention paid to the formatting macros in >> >> src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp >> src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp >> >> which are so broken that format checking in those two files is completely disabled. >> >> 4) incremental removal of PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC >> (This should follow resolution of follow-on bug (1), so that the PTR_FORMAT option is available). >> >> New webrevs (alleged backport included for completeness): >> >> jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/ >> jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-08a/ >> >> (Note that copyrights will be inserted mechanically later; there is also a single instance >> of a .cpp file with an extra space on one line, that will also be fixed mechanically.) >> >> New testing: >> >> jdk9: >> Mavericks >> Mountain Lion >> Solaris 11 SPARC >> Ubuntu 32-bit, gcc-4.8 >> Ubuntu 64-bit, gcc-4.8 >> JPRT build, including closed >> >> jdk8u: >> Mountain Lion >> Ubuntu 32-bit, gcc-4.8 >> Ubuntu 64-bit, gcc-4.8 >> JPRT build, including closed >> >> Possible complications to backporting fixes from nine to 8u20 is an issue, thus >> I am also working on the 8u version of this concurrently. >> >> Is it soup yet? >> >> David >> >> From david.holmes at oracle.com Fri May 9 03:04:45 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 09 May 2014 13:04:45 +1000 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> Message-ID: <536C45CD.6050302@oracle.com> On 9/05/2014 7:08 AM, David Chase wrote: > On 2014-05-07, at 6:38 PM, Vladimir Kozlov wrote: >> In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. > > Fixed. > > I looked into the issue of changing PTR_FORMAT to be a platform-dependent variation on %p; > however it appears to be not-possible to do this without changing with some formatted pointer > outputs (I've tried hard not to change outputs at all, though there have been some changes at > reviewer request), and because this may affect user-visible outputs (e.g., parsing GC logs) I'd > rather not do it as part of this giant patch -- if there's a problem, this is a big revert. But surely the p2i + INTPTR_FORMAT change is an even bigger revert! Unless you propose to keep using PTR_FORMAT even with p2i ? (In case it is half as big a revert but still big). Put another way, once you've done the p2i+INTPTR_FORMAT change, what incentive is there to fix the %p issue and revert everything you just changed? This seems like a now-or-never fix to me. David H. ------- > This bug fix also deals with https://bugs.openjdk.java.net/browse/JDK-8029996 -- that is, > all printf-like methods are tagged as being printf-like, and all carn-inducing uses of those > methods has been "fixed", either through actual correction of the format, or by selective > muting of the gcc warnings when they were merely gcc being picky about different names > for the same number of integer bytes (I think these should still be attended to incrementally). > > I think the follow-on bugs are: > > 1) investigate definition of PTR_FORMAT as some form of %p, together with a clean separation > of PTR_FORMAT and INTPTR_FORMAT in formatting (we should be nearly there already). > Do NOT do a wholesale replace of INTPTR_FORMAT. > > 2) incremental removal of annoying *and* unnecessary INTPTR_FORMAT/p2i where the output > is not intended for external consumption. > > 3) Attention paid to the formatting macros in > > src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp > src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp > > which are so broken that format checking in those two files is completely disabled. > > 4) incremental removal of PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC > (This should follow resolution of follow-on bug (1), so that the PTR_FORMAT option is available). > > New webrevs (alleged backport included for completeness): > > jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/ > jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-08a/ > > (Note that copyrights will be inserted mechanically later; there is also a single instance > of a .cpp file with an extra space on one line, that will also be fixed mechanically.) > > New testing: > > jdk9: > Mavericks > Mountain Lion > Solaris 11 SPARC > Ubuntu 32-bit, gcc-4.8 > Ubuntu 64-bit, gcc-4.8 > JPRT build, including closed > > jdk8u: > Mountain Lion > Ubuntu 32-bit, gcc-4.8 > Ubuntu 64-bit, gcc-4.8 > JPRT build, including closed > > Possible complications to backporting fixes from nine to 8u20 is an issue, thus > I am also working on the 8u version of this concurrently. > > Is it soup yet? > > David > > From david.holmes at oracle.com Fri May 9 03:11:00 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 09 May 2014 13:11:00 +1000 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot In-Reply-To: <536BA28D.7030808@oracle.com> References: <536B7CF0.6010508@oracle.com> <536B9E36.6090802@oracle.com> <536BA28D.7030808@oracle.com> Message-ID: <536C4744.4080601@oracle.com> To be reviewed this needs a webrev on cr.openjdk.java.net Thanks, David On 9/05/2014 1:28 AM, Igor Ignatyev wrote: > // cc'ing hotspot-dev instaed of compiler, runtime and gc lists. > > > On 05/08/2014 07:09 PM, Filipp Zhinkin wrote: >> Andrey, >> >> I've CC'ed compiler and runtime mailing list, because you're changes >> affect test for other components as too. >> >> I don't like your solution (but I'm not a reviewer, so treat my words >> just as suggestion), >> because we'll have to write more meta information for each test and it >> is very easy to >> forget to install WhiteBoxPermission if you don't test your test with >> some security manager. >> >> From my point of view, it will be better to extend ClassFileInstaller >> so it will copy not only >> a class whose name was passed as an arguments, but also all inner >> classes of that class. >> And if someone want copy only specified class without inner classes, >> then some option >> could be added to ClassFileInstaller to force such behaviour. >> >> Thanks, >> Filipp. >> >> On 05/08/2014 04:47 PM, Andrey Zakharov wrote: >>> Hi! >>> Suggesting patch with fixes for >>> https://bugs.openjdk.java.net/browse/JDK-8011397 >>> >>> webrev: >>> https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz >>> patch: >>> https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPermission >>> >>> >>> >>> >>> Thanks. >> From david.holmes at oracle.com Fri May 9 04:07:52 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 09 May 2014 14:07:52 +1000 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: References: <531F05D3.6020008@oracle.com> Message-ID: <536C5498.80903@oracle.com> Hi Mike, This looks okay to me, but of course the proof is in the building and testing. David On 4/05/2014 1:24 PM, Mike Duigou wrote: > Hello; > > Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. > > This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. > > https://bugs.openjdk.java.net/browse/JDK-8032045 > http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ > > Mike > > On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: > >> On 2014-03-11 00:49, Mike Duigou wrote: >>> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>> >>> This version is, hopefully, almost ready to be pushed. >> >> I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. >> >> The top-level build changes looks fine. Thank you Mike for cleaning things up! >> >> /Magnus >> >>> >>> Mike >>> >>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>> >>>> Hello all; >>>> >>>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>>> >>>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>>> >>>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>>> >>>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>>> >>>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>>> >>>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>> >>>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>> >>>> Thanks, >>>> >>>> Mike >> > From john.r.rose at oracle.com Fri May 9 05:52:35 2014 From: john.r.rose at oracle.com (John Rose) Date: Thu, 8 May 2014 22:52:35 -0700 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536C45CD.6050302@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536C45CD.6050302@oracle.com> Message-ID: On May 8, 2014, at 8:04 PM, David Holmes wrote: > But surely the p2i + INTPTR_FORMAT change is an even bigger revert! Unless you propose to keep using PTR_FORMAT even with p2i ? (In case it is half as big a revert but still big). > > Put another way, once you've done the p2i+INTPTR_FORMAT change, what incentive is there to fix the %p issue and revert everything you just changed? This seems like a now-or-never fix to me. It would be. And I advise "never". I don't think we should chase "%p". (Or continue chasing Chase.) ? John From per.liden at oracle.com Fri May 9 07:36:58 2014 From: per.liden at oracle.com (Per Liden) Date: Fri, 09 May 2014 09:36:58 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <2119064.xStVBG7Sy1@ehelin-laptop> References: <5360DBF1.3050807@oracle.com> <53679AFC.6010907@oracle.com> <53685794.3070205@oracle.com> <2119064.xStVBG7Sy1@ehelin-laptop> Message-ID: <536C859A.3040905@oracle.com> Hi, On 05/08/2014 10:09 AM, Erik Helin wrote: > On Monday 05 May 2014 20:31:32 PM Jon Masamitsu wrote: >> On 5/5/2014 7:06 AM, Erik Helin wrote: >>> Hi Jon, >>> >>> thanks for having a look at the patch! >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Erik, >>>> >>>> Change looks good. >>> >>> Thanks! >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Please add a comment explaining why we're using committed now and why >>>> capacity did not work. Basically an abbreviated version of your >>>> explanation >>>> below. >>> >>> The old code in compute_new_size was the last code in metaspace making >>> use of "capacity" as a concept. My idea was to remove "capacity" >>> completely from the metaspace code after this change. >>> >>> Do you think that we still should add a comment discussing a concept >>> (capacity) that the code no longer will use? Is there a possibility >>> that people that are new to the code will become confused? >> >> Capacity is a concept that is present in the GC code. >> I really, really don't want to see capacity come back >> into the metaspace code and adding a comment is >> the only way I can think of to prevent it. > > I've uploaded a new patch with a comment: > http://cr.openjdk.java.net/~ehelin/8034852/webrev.01/ > > I've also added the description of the problem in this reveiew request as a > comment in the bug: > https://bugs.openjdk.java.net/browse/JDK-8034852 > > Hopefully, if someone wants to change used_after_gc, they will read the > comment, maybe do a "hg annotate" to see the change that caused the comment, > look up the bug and see the full description of the problem > (since the comment talks about serious bugs in the past). > > What do you think? Still looks good to me (good comment!) /Per > > Thanks, > Erik > >> Jon >> >>> Thanks, >>> Erik >>> >>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>> Thanks. >>>> >>>> Jon >>>> >>>> On 4/30/2014 4:18 AM, Erik Helin wrote: >>>>> Hi all, >>>>> >>>>> this patch solves a rather tricky problem with the sizing of Metaspace. >>>>> >>>>> The issue happens when the GC threshold for Metaspace (called >>>>> "capacity_until_GC" in the code) becomes less than the committed >>>>> memory for Metaspace. Any calls to Metaspace::allocate that requires >>>>> committing more memory will then fail in >>>>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >>>>> MetaspaceAux::committed_memory(). The effect will be a >>>>> full GC and after the GC we try to expand and allocate. After the >>>>> >>>>> expansion and before the allocation, one of two things can happen: >>>>> 1. capacity_until_GC is larger than the committed memory after the >>>>> >>>>> expansion. The allocation will now succeed, but the next allocation >>>>> requiring a new chunk will *again* trigger a full GC. This pattern >>>>> will repeat itself for each new allocation request requiring a new >>>>> chunk. >>>>> >>>>> 2. capacity_until_GC is still less than the committed memory even >>>>> >>>>> after the expansion. We throw a Java OOME (incorrectly). >>>>> >>>>> How can the GC threshold for Metaspace be less than the committed >>>>> memory? The problem is that MetaspaceGC::compute_new_size uses the >>>>> field >>>>> _allocated_capacity for describing the amount of memory in Metaspace >>>>> that is "in use". _allocated_capacity does not consider the memory in >>>>> the chunk free lists to be "in use", since memory in the chunk free >>>>> lists are supposed to be available for new allocations. The problem is >>>>> that the chunk free lists can become fragmented, and then the memory >>>>> is not available for all kinds of allocations. >>>>> >>>>> This patch change MetaspaceGC::compute_new_size to use >>>>> MetaspaceAux::committed_memory for describing how much memory that is >>>>> "in use". The effect will be that memory in the chunk free lists >>>>> will no >>>>> longer be considered "in use" (but will of course be used for future >>>>> allocations where possible). This will prevent capacity_until_GC from >>>>> shrinking below the committed memory "by definiton", since >>>>> capacity_until_GC can't be lower than the memory that is "in use". >>>>> >>>>> Based on the results from the perf testing (see below), this change >>>>> has no performance impact. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >>>>> >>>>> Testing: >>>>> - JPRT >>>>> >>>>> - Ad-hoc testing: >>>>> - Kitchensink >>>>> - Dacapo >>>>> - Medrec >>>>> - runThese >>>>> - Parallel Class Loading testlist >>>>> - Metaspace testlist >>>>> - GC nightly testlist >>>>> >>>>> - Perf testing: >>>>> - SPECjbb2005 >>>>> - SPECjbb2013 >>>>> - Derby >>>>> >>>>> - Derby regression tests >>>>> >>>>> Thanks, >>>>> Erik > From erik.helin at oracle.com Fri May 9 07:47:23 2014 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 09 May 2014 09:47:23 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <536BBED6.4070702@oracle.com> References: <5360DBF1.3050807@oracle.com> <53679AFC.6010907@oracle.com> <53685794.3070205@oracle.com> <2119064.xStVBG7Sy1@ehelin-laptop> <536BBED6.4070702@oracle.com> Message-ID: <536C880B.7090402@oracle.com> On 2014-05-08 19:28, Jon Masamitsu wrote: > > On 05/08/2014 01:09 AM, Erik Helin wrote: >> On Monday 05 May 2014 20:31:32 PM Jon Masamitsu wrote: >>> On 5/5/2014 7:06 AM, Erik Helin wrote: >>>> Hi Jon, >>>> >>>> thanks for having a look at the patch! >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Erik, >>>>> >>>>> Change looks good. >>>> Thanks! >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Please add a comment explaining why we're using committed now and why >>>>> capacity did not work. Basically an abbreviated version of your >>>>> explanation >>>>> below. >>>> The old code in compute_new_size was the last code in metaspace making >>>> use of "capacity" as a concept. My idea was to remove "capacity" >>>> completely from the metaspace code after this change. >>>> >>>> Do you think that we still should add a comment discussing a concept >>>> (capacity) that the code no longer will use? Is there a possibility >>>> that people that are new to the code will become confused? >>> Capacity is a concept that is present in the GC code. >>> I really, really don't want to see capacity come back >>> into the metaspace code and adding a comment is >>> the only way I can think of to prevent it. >> I've uploaded a new patch with a comment: >> http://cr.openjdk.java.net/~ehelin/8034852/webrev.01/ >> >> I've also added the description of the problem in this reveiew request >> as a >> comment in the bug: >> https://bugs.openjdk.java.net/browse/JDK-8034852 >> >> Hopefully, if someone wants to change used_after_gc, they will read the >> comment, maybe do a "hg annotate" to see the change that caused the >> comment, >> look up the bug and see the full description of the problem >> (since the comment talks about serious bugs in the past). >> >> What do you think? > > I like it. Thanks. > > Reviewed. Thanks! Erik > Jon > >> Thanks, >> Erik >> >>> Jon >>> >>>> Thanks, >>>> Erik >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Thanks. >>>>> >>>>> Jon >>>>> >>>>> On 4/30/2014 4:18 AM, Erik Helin wrote: >>>>>> Hi all, >>>>>> >>>>>> this patch solves a rather tricky problem with the sizing of >>>>>> Metaspace. >>>>>> >>>>>> The issue happens when the GC threshold for Metaspace (called >>>>>> "capacity_until_GC" in the code) becomes less than the committed >>>>>> memory for Metaspace. Any calls to Metaspace::allocate that requires >>>>>> committing more memory will then fail in >>>>>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >>>>>> MetaspaceAux::committed_memory(). The effect will be a >>>>>> full GC and after the GC we try to expand and allocate. After the >>>>>> >>>>>> expansion and before the allocation, one of two things can happen: >>>>>> 1. capacity_until_GC is larger than the committed memory after the >>>>>> expansion. The allocation will now succeed, but the next >>>>>> allocation >>>>>> requiring a new chunk will *again* trigger a full GC. This >>>>>> pattern >>>>>> will repeat itself for each new allocation request requiring >>>>>> a new >>>>>> chunk. >>>>>> 2. capacity_until_GC is still less than the committed memory even >>>>>> after the expansion. We throw a Java OOME (incorrectly). >>>>>> >>>>>> How can the GC threshold for Metaspace be less than the committed >>>>>> memory? The problem is that MetaspaceGC::compute_new_size uses the >>>>>> field >>>>>> _allocated_capacity for describing the amount of memory in Metaspace >>>>>> that is "in use". _allocated_capacity does not consider the memory in >>>>>> the chunk free lists to be "in use", since memory in the chunk free >>>>>> lists are supposed to be available for new allocations. The >>>>>> problem is >>>>>> that the chunk free lists can become fragmented, and then the memory >>>>>> is not available for all kinds of allocations. >>>>>> >>>>>> This patch change MetaspaceGC::compute_new_size to use >>>>>> MetaspaceAux::committed_memory for describing how much memory that is >>>>>> "in use". The effect will be that memory in the chunk free lists >>>>>> will no >>>>>> longer be considered "in use" (but will of course be used for future >>>>>> allocations where possible). This will prevent capacity_until_GC from >>>>>> shrinking below the committed memory "by definiton", since >>>>>> capacity_until_GC can't be lower than the memory that is "in use". >>>>>> >>>>>> Based on the results from the perf testing (see below), this change >>>>>> has no performance impact. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >>>>>> >>>>>> Testing: >>>>>> - JPRT >>>>>> >>>>>> - Ad-hoc testing: >>>>>> - Kitchensink >>>>>> - Dacapo >>>>>> - Medrec >>>>>> - runThese >>>>>> - Parallel Class Loading testlist >>>>>> - Metaspace testlist >>>>>> - GC nightly testlist >>>>>> >>>>>> - Perf testing: >>>>>> - SPECjbb2005 >>>>>> - SPECjbb2013 >>>>>> - Derby >>>>>> >>>>>> - Derby regression tests >>>>>> >>>>>> Thanks, >>>>>> Erik > From erik.joelsson at oracle.com Fri May 9 07:52:06 2014 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 09 May 2014 09:52:06 +0200 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: References: <531F05D3.6020008@oracle.com> Message-ID: <536C8926.4070808@oracle.com> Looks good to me. /Erik On 2014-05-04 05:24, Mike Duigou wrote: > Hello; > > Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. > > This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. > > https://bugs.openjdk.java.net/browse/JDK-8032045 > http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ > > Mike > > On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: > >> On 2014-03-11 00:49, Mike Duigou wrote: >>> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>> >>> This version is, hopefully, almost ready to be pushed. >> I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. >> >> The top-level build changes looks fine. Thank you Mike for cleaning things up! >> >> /Magnus >> >>> Mike >>> >>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>> >>>> Hello all; >>>> >>>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>>> >>>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>>> >>>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>>> >>>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>>> >>>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>>> >>>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>> >>>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>> >>>> Thanks, >>>> >>>> Mike From erik.helin at oracle.com Fri May 9 07:47:51 2014 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 09 May 2014 09:47:51 +0200 Subject: RFR: 8034852: Shrinking of Metaspace high-water-mark causes incorrect OutOfMemoryErrors or back-to-back GCs In-Reply-To: <536C859A.3040905@oracle.com> References: <5360DBF1.3050807@oracle.com> <53679AFC.6010907@oracle.com> <53685794.3070205@oracle.com> <2119064.xStVBG7Sy1@ehelin-laptop> <536C859A.3040905@oracle.com> Message-ID: <536C8827.7020502@oracle.com> On 2014-05-09 09:36, Per Liden wrote: > Hi, > > On 05/08/2014 10:09 AM, Erik Helin wrote: >> On Monday 05 May 2014 20:31:32 PM Jon Masamitsu wrote: >>> On 5/5/2014 7:06 AM, Erik Helin wrote: >>>> Hi Jon, >>>> >>>> thanks for having a look at the patch! >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Erik, >>>>> >>>>> Change looks good. >>>> >>>> Thanks! >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Please add a comment explaining why we're using committed now and why >>>>> capacity did not work. Basically an abbreviated version of your >>>>> explanation >>>>> below. >>>> >>>> The old code in compute_new_size was the last code in metaspace making >>>> use of "capacity" as a concept. My idea was to remove "capacity" >>>> completely from the metaspace code after this change. >>>> >>>> Do you think that we still should add a comment discussing a concept >>>> (capacity) that the code no longer will use? Is there a possibility >>>> that people that are new to the code will become confused? >>> >>> Capacity is a concept that is present in the GC code. >>> I really, really don't want to see capacity come back >>> into the metaspace code and adding a comment is >>> the only way I can think of to prevent it. >> >> I've uploaded a new patch with a comment: >> http://cr.openjdk.java.net/~ehelin/8034852/webrev.01/ >> >> I've also added the description of the problem in this reveiew request >> as a >> comment in the bug: >> https://bugs.openjdk.java.net/browse/JDK-8034852 >> >> Hopefully, if someone wants to change used_after_gc, they will read the >> comment, maybe do a "hg annotate" to see the change that caused the >> comment, >> look up the bug and see the full description of the problem >> (since the comment talks about serious bugs in the past). >> >> What do you think? > > Still looks good to me (good comment!) Thanks! Erik > /Per > >> >> Thanks, >> Erik >> >>> Jon >>> >>>> Thanks, >>>> Erik >>>> >>>> On 2014-05-01 19:34, Jon Masamitsu wrote: >>>>> Thanks. >>>>> >>>>> Jon >>>>> >>>>> On 4/30/2014 4:18 AM, Erik Helin wrote: >>>>>> Hi all, >>>>>> >>>>>> this patch solves a rather tricky problem with the sizing of >>>>>> Metaspace. >>>>>> >>>>>> The issue happens when the GC threshold for Metaspace (called >>>>>> "capacity_until_GC" in the code) becomes less than the committed >>>>>> memory for Metaspace. Any calls to Metaspace::allocate that requires >>>>>> committing more memory will then fail in >>>>>> MetaspaceGC::allowed_expansion, because capacity_until_GC() < >>>>>> MetaspaceAux::committed_memory(). The effect will be a >>>>>> full GC and after the GC we try to expand and allocate. After the >>>>>> >>>>>> expansion and before the allocation, one of two things can happen: >>>>>> 1. capacity_until_GC is larger than the committed memory after the >>>>>> >>>>>> expansion. The allocation will now succeed, but the next >>>>>> allocation >>>>>> requiring a new chunk will *again* trigger a full GC. This >>>>>> pattern >>>>>> will repeat itself for each new allocation request requiring >>>>>> a new >>>>>> chunk. >>>>>> >>>>>> 2. capacity_until_GC is still less than the committed memory even >>>>>> >>>>>> after the expansion. We throw a Java OOME (incorrectly). >>>>>> >>>>>> How can the GC threshold for Metaspace be less than the committed >>>>>> memory? The problem is that MetaspaceGC::compute_new_size uses the >>>>>> field >>>>>> _allocated_capacity for describing the amount of memory in Metaspace >>>>>> that is "in use". _allocated_capacity does not consider the memory in >>>>>> the chunk free lists to be "in use", since memory in the chunk free >>>>>> lists are supposed to be available for new allocations. The >>>>>> problem is >>>>>> that the chunk free lists can become fragmented, and then the memory >>>>>> is not available for all kinds of allocations. >>>>>> >>>>>> This patch change MetaspaceGC::compute_new_size to use >>>>>> MetaspaceAux::committed_memory for describing how much memory that is >>>>>> "in use". The effect will be that memory in the chunk free lists >>>>>> will no >>>>>> longer be considered "in use" (but will of course be used for future >>>>>> allocations where possible). This will prevent capacity_until_GC from >>>>>> shrinking below the committed memory "by definiton", since >>>>>> capacity_until_GC can't be lower than the memory that is "in use". >>>>>> >>>>>> Based on the results from the perf testing (see below), this change >>>>>> has no performance impact. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >>>>>> >>>>>> Testing: >>>>>> - JPRT >>>>>> >>>>>> - Ad-hoc testing: >>>>>> - Kitchensink >>>>>> - Dacapo >>>>>> - Medrec >>>>>> - runThese >>>>>> - Parallel Class Loading testlist >>>>>> - Metaspace testlist >>>>>> - GC nightly testlist >>>>>> >>>>>> - Perf testing: >>>>>> - SPECjbb2005 >>>>>> - SPECjbb2013 >>>>>> - Derby >>>>>> >>>>>> - Derby regression tests >>>>>> >>>>>> Thanks, >>>>>> Erik >> > From mike.duigou at oracle.com Fri May 9 08:03:48 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Fri, 9 May 2014 01:03:48 -0700 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536C45CD.6050302@oracle.com> Message-ID: <236D604F-0752-41A4-A837-94143B2EEC66@oracle.com> On May 8 2014, at 22:52 , John Rose wrote: > On May 8, 2014, at 8:04 PM, David Holmes wrote: > >> But surely the p2i + INTPTR_FORMAT change is an even bigger revert! Unless you propose to keep using PTR_FORMAT even with p2i ? (In case it is half as big a revert but still big). >> >> Put another way, once you've done the p2i+INTPTR_FORMAT change, what incentive is there to fix the %p issue and revert everything you just changed? This seems like a now-or-never fix to me. > > It would be. And I advise "never". I don't think we should chase "%p". (Or continue chasing Chase.) Why not? using %X for a non integer format has always seemed lame. For a few environments I've even had nice features such as %p of a function pointer printing out the name of the function. Why is %p not worth the effort? (Or why is %X better?) Mike From mikael.gerdin at oracle.com Fri May 9 08:13:09 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 09 May 2014 10:13:09 +0200 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot In-Reply-To: <536BA28D.7030808@oracle.com> References: <536B7CF0.6010508@oracle.com> <536B9E36.6090802@oracle.com> <536BA28D.7030808@oracle.com> Message-ID: <3503372.bSN5QEX8PY@mgerdin03> On Thursday 08 May 2014 19.28.13 Igor Ignatyev wrote: > // cc'ing hotspot-dev instaed of compiler, runtime and gc lists. > > On 05/08/2014 07:09 PM, Filipp Zhinkin wrote: > > Andrey, > > > > I've CC'ed compiler and runtime mailing list, because you're changes > > affect test for other components as too. > > > > I don't like your solution (but I'm not a reviewer, so treat my words > > just as suggestion), > > because we'll have to write more meta information for each test and it > > is very easy to > > forget to install WhiteBoxPermission if you don't test your test with > > some security manager. > > > > From my point of view, it will be better to extend ClassFileInstaller > > > > so it will copy not only > > a class whose name was passed as an arguments, but also all inner > > classes of that class. > > And if someone want copy only specified class without inner classes, > > then some option > > could be added to ClassFileInstaller to force such behaviour. Perhaps this is a good time to get rid of ClassFileInstaller altogether? https://bugs.openjdk.java.net/browse/JDK-8009117 The reason for its existence is that the WhiteBox class needs to be on the boot class path. If we can live with having all the test's classes on the boot class path then we could use the /bootclasspath option in jtreg as stated in the RFE. /Mikael > > > > Thanks, > > Filipp. > > > > On 05/08/2014 04:47 PM, Andrey Zakharov wrote: > >> Hi! > >> Suggesting patch with fixes for > >> https://bugs.openjdk.java.net/browse/JDK-8011397 > >> > >> webrev: https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz > >> patch: > >> https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPer > >> mission > >> > >> > >> > >> Thanks. From stefan.karlsson at oracle.com Fri May 9 08:20:44 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 09 May 2014 10:20:44 +0200 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> Message-ID: <536C8FDC.9050602@oracle.com> On 2014-05-07 20:35, David Chase wrote: > Re-revised patch. > > Main changes from previous: > > 1) 'print_raw(' introductions replaced with 'print("%s", ' with the exception that the autogenerated code in adlc still uses print_raw because that still seemed slightly less annoying than dealing with the quoted and unquoted format strings appearing in the same fprintf format. Print_raw was generally judged to be kinda ugly, and it also turned out that there were other stream-like classes with no "raw" equivalent so the "%s", idiom had to be used elsewhere anyway. I didn't replace cr() with print_cr("%s", "") because there were opposing opinions (and not just mine) as to which choice was uglier. > > 2) PRIxPTR introductions replaced with INTPTR_FORMAT. > > 3) PTR_FORMAT introductions replaced with INTPTR_FORMAT -- though they are the same, we are printing intptr_t, so the format should be INTPTR_FORMAT. > > Note that neither PRIxPTR nor PTR_FORMAT are banished -- I merely did not make the confusion worse. > If we want to normalize those out altogether, that is another cleanup. > > 4) I followed Henry Jen's suggestions for sprinkling in more format attributes to reduce the need for disabled checking, and removed the check-disables that were no longer necessary (and I think I got them all). > > 5) To help ensure against future portability bit-rot, I tracked down more functions/methods that were eligible for format attributes. This lead to another round of format-use corrections under both gcc and clang (different sets of corrections, I took their union, obviously). > > 6) I also prepared a backport to 8u to look for any glitches there (there were glitches) and see if they had any influence on the patches to 9 (they didn't). > > Webrevs: > jdk9: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-07a/ I went through the GC changes and it looks like it does what you intend. I have a couple of things that I would like to get cleaned up before this gets pushed. 1) You arbitrarily change some of the PTR_FORMAT to INTPTR_FORMAT and leave others as PTR_FORMAT. Can you leave all of them as PTR_FORMAT in the GC code? http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/concurrentMark.cpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1BiasedArray.cpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CardCounts.hpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/heapRegionSeq.inline.hpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/parNew/parOopClosures.inline.hpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/parallelScavenge/parMarkBitMap.hpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.inline.hpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/shared/immutableSpace.cpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_interface/collectedHeap.cpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/memory/cardTableModRefBS.cpp.udiff.html http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/memory/genCollectedHeap.cpp.udiff.html 2) Could you change this to use %u instead: *! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by%zd, larger than word size?", shift_by));* *! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by_" SSIZE_FORMAT ", larger than word size?", (size_t)_ shift_by));* http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1BiasedArray.hpp.udiff.html 3) These FIXME comments are not really helpful. Could you give meaningful comments instead. http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp.frames.html 25 #if !defined(__clang_major__) && defined(__GNUC__) 26 #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. 27 #endif http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp.udiff.html + #ifndef __clang_major__ + #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. + #endif 4) For the record, I find it highly annoying that we need to mess up the code base with all these p2i, but I understand the motivation why we need it for now. If you fix 1-3 I'm fine with this patch, from a GC perspective. thanks, StefanK > jdk8u: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-07b/ > > Testing: > Built "by hand" fastdebug on > Mountain Lion + XCode 4.6.3 > Mavericks + XCode 5.1.1 > Ubuntu 32-bit + gcc 4.8 > > JPRT of hotspot building on all platforms. > > JTREG of hotspot/{compiler,gc,runtime} on Mountain Lion > > 8u has been build-tested on Mountain Lion and Ubuntu-32-gcc-4.8 -- out-of-the-box new build still fails to go on Mavericks, though I understand it can be persuaded with enough flags. > > I'm told that a patch of this size might need careful timing both because it splatters across all of hotspot, and because it will complicate any backports of bug fixes to 8, which is why I prepared a backport patch in addition to the regular one, both to accelerate that process when we need it to happen and to give a feel for what the patch would look like and touch. > From david.r.chase at oracle.com Fri May 9 11:03:11 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 9 May 2014 07:03:11 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536C8FDC.9050602@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> Message-ID: <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> On 2014-05-09, at 4:20 AM, Stefan Karlsson wrote: > I went through the GC changes and it looks like it does what you intend. I have a couple of things that I would like to get cleaned up before this gets pushed. > > 1) You arbitrarily change some of the PTR_FORMAT to INTPTR_FORMAT and leave others as PTR_FORMAT. Can you leave all of them as PTR_FORMAT in the GC code? If I didn't change all PTR_FORMAT to INTPTR_FORMAT, it was either because the PTR_FORMAT was already matched with an intptr_t and did not generate a warning, or because a file was so full of warnings that I just slapped a MUTE_WARNINGS macro on the top of it to put this off till later. The intent, based on feedback from other reviewers, is to consistently print intptr_t with INTPTR_FORMAT, and to change PTR_FORMAT to INTPTR_FORMAT. However, I did not go looking for additional opportunities to do this beyond the fixes I had to make to get the format-warning code happy, because the patch is already large. The difficulty with doing what you ask is that it truly does close the door on introducing PTR_FORMAT being some variation on %p. I was under the impression that GC printing was what is most exposed to end users with automated log-parsing tools, so if we ever did convert PTR_FORMAT to %p, this would have a user-visible impact. That this also aims us in a direction where we have two ugly macros (PTR_FORMAT and INTPTR_FORMAT) defined to mean exactly the same thing is some suggestion that this would be a mistake (i.e., that the status quo, which the other reviewers wanted me to move away from, is currently a mistake). > 2) Could you change this to use %u instead: > > ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by %zd, larger than word size?", shift_by)); > ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by " SSIZE_FORMAT ", larger than word size?", (size_t) shift_by)); > http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1BiasedArray.hpp.udiff.html i.e., cast shift_by to (int), then. Will do. > 3) These FIXME comments are not really helpful. Could you give meaningful comments instead. > > http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp.frames.html > 25 #if !defined(__clang_major__) && defined(__GNUC__) > 26 #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. > 27 #endif > > http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp.udiff.html > + #ifndef __clang_major__ > + #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. > + #endif Would this be better? // FIXME ,the GC code has formats with mismatched arg counts embedded in complex macros. Someone who knows what is intended needs to fix this. Disable this macro definition and compile for more information. > 4) For the record, I find it highly annoying that we need to mess up the code base with all these p2i, but I understand the motivation why we need it for now. > > If you fix 1-3 I'm fine with this patch, from a GC perspective. > > thanks, > StefanK From markus.gronlund at oracle.com Fri May 9 11:11:29 2014 From: markus.gronlund at oracle.com (=?iso-8859-1?B?TWFya3VzIEdy9m5sdW5k?=) Date: Fri, 9 May 2014 04:11:29 -0700 (PDT) Subject: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 In-Reply-To: <090301cf6b05$e1492730$a3db7590$@oracle.com> References: <090301cf6b05$e1492730$a3db7590$@oracle.com> Message-ID: Hi Christian, I just tested the patch and it seems to fix up the newly added preprocessor defines correctly in VS.NET. Thanks for fixing this! /Markus -----Original Message----- From: Christian Tornqvist Sent: den 8 maj 2014 23:39 To: hotspot-dev at openjdk.java.net Subject: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 Hi everyone, Small fix for issues with generation of project files for Visual Studio. Recent changes in how Hotspot versioning works broke the Visual Studio build, the fix has been tested by me, Harold and Lois. Webrev can be found at: http://cr.openjdk.java.net/~ctornqvi/webrev/8042471/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8042471 Thanks, Christian From david.r.chase at oracle.com Fri May 9 11:28:49 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 9 May 2014 07:28:49 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536C45CD.6050302@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536C45CD.6050302@oracle.com> Message-ID: <81CFC53D-E046-4C4F-98C3-5B1CF0C44503@oracle.com> On 2014-05-08, at 11:04 PM, David Holmes wrote: > On 9/05/2014 7:08 AM, David Chase wrote: >> On 2014-05-07, at 6:38 PM, Vladimir Kozlov wrote: >>> In memnode.cpp use INTX_FORMAT instead of %" PRIdPTR. >> >> Fixed. >> >> I looked into the issue of changing PTR_FORMAT to be a platform-dependent variation on %p; >> however it appears to be not-possible to do this without changing with some formatted pointer >> outputs (I've tried hard not to change outputs at all, though there have been some changes at >> reviewer request), and because this may affect user-visible outputs (e.g., parsing GC logs) I'd >> rather not do it as part of this giant patch -- if there's a problem, this is a big revert. > > But surely the p2i + INTPTR_FORMAT change is an even bigger revert! Unless you propose to keep using PTR_FORMAT even with p2i ? (In case it is half as big a revert but still big). It depends on how you measure the size of the change/revert. My goal, undermined slightly by Vladimir K insisting on INTPTR_FORMAT (%016x or %08x) where we had previously just used %x in some compiler code, was to have ZERO changes in the output after this cleanup. The intent is to only deal with format compilation warnings across all platforms, with a good bit of future-proofing and honesty-about-our-warnings thrown in for good measure. Wholesale change to PTR_FORMAT guarantees a change to output on most platforms. I don't think we would ever do a full revert. Anywhere that we print pointers that end up in the hands of users, in particular users with automated log parsers, we need to be careful about gratuitous change. We're in the Java business, after all, if our customers wanted lots of platform-to-platform gratuitous variation, they'd use C++ instead. So any revert would be incremental; once we make PTR_FORMAT work across platforms for actual pointers, if someone is working on some of our own code and wants to get rid of INTPTR_FORMAT and p2i, they can. > Put another way, once you've done the p2i+INTPTR_FORMAT change, what incentive is there to fix the %p issue and revert everything you just changed? This seems like a now-or-never fix to me. The incentive is that our source code is quite ugly, and we'd like to make it prettier when we work on it. David From stefan.karlsson at oracle.com Fri May 9 11:27:06 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 09 May 2014 13:27:06 +0200 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> Message-ID: <536CBB8A.5000005@oracle.com> On 2014-05-09 13:03, David Chase wrote: > On 2014-05-09, at 4:20 AM, Stefan Karlsson wrote: >> I went through the GC changes and it looks like it does what you intend. I have a couple of things that I would like to get cleaned up before this gets pushed. >> >> 1) You arbitrarily change some of the PTR_FORMAT to INTPTR_FORMAT and leave others as PTR_FORMAT. Can you leave all of them as PTR_FORMAT in the GC code? > If I didn't change all PTR_FORMAT to INTPTR_FORMAT, it was either because the PTR_FORMAT was already matched with an intptr_t and did not generate a warning, or because a file was so full of warnings that I just slapped a MUTE_WARNINGS macro on the top of it to put this off till later. The intent, based on feedback from other reviewers, is to consistently print intptr_t with INTPTR_FORMAT, and to change PTR_FORMAT to INTPTR_FORMAT. However, I did not go looking for additional opportunities to do this beyond the fixes I had to make to get the format-warning code happy, because the patch is already large. I don't think that's the case. For example, look at: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/concurrentMark.cpp.udiff.html You change one PTR_FORMAT, but not the other three: HeapWord* b = hr->bottom(); HeapWord* e = hr->end(); HeapWord* t = hr->top(); HeapWord* p = _g1h->top_at_mark_start(hr, _vo); _out->print_cr("** ["PTR_FORMAT", "PTR_FORMAT"] top: "PTR_FORMAT" " *! "TAMS: "PTR_FORMAT, b, e, t, p);* *! "TAMS: "_ INTPTR_FORMAT, p2i(b), p2i(e), p2i(t), p2i(p)_);* > > The difficulty with doing what you ask is that it truly does close the door on introducing PTR_FORMAT being some variation on %p. I was under the impression that GC printing was what is most exposed to end users with automated log-parsing tools, so if we ever did convert PTR_FORMAT to %p, this would have a user-visible impact. That this also aims us in a direction where we have two ugly macros (PTR_FORMAT and INTPTR_FORMAT) defined to mean exactly the same thing is some suggestion that this would be a mistake (i.e., that the status quo, which the other reviewers wanted me to move away from, is currently a mistake). It doesn't make sense to change some of our pointers to use INTPTR_FORMAT, while other places use PTR_FORMAT. It just makes the code inconsistent (yes, I know we already have the inconsistency in some places.) IMO, if we want to print a pointer we should use PTR_FORMAT and workaround the format problem by using p2i. If that's not what we want, we should get rid of PTR_FORMAT. But arbitrarily changing some usage of PTR_FORMAT to INTPTR_FORMAT doesn't help us in any direction. > >> 2) Could you change this to use %u instead: >> >> ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by %zd, larger than word size?", shift_by)); >> ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by " SSIZE_FORMAT ", larger than word size?", (size_t) shift_by)); >> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1BiasedArray.hpp.udiff.html > i.e., cast shift_by to (int), then. Will do. No, shift_int is an uint so no casting is needed. > >> 3) These FIXME comments are not really helpful. Could you give meaningful comments instead. >> >> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp.frames.html >> 25 #if !defined(__clang_major__) && defined(__GNUC__) >> 26 #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. >> 27 #endif >> >> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp.udiff.html >> + #ifndef __clang_major__ >> + #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. >> + #endif > Would this be better? > // FIXME ,the GC code has formats with mismatched arg counts embedded in complex macros. Someone who knows what is intended needs to fix this. Disable this macro definition and compile for more information. OK. I'll create a bug after you have pushed this. StefanK > >> 4) For the record, I find it highly annoying that we need to mess up the code base with all these p2i, but I understand the motivation why we need it for now. >> >> If you fix 1-3 I'm fine with this patch, from a GC perspective. >> >> thanks, >> StefanK From david.r.chase at oracle.com Fri May 9 11:51:23 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 9 May 2014 07:51:23 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536CBB8A.5000005@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> <536CBB8A.5000005@oracle.com> Message-ID: <29A77CA8-C06B-468C-AC9B-D6DACE787B69@oracle.com> On 2014-05-09, at 7:27 AM, Stefan Karlsson wrote: > On 2014-05-09 13:03, David Chase wrote: >> On 2014-05-09, at 4:20 AM, Stefan Karlsson >> wrote: >> >>> I went through the GC changes and it looks like it does what you intend. I have a couple of things that I would like to get cleaned up before this gets pushed. >>> >>> 1) You arbitrarily change some of the PTR_FORMAT to INTPTR_FORMAT and leave others as PTR_FORMAT. Can you leave all of them as PTR_FORMAT in the GC code? >>> >> If I didn't change all PTR_FORMAT to INTPTR_FORMAT, it was either because the PTR_FORMAT was already matched with an intptr_t and did not generate a warning, or because a file was so full of warnings that I just slapped a MUTE_WARNINGS macro on the top of it to put this off till later. The intent, based on feedback from other reviewers, is to consistently print intptr_t with INTPTR_FORMAT, and to change PTR_FORMAT to INTPTR_FORMAT. However, I did not go looking for additional opportunities to do this beyond the fixes I had to make to get the format-warning code happy, because the patch is already large. > > I don't think that's the case. For example, look at: > http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/concurrentMark.cpp.udiff.html > > You change one PTR_FORMAT, but not the other three: > HeapWord* b = hr->bottom(); > HeapWord* e = hr->end(); > HeapWord* t = hr->top(); > HeapWord* p = _g1h->top_at_mark_start(hr, _vo); > _out->print_cr("** ["PTR_FORMAT", "PTR_FORMAT"] top: "PTR_FORMAT" " > ! "TAMS: "PTR_FORMAT, b, e, t, p); > ! "TAMS: " INTPTR_FORMAT, p2i(b), p2i(e), p2i(t), p2i(p)); I didn't need to touch the previous line, so I didn't. What happened here was other reviewers strongly felt that the name of the format for an intptr_t should be INTPTR_FORMAT. I therefore did a search+replace across the entire patch replacing all instances of (word) PTR_FORMAT with INTPTR_FORMAT. Because I did not need to touch the previous line in the patch, I did not change those instances. Ditto for print_raw and print_raw_cr -- they're not in the patch anymore, but there may be other uses elsewhere in the code that I did not patch. I can't do what you ask without annoying other reviewers. And if I do what you ask, it then (probably) forces a wholesale revisit of GC code if/when we attempt to redefine PTR_FORMAT to be truly for pointers and get rid of p2i in our internal-use-only debugging code. (We'll have to do a lot of change anyhow to get rid of the remaining PTR_FORMATs). >> The difficulty with doing what you ask is that it truly does close the door on introducing PTR_FORMAT being some variation on %p. I was under the impression that GC printing was what is most exposed to end users with automated log-parsing tools, so if we ever did convert PTR_FORMAT to %p, this would have a user-visible impact. That this also aims us in a direction where we have two ugly macros (PTR_FORMAT and INTPTR_FORMAT) defined to mean exactly the same thing is some suggestion that this would be a mistake (i.e., that the status quo, which the other reviewers wanted me to move away from, is currently a mistake). >> > > It doesn't make sense to change some of our pointers to use INTPTR_FORMAT, while other places use PTR_FORMAT. It just makes the code inconsistent (yes, I know we already have the inconsistency in some places.) IMO, if we want to print a pointer we should use PTR_FORMAT and workaround the format problem by using p2i. If that's not what we want, we should get rid of PTR_FORMAT. But arbitrarily changing some usage of PTR_FORMAT to INTPTR_FORMAT doesn't help us in any direction. I could make my patch even larger and change all instances of PTR_FORMAT to INTPTR_FORMAT. That would be (1) completely consistent in our formatting in the code and (2) consistent with feedback from other reviewers and (3) no enduser visible changes. It would take us even further down the road towards defining PTR_FORMAT to be truly for pointers (though not with a reliable-width output, and with different CApitAlizAtion on Windows). How do you (and anyone else reading this) feel about that? I would rather do it in a second step, because it would also be a large change. From staffan.larsen at oracle.com Fri May 9 12:02:05 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 9 May 2014 14:02:05 +0200 Subject: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 In-Reply-To: <090301cf6b05$e1492730$a3db7590$@oracle.com> References: <090301cf6b05$e1492730$a3db7590$@oracle.com> Message-ID: <8F2CC5C1-0F49-4139-AFFF-60B33C1C1510@oracle.com> L96: HOTSPOT_RELEASE_VERSION=$(JRE_RELEASE_VERSION) Why do all other assignments have lots of "\\\" around the value, but not this one? Looks good otherwise, /Staffan On 8 maj 2014, at 23:38, Christian Tornqvist wrote: > Hi everyone, > > > > Small fix for issues with generation of project files for Visual Studio. > Recent changes in how Hotspot versioning works broke the Visual Studio > build, the fix has been tested by me, Harold and Lois. > > > > Webrev can be found at: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8042471/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8042471 > > > > Thanks, > > Christian > From christian.tornqvist at oracle.com Fri May 9 12:12:12 2014 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Fri, 9 May 2014 08:12:12 -0400 Subject: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 In-Reply-To: <8F2CC5C1-0F49-4139-AFFF-60B33C1C1510@oracle.com> References: <090301cf6b05$e1492730$a3db7590$@oracle.com> <8F2CC5C1-0F49-4139-AFFF-60B33C1C1510@oracle.com> Message-ID: <09a501cf6b7f$e8f1c0d0$bad54270$@oracle.com> Hi Staffan, This is the fix for the first problem I hit: Invalid command line switch for "CL.exe". The value [HOTSPOT_RELEASE_VERSION=""1.9.0"""] contains an odd number of double-quote characters. Only even numbers of literal double-quote characters are acceptable to command line tools. I can't say for sure but I think this is because of the quotations of JRE_RELEASE_VERSION earlier so we somehow end up with unmatched quotes. The change http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/2fd040528b2a introduced this assignment. Since JRE_RELEASE_VERSION is already quoted on L89 / L91 I think it's enough to just assign the value directly. Thanks, Christian -----Original Message----- From: Staffan Larsen [mailto:staffan.larsen at oracle.com] Sent: Friday, May 9, 2014 8:02 AM To: Christian Tornqvist Cc: hotspot-dev at openjdk.java.net Developers Subject: Re: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 L96: HOTSPOT_RELEASE_VERSION=$(JRE_RELEASE_VERSION) Why do all other assignments have lots of "\\\" around the value, but not this one? Looks good otherwise, /Staffan On 8 maj 2014, at 23:38, Christian Tornqvist wrote: > Hi everyone, > > > > Small fix for issues with generation of project files for Visual Studio. > Recent changes in how Hotspot versioning works broke the Visual Studio > build, the fix has been tested by me, Harold and Lois. > > > > Webrev can be found at: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8042471/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8042471 > > > > Thanks, > > Christian > From david.r.chase at oracle.com Fri May 9 12:12:48 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 9 May 2014 08:12:48 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <29A77CA8-C06B-468C-AC9B-D6DACE787B69@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> <536CBB8A.5000005@oracle.com> <29A77CA8-C06B-468C-AC9B-D6DACE787B69@oracle.com> Message-ID: <2B74E7B6-CE35-4F77-A3DB-7D80EF2EA392@oracle.com> FYI, a quick census of jdk9 hotspot/src post-patch reveals 617 lines (.hpp and .cpp) containing PTR_FORMAT and 803 lines containing INTPTR_FORMAT (search as word, obviously). This would add about 3000-5000 lines to the existing 7200 line patch. However, the semantic content of the additional changes is trivial; the work could be done almost entirely mechanically (and would be done almost entirely mechanically to reduce the risk of human error). Opinions? David From staffan.larsen at oracle.com Fri May 9 12:21:02 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 9 May 2014 14:21:02 +0200 Subject: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 In-Reply-To: <09a501cf6b7f$e8f1c0d0$bad54270$@oracle.com> References: <090301cf6b05$e1492730$a3db7590$@oracle.com> <8F2CC5C1-0F49-4139-AFFF-60B33C1C1510@oracle.com> <09a501cf6b7f$e8f1c0d0$bad54270$@oracle.com> Message-ID: OK. As long as it is intentional and works I am happy. /Staffan On 9 maj 2014, at 14:12, Christian Tornqvist wrote: > Hi Staffan, > > This is the fix for the first problem I hit: > > Invalid command line switch for "CL.exe". The value > [HOTSPOT_RELEASE_VERSION=""1.9.0"""] contains an odd number of double-quote > characters. Only even numbers of literal double-quote characters are > acceptable to command line tools. > > I can't say for sure but I think this is because of the quotations of > JRE_RELEASE_VERSION earlier so we somehow end up with unmatched quotes. The > change > http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/2fd040528b2a introduced > this assignment. > > Since JRE_RELEASE_VERSION is already quoted on L89 / L91 I think it's enough > to just assign the value directly. > > Thanks, > Christian > > -----Original Message----- > From: Staffan Larsen [mailto:staffan.larsen at oracle.com] > Sent: Friday, May 9, 2014 8:02 AM > To: Christian Tornqvist > Cc: hotspot-dev at openjdk.java.net Developers > Subject: Re: RFR(S): 8042471 - Unable to build JDK 9 Hotspot within VS2010 > > L96: HOTSPOT_RELEASE_VERSION=$(JRE_RELEASE_VERSION) > > Why do all other assignments have lots of "\\\" around the value, but not > this one? > > Looks good otherwise, > /Staffan > > On 8 maj 2014, at 23:38, Christian Tornqvist > wrote: > >> Hi everyone, >> >> >> >> Small fix for issues with generation of project files for Visual Studio. >> Recent changes in how Hotspot versioning works broke the Visual Studio >> build, the fix has been tested by me, Harold and Lois. >> >> >> >> Webrev can be found at: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8042471/webrev.00/ >> >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8042471 >> >> >> >> Thanks, >> >> Christian >> > > From david.r.chase at oracle.com Fri May 9 15:14:11 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 9 May 2014 11:14:11 -0400 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <2B74E7B6-CE35-4F77-A3DB-7D80EF2EA392@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> <536CBB8A.5000005@oracle.com> <29A77CA8-C06B-468C-AC9B-D6DACE787B69@oracle.com> <2B74E7B6-CE35-4F77-A3DB-7D80EF2EA392@oracle.com> Message-ID: <45627D75-001B-4496-A03F-E4DC89B43403@oracle.com> Revised patch, after some offline discussion with Stefan: webrev: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ interdiff from previous: http://cr.openjdk.java.net/~drchase/8037816/9.0508b-0509b.interdiff (I recommend the interdiff) The difference is that in the GC code, where PTR_FORMAT and INTPTR_FORMAT have been used to express an intention (even though C++ compilers and standards conspire against this) to preserve that intention. There's likely to be some future change, but it is important to preserve the intention till that change. There was no change to the 8u patch (i.e., the use of PTR_FORMAT in the GC code was preserved there already; I'm going to check the rest of the 8u patch to see if the INTPTR_FORMAT introductions there conform to goals for the other parts of the code). Testing so far has been minimal, but this is an extremely lightweight patch, and PTR_FORMAT and INTPTR_FORMAT currently have identical macro expansion. I thought I would get this up for discussion/review first, then get the testing started. David From stefan.karlsson at oracle.com Fri May 9 15:21:11 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 09 May 2014 17:21:11 +0200 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <45627D75-001B-4496-A03F-E4DC89B43403@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> <536CBB8A.5000005@oracle.com> <29A77CA8-C06B-468C-AC9B-D6DACE787B69@oracle.com> <2B74E7B6-CE35-4F77-A3DB-7D80EF2EA392@oracle.com> <45627D75-001B-4496-A03F-E4DC89B43403@oracle.com> Message-ID: <536CF267.3030201@oracle.com> On 2014-05-09 17:14, David Chase wrote: > Revised patch, after some offline discussion with Stefan: > > webrev: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ > > interdiff from previous: http://cr.openjdk.java.net/~drchase/8037816/9.0508b-0509b.interdiff > > (I recommend the interdiff) > > The difference is that in the GC code, where PTR_FORMAT and INTPTR_FORMAT have > been used to express an intention (even though C++ compilers and standards conspire > against this) to preserve that intention. There's likely to be some future change, but it is > important to preserve the intention till that change. > > There was no change to the 8u patch (i.e., the use of PTR_FORMAT in the GC code was > preserved there already; I'm going to check the rest of the 8u patch to see if the INTPTR_FORMAT > introductions there conform to goals for the other parts of the code). > > Testing so far has been minimal, but this is an extremely lightweight patch, and PTR_FORMAT > and INTPTR_FORMAT currently have identical macro expansion. I thought I would get this up > for discussion/review first, then get the testing started. Thanks for being accommodating with this! StefanK > > David > From alejandro.murillo at oracle.com Fri May 9 17:57:38 2014 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Fri, 09 May 2014 11:57:38 -0600 Subject: RFR (s) 8042661 : Enhance the fix for 8030011 to avoid warnings when using -Wstring-plus-int Message-ID: <536D1712.2090406@oracle.com> Can I get a couple of reviews for this change? webrev: http://cr.openjdk.java.net/~amurillo/9/8042661/ bug: https://bugs.openjdk.java.net/browse/JDK-8042661 Change is required to avoid warning when compiling with -Wstring-plus-int tested with jprt and also on a standalone mac setup Thanks -- Alejandro From vladimir.kozlov at oracle.com Fri May 9 18:18:33 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 09 May 2014 11:18:33 -0700 Subject: RFR (s) 8042661 : Enhance the fix for 8030011 to avoid warnings when using -Wstring-plus-int In-Reply-To: <536D1712.2090406@oracle.com> References: <536D1712.2090406@oracle.com> Message-ID: <536D1BF9.9030607@oracle.com> David Chase is fixing it as part of his large changes 8037816: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/src/share/vm/runtime/vm_version.cpp.udiff.html Vladimir On 5/9/14 10:57 AM, Alejandro E Murillo wrote: > > Can I get a couple of reviews for this change? > > webrev: http://cr.openjdk.java.net/~amurillo/9/8042661/ > bug: https://bugs.openjdk.java.net/browse/JDK-8042661 > > Change is required to avoid warning when compiling with -Wstring-plus-int > tested with jprt and also on a standalone mac setup > > Thanks > From serguei.spitsyn at oracle.com Fri May 9 18:20:58 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 09 May 2014 11:20:58 -0700 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found Message-ID: <536D1C8A.4080503@oracle.com> Please, review the fix for: https://bugs.openjdk.java.net/browse/JDK-8042796 Open webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 Summary: This is a Nightly Stabilization issue that was caused by a combination of two problems: - A regression introduced by the fix of: https://bugs.openjdk.java.net/browse/JDK-7182152 - An SQE testbase infra regression: https://bugs.openjdk.java.net/browse/INTJDK-7611018 A number of the vm.mlvm tests hits this guarantee taht was added by 7182152 (must be relaxed). The issue is with the deleted static private methods that are still present in the CP cache. The fix is to mark the deleted methods with the flag JVM_ACC_IS_DELETED and then use it to relax the guarantee condition. Testing: Running the failing tests: vm.mlvm.indy.func.jvmti In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs on sparcv9 and amd64. Thanks, Serguei From david.r.chase at oracle.com Fri May 9 18:28:06 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 9 May 2014 14:28:06 -0400 Subject: Wish to push fix for 8037816 into main Message-ID: Here's the webrev, minus automated copyright insertions: http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ It's large, it touches a lot, and I can build on Mavericks (with the addition of a one-time fix to header files, not sure how to automate that). It sets attributes to turn on format checking, and formats are mostly/often checked -- there are disabling macros for problematic files, but I fixed all the true stinkers by hand. I have thumbs up from Vladimir Kozlov and Stefan Karlsson. And this gets it in everyone's face sooner, because we might want to do this to 8u20 just so that backports won't be too horrible. (A backport patch is already prepared, being further studied for mistakes, and has been JPRT-tested.) Opinions? David From alejandro.murillo at oracle.com Fri May 9 18:30:31 2014 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Fri, 09 May 2014 12:30:31 -0600 Subject: RFR (s) 8042661 : Enhance the fix for 8030011 to avoid warnings when using -Wstring-plus-int In-Reply-To: <536D1BF9.9030607@oracle.com> References: <536D1712.2090406@oracle.com> <536D1BF9.9030607@oracle.com> Message-ID: <536D1EC7.2090508@oracle.com> sounds good! I think is better if they go together, so I will close this bug. thanks Alejandro On 5/9/2014 12:18 PM, Vladimir Kozlov wrote: > David Chase is fixing it as part of his large changes 8037816: > > http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/src/share/vm/runtime/vm_version.cpp.udiff.html > > > Vladimir > > On 5/9/14 10:57 AM, Alejandro E Murillo wrote: >> >> Can I get a couple of reviews for this change? >> >> webrev: http://cr.openjdk.java.net/~amurillo/9/8042661/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8042661 >> >> Change is required to avoid warning when compiling with >> -Wstring-plus-int >> tested with jprt and also on a standalone mac setup >> >> Thanks >> -- Alejandro From vladimir.kozlov at oracle.com Fri May 9 18:37:24 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 09 May 2014 11:37:24 -0700 Subject: Wish to push fix for 8037816 into main In-Reply-To: References: Message-ID: <536D2064.60807@oracle.com> Good. We want to push it into main jdk9/hs/hotspot and let groups to pull it into their repos. It will allow to test it in all our configurations. Thanks, Vladimir On 5/9/14 11:28 AM, David Chase wrote: > Here's the webrev, minus automated copyright insertions: > > http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ > > It's large, it touches a lot, and I can build on Mavericks > (with the addition of a one-time fix to header files, not sure > how to automate that). It sets attributes to turn on format checking, > and formats are mostly/often checked -- there are disabling macros > for problematic files, but I fixed all the true stinkers by hand. > > I have thumbs up from Vladimir Kozlov and Stefan Karlsson. > > And this gets it in everyone's face sooner, because we might > want to do this to 8u20 just so that backports won't be too horrible. > (A backport patch is already prepared, being further studied for > mistakes, and has been JPRT-tested.) > > Opinions? > > David > From vladimir.kozlov at oracle.com Fri May 9 18:39:28 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 09 May 2014 11:39:28 -0700 Subject: [8u20] RFR: 8029302: Performance regression in Math.pow intrinsic In-Reply-To: <536D1F47.1090104@oracle.com> References: <536D1F47.1090104@oracle.com> Message-ID: <536D20E0.3060106@oracle.com> Good. Note, 8u20 requests should be sent to hotspot-dev. Thanks, Vladimir On 5/9/14 11:32 AM, Niclas Adlertz wrote: > Hi all, > > Please review the backport of 8029302 together with 8042052, where 8042052 fixed an issue introduced by 8029302. > > The changesets apply cleanly. I intend push 8029302 and 8042052 as separate changesets. > > 8029302: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6314 > http://cr.openjdk.java.net/~adlertz/JDK-8029302/webrev01/ > > 8042052: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6385 > http://cr.openjdk.java.net/~anoll/8042052/webrev.01/ > > Kind Regards, > Niclas Adlertz From alejandro.murillo at oracle.com Fri May 9 18:44:17 2014 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Fri, 09 May 2014 12:44:17 -0600 Subject: Wish to push fix for 8037816 into main In-Reply-To: <536D2064.60807@oracle.com> References: <536D2064.60807@oracle.com> Message-ID: <536D2201.5010106@oracle.com> I just took the weekly snapshot, so now is a good timing to do that, as then it will be picked up by all the group repos when they synch down Alejandro On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: > Good. > > We want to push it into main jdk9/hs/hotspot and let groups to pull it > into their repos. It will allow to test it in all our configurations. > > Thanks, > Vladimir > > On 5/9/14 11:28 AM, David Chase wrote: >> Here's the webrev, minus automated copyright insertions: >> >> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >> >> It's large, it touches a lot, and I can build on Mavericks >> (with the addition of a one-time fix to header files, not sure >> how to automate that). It sets attributes to turn on format checking, >> and formats are mostly/often checked -- there are disabling macros >> for problematic files, but I fixed all the true stinkers by hand. >> >> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >> >> And this gets it in everyone's face sooner, because we might >> want to do this to 8u20 just so that backports won't be too horrible. >> (A backport patch is already prepared, being further studied for >> mistakes, and has been JPRT-tested.) >> >> Opinions? >> >> David >> -- Alejandro From coleen.phillimore at oracle.com Fri May 9 18:53:02 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 09 May 2014 14:53:02 -0400 Subject: Wish to push fix for 8037816 into main In-Reply-To: <536D2201.5010106@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> Message-ID: <536D240E.4040902@oracle.com> This is fine with me but only if someone looks at hs main test results and sends out a report of anything new that fails before we pull it down into the group repositories. I'll probably pull hs-rt tomorrow morning (or Sunday night) then. thanks, Coleen On 5/9/14, 2:44 PM, Alejandro E Murillo wrote: > I just took the weekly snapshot, > so now is a good timing to do that, as then it will be picked up > by all the group repos when they synch down > > Alejandro > > On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >> Good. >> >> We want to push it into main jdk9/hs/hotspot and let groups to pull >> it into their repos. It will allow to test it in all our configurations. >> >> Thanks, >> Vladimir >> >> On 5/9/14 11:28 AM, David Chase wrote: >>> Here's the webrev, minus automated copyright insertions: >>> >>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>> >>> It's large, it touches a lot, and I can build on Mavericks >>> (with the addition of a one-time fix to header files, not sure >>> how to automate that). It sets attributes to turn on format checking, >>> and formats are mostly/often checked -- there are disabling macros >>> for problematic files, but I fixed all the true stinkers by hand. >>> >>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>> >>> And this gets it in everyone's face sooner, because we might >>> want to do this to 8u20 just so that backports won't be too horrible. >>> (A backport patch is already prepared, being further studied for >>> mistakes, and has been JPRT-tested.) >>> >>> Opinions? >>> >>> David >>> > From alejandro.murillo at oracle.com Fri May 9 18:58:48 2014 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Fri, 09 May 2014 12:58:48 -0600 Subject: Wish to push fix for 8037816 into main In-Reply-To: <536D240E.4040902@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> <536D240E.4040902@oracle.com> Message-ID: <536D2568.3080402@oracle.com> I've been monitoring those lately, so I will check that tomorrow I'll keep the gatekeepers posted Alejandro On 5/9/2014 12:53 PM, Coleen Phillimore wrote: > > This is fine with me but only if someone looks at hs main test results > and sends out a report of anything new that fails before we pull it > down into the group repositories. I'll probably pull hs-rt tomorrow > morning (or Sunday night) then. > > thanks, > Coleen > > On 5/9/14, 2:44 PM, Alejandro E Murillo wrote: >> I just took the weekly snapshot, >> so now is a good timing to do that, as then it will be picked up >> by all the group repos when they synch down >> >> Alejandro >> >> On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >>> Good. >>> >>> We want to push it into main jdk9/hs/hotspot and let groups to pull >>> it into their repos. It will allow to test it in all our >>> configurations. >>> >>> Thanks, >>> Vladimir >>> >>> On 5/9/14 11:28 AM, David Chase wrote: >>>> Here's the webrev, minus automated copyright insertions: >>>> >>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>>> >>>> It's large, it touches a lot, and I can build on Mavericks >>>> (with the addition of a one-time fix to header files, not sure >>>> how to automate that). It sets attributes to turn on format checking, >>>> and formats are mostly/often checked -- there are disabling macros >>>> for problematic files, but I fixed all the true stinkers by hand. >>>> >>>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>>> >>>> And this gets it in everyone's face sooner, because we might >>>> want to do this to 8u20 just so that backports won't be too horrible. >>>> (A backport patch is already prepared, being further studied for >>>> mistakes, and has been JPRT-tested.) >>>> >>>> Opinions? >>>> >>>> David >>>> >> > -- Alejandro From david.r.chase at oracle.com Fri May 9 20:01:12 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 9 May 2014 16:01:12 -0400 Subject: Wish to push fix for 8037816 into main In-Reply-To: <536D2201.5010106@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> Message-ID: <010F819A-0040-4296-95D9-E451D8E4FDC5@oracle.com> Alejandro, which repository do I push to? http://hg.openjdk.java.net/jdk9/dev/hotspot? David On 2014-05-09, at 2:44 PM, Alejandro E Murillo wrote: > I just took the weekly snapshot, > so now is a good timing to do that, as then it will be picked up > by all the group repos when they synch down > > Alejandro > > On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >> Good. >> >> We want to push it into main jdk9/hs/hotspot and let groups to pull it into their repos. It will allow to test it in all our configurations. >> >> Thanks, >> Vladimir >> >> On 5/9/14 11:28 AM, David Chase wrote: >>> Here's the webrev, minus automated copyright insertions: >>> >>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>> >>> It's large, it touches a lot, and I can build on Mavericks >>> (with the addition of a one-time fix to header files, not sure >>> how to automate that). It sets attributes to turn on format checking, >>> and formats are mostly/often checked -- there are disabling macros >>> for problematic files, but I fixed all the true stinkers by hand. >>> >>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>> >>> And this gets it in everyone's face sooner, because we might >>> want to do this to 8u20 just so that backports won't be too horrible. >>> (A backport patch is already prepared, being further studied for >>> mistakes, and has been JPRT-tested.) >>> >>> Opinions? >>> >>> David >>> > > -- > Alejandro > From alejandro.murillo at oracle.com Fri May 9 20:02:45 2014 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Fri, 09 May 2014 14:02:45 -0600 Subject: Wish to push fix for 8037816 into main In-Reply-To: <010F819A-0040-4296-95D9-E451D8E4FDC5@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> <010F819A-0040-4296-95D9-E451D8E4FDC5@oracle.com> Message-ID: <536D3465.7020006@oracle.com> No, this is the one: http://hg.openjdk.java.net/jdk9/hs/hotspot Alejandro On 5/9/2014 2:01 PM, David Chase wrote: > Alejandro, which repository do I push to? > > http://hg.openjdk.java.net/jdk9/dev/hotspot? > > David > > > On 2014-05-09, at 2:44 PM, Alejandro E Murillo wrote: > >> I just took the weekly snapshot, >> so now is a good timing to do that, as then it will be picked up >> by all the group repos when they synch down >> >> Alejandro >> >> On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >>> Good. >>> >>> We want to push it into main jdk9/hs/hotspot and let groups to pull it into their repos. It will allow to test it in all our configurations. >>> >>> Thanks, >>> Vladimir >>> >>> On 5/9/14 11:28 AM, David Chase wrote: >>>> Here's the webrev, minus automated copyright insertions: >>>> >>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>>> >>>> It's large, it touches a lot, and I can build on Mavericks >>>> (with the addition of a one-time fix to header files, not sure >>>> how to automate that). It sets attributes to turn on format checking, >>>> and formats are mostly/often checked -- there are disabling macros >>>> for problematic files, but I fixed all the true stinkers by hand. >>>> >>>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>>> >>>> And this gets it in everyone's face sooner, because we might >>>> want to do this to 8u20 just so that backports won't be too horrible. >>>> (A backport patch is already prepared, being further studied for >>>> mistakes, and has been JPRT-tested.) >>>> >>>> Opinions? >>>> >>>> David >>>> >> -- >> Alejandro >> -- Alejandro From john.r.rose at oracle.com Fri May 9 20:19:00 2014 From: john.r.rose at oracle.com (John Rose) Date: Fri, 9 May 2014 13:19:00 -0700 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <236D604F-0752-41A4-A837-94143B2EEC66@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536AB5F0.2080706@oracle.com> <536C45CD.6050302@oracle.com> <236D604F-0752-41A4-A837-94143B2EEC66@oracle.com> Message-ID: <6ABF3C22-AD09-4908-972A-8F54A84E2A92@oracle.com> On May 9, 2014, at 1:03 AM, Mike Duigou wrote: > > On May 8 2014, at 22:52 , John Rose wrote: > >> On May 8, 2014, at 8:04 PM, David Holmes wrote: >> >>> But surely the p2i + INTPTR_FORMAT change is an even bigger revert! Unless you propose to keep using PTR_FORMAT even with p2i ? (In case it is half as big a revert but still big). >>> >>> Put another way, once you've done the p2i+INTPTR_FORMAT change, what incentive is there to fix the %p issue and revert everything you just changed? This seems like a now-or-never fix to me. >> >> It would be. And I advise "never". I don't think we should chase "%p". (Or continue chasing Chase.) > > Why not? I guess it's a fair question, though in part it is also a FAQ: http://stackoverflow.com/questions/2369541/where-is-p-useful-with-printf > using %X for a non integer format has always seemed lame. For a few environments I've even had nice features such as %p of a function pointer printing out the name of the function. Why is %p not worth the effort? (Or why is %X better?) Our platform needs to behave with Java-level portability, which is more stringent than C-level portability. Output from the JVM should not use %p, if we intend users to see it, because %p has platform-dependent meaning. In practical terms, this means we should not be using %p anywhere, because it is sometimes hard to predict when something printed by the JVM graduates from an internal debugging aid to something users like to look at. (BTW, I don't want to make a firm process for distinguishing the two things; we have better things to do.) The variations are sometimes by design: Sometimes %p has "nice" features like printing the name of the pointer instead of the value (most often the name "null"). More often it is because %p has not-nice mis-features due to its shady specification, like sometimes ignoring the '0' in '%0p' or shouting in upper case hex, or unpredictably prepending '0x'. This is not one of this cases where some well-meaning cleanup programmer is throwing away something useful. The misbehavior of %p is one of the central reasons our formatting code is painful. (The lack of portable size-specific format modifiers is the other.) For integer or address output that is going to represent the JVM's activities to customers, we need to be satisfied with signed decimal, unsigned decimal, and hexadecimal, in 32 and 64 bit formats. Those are that formats that we can reliably squeeze out of printf and friends (with considerable effort). Or else we need to roll our own pointer presentations (see the disassembler). But, as I said, we should stop chasing %p. It is a will-o-the-wisp that has (once again) led us into a swamp. ? John From christian.thalinger at oracle.com Sat May 10 01:54:09 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 9 May 2014 15:54:09 -1000 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: <536CBB8A.5000005@oracle.com> References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> <536CBB8A.5000005@oracle.com> Message-ID: On May 9, 2014, at 1:27 AM, Stefan Karlsson wrote: > > On 2014-05-09 13:03, David Chase wrote: >> On 2014-05-09, at 4:20 AM, Stefan Karlsson wrote: >>> I went through the GC changes and it looks like it does what you intend. I have a couple of things that I would like to get cleaned up before this gets pushed. >>> >>> 1) You arbitrarily change some of the PTR_FORMAT to INTPTR_FORMAT and leave others as PTR_FORMAT. Can you leave all of them as PTR_FORMAT in the GC code? >> If I didn't change all PTR_FORMAT to INTPTR_FORMAT, it was either because the PTR_FORMAT was already matched with an intptr_t and did not generate a warning, or because a file was so full of warnings that I just slapped a MUTE_WARNINGS macro on the top of it to put this off till later. The intent, based on feedback from other reviewers, is to consistently print intptr_t with INTPTR_FORMAT, and to change PTR_FORMAT to INTPTR_FORMAT. However, I did not go looking for additional opportunities to do this beyond the fixes I had to make to get the format-warning code happy, because the patch is already large. > > I don't think that's the case. For example, look at: > http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/concurrentMark.cpp.udiff.html > > You change one PTR_FORMAT, but not the other three: > > HeapWord* b = hr->bottom(); > HeapWord* e = hr->end(); > HeapWord* t = hr->top(); > HeapWord* p = _g1h->top_at_mark_start(hr, _vo); > _out->print_cr("** ["PTR_FORMAT", "PTR_FORMAT"] top: "PTR_FORMAT" " > *! "TAMS: "PTR_FORMAT, b, e, t, p);* > *! "TAMS: "_ INTPTR_FORMAT, p2i(b), p2i(e), p2i(t), p2i(p)_);* > > >> >> The difficulty with doing what you ask is that it truly does close the door on introducing PTR_FORMAT being some variation on %p. I was under the impression that GC printing was what is most exposed to end users with automated log-parsing tools, so if we ever did convert PTR_FORMAT to %p, this would have a user-visible impact. That this also aims us in a direction where we have two ugly macros (PTR_FORMAT and INTPTR_FORMAT) defined to mean exactly the same thing is some suggestion that this would be a mistake (i.e., that the status quo, which the other reviewers wanted me to move away from, is currently a mistake). > > It doesn't make sense to change some of our pointers to use INTPTR_FORMAT, while other places use PTR_FORMAT. It just makes the code inconsistent (yes, I know we already have the inconsistency in some places.) IMO, if we want to print a pointer we should use PTR_FORMAT and workaround the format problem by using p2i. If that's not what we want, we should get rid of PTR_FORMAT. But arbitrarily changing some usage of PTR_FORMAT to INTPTR_FORMAT doesn't help us in any direction. > >> >>> 2) Could you change this to use %u instead: >>> >>> ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by %zd, larger than word size?", shift_by)); >>> ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by " SSIZE_FORMAT ", larger than word size?", (size_t) shift_by)); >>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1BiasedArray.hpp.udiff.html >> i.e., cast shift_by to (int), then. Will do. > > No, shift_int is an uint so no casting is needed. ?which is ironic in itself :-) > >> >>> 3) These FIXME comments are not really helpful. Could you give meaningful comments instead. >>> >>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp.frames.html >>> 25 #if !defined(__clang_major__) && defined(__GNUC__) >>> 26 #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. >>> 27 #endif >>> >>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp.udiff.html >>> + #ifndef __clang_major__ >>> + #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. >>> + #endif >> Would this be better? >> // FIXME ,the GC code has formats with mismatched arg counts embedded in complex macros. Someone who knows what is intended needs to fix this. Disable this macro definition and compile for more information. > > OK. I'll create a bug after you have pushed this. > > StefanK > >> >>> 4) For the record, I find it highly annoying that we need to mess up the code base with all these p2i, but I understand the motivation why we need it for now. >>> >>> If you fix 1-3 I'm fine with this patch, from a GC perspective. >>> >>> thanks, >>> StefanK From stefan.karlsson at oracle.com Sat May 10 07:24:37 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Sat, 10 May 2014 09:24:37 +0200 Subject: [9] RFR(L): 8037816 : Fix for 8036122 breaks build with Xcode5/clang In-Reply-To: References: <160E09C5-6286-4423-8F56-96596B91E71B@oracle.com> <5355CC9D.3090609@oracle.com> <0FE8263B-A70D-48D2-8D51-EE78A70C923B@oracle.com> <5359F2AE.8010808@oracle.com> <2F5B7196-8CB1-4D70-92AE-E2FE76A1378D@oracle.com> <535DFE3B.60903@oracle.com> <5360BDC8.4050302@oracle.com> <22AFD948-6177-4694-B142-8C34F9F7A833@oracle.com> <5361C68E.9040002@oracle.com> <9172F1EA-D179-42EF-8376-A0085397EB87@oracle.com> <536C8FDC.9050602@oracle.com> <1073BAA7-8941-46B2-AF51-3D2BD3B70503@oracle.com> <536CBB8A.5000005@oracle.com> Message-ID: <536DD435.7030205@oracle.com> On 2014-05-10 03:54, Christian Thalinger wrote: > > On May 9, 2014, at 1:27 AM, Stefan Karlsson > > wrote: > >> >> On 2014-05-09 13:03, David Chase wrote: >>> On 2014-05-09, at 4:20 AM, Stefan Karlsson >>> > wrote: >>>> I went through the GC changes and it looks like it does what you >>>> intend. I have a couple of things that I would like to get cleaned >>>> up before this gets pushed. >>>> >>>> 1) You arbitrarily change some of the PTR_FORMAT to INTPTR_FORMAT >>>> and leave others as PTR_FORMAT. Can you leave all of them as >>>> PTR_FORMAT in the GC code? >>> If I didn't change all PTR_FORMAT to INTPTR_FORMAT, it was either >>> because the PTR_FORMAT was already matched with an intptr_t and did >>> not generate a warning, or because a file was so full of warnings >>> that I just slapped a MUTE_WARNINGS macro on the top of it to put >>> this off till later. The intent, based on feedback from other >>> reviewers, is to consistently print intptr_t with INTPTR_FORMAT, and >>> to change PTR_FORMAT to INTPTR_FORMAT. However, I did not go >>> looking for additional opportunities to do this beyond the fixes I >>> had to make to get the format-warning code happy, because the patch >>> is already large. >> >> I don't think that's the case. For example, look at: >> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/concurrentMark.cpp.udiff.html >> >> >> You change one PTR_FORMAT, but not the other three: >> >> HeapWord* b = hr->bottom(); >> HeapWord* e = hr->end(); >> HeapWord* t = hr->top(); >> HeapWord* p = _g1h->top_at_mark_start(hr, _vo); >> _out->print_cr("** ["PTR_FORMAT", "PTR_FORMAT"] top: "PTR_FORMAT" " >> *! "TAMS: "PTR_FORMAT, b, e, t, p);* >> *! "TAMS: "_ INTPTR_FORMAT, p2i(b), p2i(e), >> p2i(t), p2i(p)_);* >> >> >>> >>> The difficulty with doing what you ask is that it truly does close >>> the door on introducing PTR_FORMAT being some variation on %p. I >>> was under the impression that GC printing was what is most exposed >>> to end users with automated log-parsing tools, so if we ever did >>> convert PTR_FORMAT to %p, this would have a user-visible impact. >>> That this also aims us in a direction where we have two ugly macros >>> (PTR_FORMAT and INTPTR_FORMAT) defined to mean exactly the same >>> thing is some suggestion that this would be a mistake (i.e., that >>> the status quo, which the other reviewers wanted me to move away >>> from, is currently a mistake). >> >> It doesn't make sense to change some of our pointers to use >> INTPTR_FORMAT, while other places use PTR_FORMAT. It just makes the >> code inconsistent (yes, I know we already have the inconsistency in >> some places.) IMO, if we want to print a pointer we should use >> PTR_FORMAT and workaround the format problem by using p2i. If that's >> not what we want, we should get rid of PTR_FORMAT. But arbitrarily >> changing some usage of PTR_FORMAT to INTPTR_FORMAT doesn't help us in >> any direction. >> >>> >>>> 2) Could you change this to use %u instead: >>>> >>>> ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by >>>> %zd, larger than word size?", shift_by)); >>>> ! assert(shift_by < sizeof(uintptr_t) * 8, err_msg("Shifting by >>>> " SSIZE_FORMAT ", larger than word size?", (size_t) shift_by)); >>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1BiasedArray.hpp.udiff.html >>>> >>> i.e., cast shift_by to (int), then. Will do. >> >> No, shift_int is an uint so no casting is needed. > > ?which is ironic in itself :-) Yeah. It's even worse if you see how that value is created ... StefanK > >> >>> >>>> 3) These FIXME comments are not really helpful. Could you give >>>> meaningful comments instead. >>>> >>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp.frames.html >>>> >>>> 25 #if !defined(__clang_major__) && defined(__GNUC__) >>>> 26 #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. >>>> 27 #endif >>>> >>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-08a/src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp.udiff.html >>>> + #ifndef __clang_major__ >>>> + #define ATTRIBUTE_PRINTF(x,y) // FIXME, formats are a mess. >>>> + #endif >>> Would this be better? >>> // FIXME ,the GC code has formats with mismatched arg counts >>> embedded in complex macros. Someone who knows what is intended >>> needs to fix this. Disable this macro definition and compile for >>> more information. >> >> OK. I'll create a bug after you have pushed this. >> >> StefanK >> >>> >>>> 4) For the record, I find it highly annoying that we need to mess >>>> up the code base with all these p2i, but I understand the >>>> motivation why we need it for now. >>>> >>>> If you fix 1-3 I'm fine with this patch, from a GC perspective. >>>> >>>> thanks, >>>> StefanK > From david.holmes at oracle.com Sat May 10 11:27:43 2014 From: david.holmes at oracle.com (David Holmes) Date: Sat, 10 May 2014 21:27:43 +1000 Subject: Wish to push fix for 8037816 into main In-Reply-To: <536D3465.7020006@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> <010F819A-0040-4296-95D9-E451D8E4FDC5@oracle.com> <536D3465.7020006@oracle.com> Message-ID: <536E0D2F.4090309@oracle.com> On 10/05/2014 6:02 AM, Alejandro E Murillo wrote: > No, this is the one: > > http://hg.openjdk.java.net/jdk9/hs/hotspot So to be clear, this is being pushed into hs/hotspot so that it will be pulled down into the hotspot group repos, but it won't get pushed up to jdk9/dev until the next snapshot - right ? Thanks, David > Alejandro > > On 5/9/2014 2:01 PM, David Chase wrote: >> Alejandro, which repository do I push to? >> >> http://hg.openjdk.java.net/jdk9/dev/hotspot? >> >> David >> >> >> On 2014-05-09, at 2:44 PM, Alejandro E Murillo >> wrote: >> >>> I just took the weekly snapshot, >>> so now is a good timing to do that, as then it will be picked up >>> by all the group repos when they synch down >>> >>> Alejandro >>> >>> On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >>>> Good. >>>> >>>> We want to push it into main jdk9/hs/hotspot and let groups to pull >>>> it into their repos. It will allow to test it in all our >>>> configurations. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 5/9/14 11:28 AM, David Chase wrote: >>>>> Here's the webrev, minus automated copyright insertions: >>>>> >>>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>>>> >>>>> It's large, it touches a lot, and I can build on Mavericks >>>>> (with the addition of a one-time fix to header files, not sure >>>>> how to automate that). It sets attributes to turn on format checking, >>>>> and formats are mostly/often checked -- there are disabling macros >>>>> for problematic files, but I fixed all the true stinkers by hand. >>>>> >>>>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>>>> >>>>> And this gets it in everyone's face sooner, because we might >>>>> want to do this to 8u20 just so that backports won't be too horrible. >>>>> (A backport patch is already prepared, being further studied for >>>>> mistakes, and has been JPRT-tested.) >>>>> >>>>> Opinions? >>>>> >>>>> David >>>>> >>> -- >>> Alejandro >>> > From david.r.chase at oracle.com Sat May 10 11:52:42 2014 From: david.r.chase at oracle.com (David Chase) Date: Sat, 10 May 2014 07:52:42 -0400 Subject: Wish to push fix for 8037816 into main In-Reply-To: <536E0D2F.4090309@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> <010F819A-0040-4296-95D9-E451D8E4FDC5@oracle.com> <536D3465.7020006@oracle.com> <536E0D2F.4090309@oracle.com> Message-ID: <8DC34F8F-4C85-457C-8058-55D032685C53@oracle.com> I think so. It went here: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/53a41e7cbe05 (With its corresponding closed change, where I goofed and left you off the list of reviewers, sorry.) David On 2014-05-10, at 7:27 AM, David Holmes wrote: > On 10/05/2014 6:02 AM, Alejandro E Murillo wrote: >> No, this is the one: >> >> http://hg.openjdk.java.net/jdk9/hs/hotspot > > So to be clear, this is being pushed into hs/hotspot so that it will be pulled down into the hotspot group repos, but it won't get pushed up to jdk9/dev until the next snapshot - right ? > > Thanks, > David > >> Alejandro >> >> On 5/9/2014 2:01 PM, David Chase wrote: >>> Alejandro, which repository do I push to? >>> >>> http://hg.openjdk.java.net/jdk9/dev/hotspot? >>> >>> David >>> >>> >>> On 2014-05-09, at 2:44 PM, Alejandro E Murillo >>> wrote: >>> >>>> I just took the weekly snapshot, >>>> so now is a good timing to do that, as then it will be picked up >>>> by all the group repos when they synch down >>>> >>>> Alejandro >>>> >>>> On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >>>>> Good. >>>>> >>>>> We want to push it into main jdk9/hs/hotspot and let groups to pull >>>>> it into their repos. It will allow to test it in all our >>>>> configurations. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 5/9/14 11:28 AM, David Chase wrote: >>>>>> Here's the webrev, minus automated copyright insertions: >>>>>> >>>>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>>>>> >>>>>> It's large, it touches a lot, and I can build on Mavericks >>>>>> (with the addition of a one-time fix to header files, not sure >>>>>> how to automate that). It sets attributes to turn on format checking, >>>>>> and formats are mostly/often checked -- there are disabling macros >>>>>> for problematic files, but I fixed all the true stinkers by hand. >>>>>> >>>>>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>>>>> >>>>>> And this gets it in everyone's face sooner, because we might >>>>>> want to do this to 8u20 just so that backports won't be too horrible. >>>>>> (A backport patch is already prepared, being further studied for >>>>>> mistakes, and has been JPRT-tested.) >>>>>> >>>>>> Opinions? >>>>>> >>>>>> David >>>>>> >>>> -- >>>> Alejandro >>>> >> From david.r.chase at oracle.com Sat May 10 18:02:55 2014 From: david.r.chase at oracle.com (David Chase) Date: Sat, 10 May 2014 14:02:55 -0400 Subject: Pointer formats, again... Message-ID: Based on discussions for the recent port-to-mavericks+clang/format-fun patch, it appears that wrapping every pointer in p2i(...) before printing is ugly. We'd like to use %p, but cannot for customer-visible formatting because C++ was standardized by a herd of cats. We can't use PTR_FORMAT for this (at least not yet) both because the change would be reasonably large, and because there are places where PTR_FORMAT is used to express an intent in the source code, and that are also customer-visible. Therefore, what we clearly need is ANOTHER FORMATTING MACRO. It would be the equivalent of (done differently with conditional inclusion) #if defined __GNUC__ #define PS_ PTR_FMT %p #else #define PS_PTR_FMT 0x%p #endif PS_PTR_FMT stands for Platform Specific PoinTeR ForMaT I'm not the least bit attached to the name, PSPF would make me just as (un)happy. This could not be used where the exact format of pointers matter, because there is no width control, sometimes it might print "0xnull" (because C++ is Awe-Some) and on windows it will print hex in uppercase. Here's my test program: #include int main(int argc, char ** argv) { printf("%%p of NULL='%p'\n", (void *) 0); printf("%%p of &main='%p'\n", &main); printf("%%4p='%4p'\n", argv); printf("%%16p='%16p'\n", argv); printf("%%016p='%016p'\n", argv); return 1; } Mountain Lion: (compilation warnings for %016p) %p of NULL='0x0' %p of &main='0x105783e60' %4p='0x7fff5a47c9d0' %16p=' 0x7fff5a47c9d0' %016p='0x007fff5a47c9d0' Ubuntu32, 13.10: (compilation warnings for %016p) %p of NULL='(nil)' %p of &main='0x804841d' %4p='0xbfacd584' %16p=' 0xbfacd584' %016p='0x000000bfacd584' Ubuntu64, 14.04: (compilation warnings for %016p) %p of NULL='(nil)' %p of &main='0x40052d' %4p='0x7fff573e68d8' %16p=' 0x7fff573e68d8' %016p='0x007fff573e68d8' Solaris11: %p of NULL='0' %p of &main='80509a0' %4p='feffeac0' %16p=' feffeac0' %016p='00000000feffeac0' I don't think we've got this to the point that I would call it a "good idea" yet. It seems to me that we'd instead need to explore the route of doing our own formatting. For that matter, do we have any assurance that the formatting of doubles on all C/C++ platforms is reproducible and clean? David From alejandro.murillo at oracle.com Sat May 10 23:34:00 2014 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Sat, 10 May 2014 17:34:00 -0600 Subject: Wish to push fix for 8037816 into main In-Reply-To: <8DC34F8F-4C85-457C-8058-55D032685C53@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> <010F819A-0040-4296-95D9-E451D8E4FDC5@oracle.com> <536D3465.7020006@oracle.com> <536E0D2F.4090309@oracle.com> <8DC34F8F-4C85-457C-8058-55D032685C53@oracle.com> Message-ID: <536EB768.7040609@oracle.com> On 5/10/2014 5:52 AM, David Chase wrote: > I think so. that's correct. They will get about a week of nightly testing on all group repos. These changes will be in next week snapshot and then integrated into jdk9/dev/hotspot around Tuesday 5/20 Alejandro > > It went here: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/53a41e7cbe05 > (With its corresponding closed change, where I goofed and left you off the list of reviewers, sorry.) > > David > > On 2014-05-10, at 7:27 AM, David Holmes wrote: > >> On 10/05/2014 6:02 AM, Alejandro E Murillo wrote: >>> No, this is the one: >>> >>> http://hg.openjdk.java.net/jdk9/hs/hotspot >> So to be clear, this is being pushed into hs/hotspot so that it will be pulled down into the hotspot group repos, but it won't get pushed up to jdk9/dev until the next snapshot - right ? >> >> Thanks, >> David >> >>> Alejandro >>> >>> On 5/9/2014 2:01 PM, David Chase wrote: >>>> Alejandro, which repository do I push to? >>>> >>>> http://hg.openjdk.java.net/jdk9/dev/hotspot? >>>> >>>> David >>>> >>>> >>>> On 2014-05-09, at 2:44 PM, Alejandro E Murillo >>>> wrote: >>>> >>>>> I just took the weekly snapshot, >>>>> so now is a good timing to do that, as then it will be picked up >>>>> by all the group repos when they synch down >>>>> >>>>> Alejandro >>>>> >>>>> On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >>>>>> Good. >>>>>> >>>>>> We want to push it into main jdk9/hs/hotspot and let groups to pull >>>>>> it into their repos. It will allow to test it in all our >>>>>> configurations. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 5/9/14 11:28 AM, David Chase wrote: >>>>>>> Here's the webrev, minus automated copyright insertions: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>>>>>> >>>>>>> It's large, it touches a lot, and I can build on Mavericks >>>>>>> (with the addition of a one-time fix to header files, not sure >>>>>>> how to automate that). It sets attributes to turn on format checking, >>>>>>> and formats are mostly/often checked -- there are disabling macros >>>>>>> for problematic files, but I fixed all the true stinkers by hand. >>>>>>> >>>>>>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>>>>>> >>>>>>> And this gets it in everyone's face sooner, because we might >>>>>>> want to do this to 8u20 just so that backports won't be too horrible. >>>>>>> (A backport patch is already prepared, being further studied for >>>>>>> mistakes, and has been JPRT-tested.) >>>>>>> >>>>>>> Opinions? >>>>>>> >>>>>>> David >>>>>>> >>>>> -- >>>>> Alejandro >>>>> -- Alejandro From alejandro.murillo at oracle.com Sat May 10 23:40:48 2014 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Sat, 10 May 2014 17:40:48 -0600 Subject: Wish to push fix for 8037816 into main In-Reply-To: <536D240E.4040902@oracle.com> References: <536D2064.60807@oracle.com> <536D2201.5010106@oracle.com> <536D240E.4040902@oracle.com> Message-ID: <536EB900.8000100@oracle.com> Nightly teasting looks good, only 1 failure marjked as new but is actually a known problem go ahead and synch when you have a chance thanks Alejandro On 5/9/2014 12:53 PM, Coleen Phillimore wrote: > > This is fine with me but only if someone looks at hs main test results > and sends out a report of anything new that fails before we pull it > down into the group repositories. I'll probably pull hs-rt tomorrow > morning (or Sunday night) then. > > thanks, > Coleen > > On 5/9/14, 2:44 PM, Alejandro E Murillo wrote: >> I just took the weekly snapshot, >> so now is a good timing to do that, as then it will be picked up >> by all the group repos when they synch down >> >> Alejandro >> >> On 5/9/2014 12:37 PM, Vladimir Kozlov wrote: >>> Good. >>> >>> We want to push it into main jdk9/hs/hotspot and let groups to pull >>> it into their repos. It will allow to test it in all our >>> configurations. >>> >>> Thanks, >>> Vladimir >>> >>> On 5/9/14 11:28 AM, David Chase wrote: >>>> Here's the webrev, minus automated copyright insertions: >>>> >>>> http://cr.openjdk.java.net/~drchase/8037816/webrev-9.05-09b/ >>>> >>>> It's large, it touches a lot, and I can build on Mavericks >>>> (with the addition of a one-time fix to header files, not sure >>>> how to automate that). It sets attributes to turn on format checking, >>>> and formats are mostly/often checked -- there are disabling macros >>>> for problematic files, but I fixed all the true stinkers by hand. >>>> >>>> I have thumbs up from Vladimir Kozlov and Stefan Karlsson. >>>> >>>> And this gets it in everyone's face sooner, because we might >>>> want to do this to 8u20 just so that backports won't be too horrible. >>>> (A backport patch is already prepared, being further studied for >>>> mistakes, and has been JPRT-tested.) >>>> >>>> Opinions? >>>> >>>> David >>>> >> > -- Alejandro From dmitry.samersoff at oracle.com Sun May 11 07:53:38 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Sun, 11 May 2014 11:53:38 +0400 Subject: Pointer formats, again... In-Reply-To: References: Message-ID: <536F2C82.3000008@oracle.com> David, Other approach might be to create a simple wrapper[1] that parse format string and fix platform-related issues. It would cost a bit of performance but makes code much more cleaner (no format macros), and allow us to don't spend efforts to this problem anymore. [1] This is an example of such wrapper I wrote about 10 years ago, to solve similar problem. http://libdms.cvs.sourceforge.net/viewvc/libdms/libdms4/src/Util/dsform.C?revision=1.4&view=markup -Dmitry On 2014-05-10 22:02, David Chase wrote: > Based on discussions for the recent port-to-mavericks+clang/format-fun patch, > it appears that wrapping every pointer in p2i(...) before printing is ugly. We'd like > to use %p, but cannot for customer-visible formatting because C++ was standardized > by a herd of cats. We can't use PTR_FORMAT for this (at least not yet) both because > the change would be reasonably large, and because there are places where PTR_FORMAT > is used to express an intent in the source code, and that are also customer-visible. > > Therefore, what we clearly need is ANOTHER FORMATTING MACRO. > > It would be the equivalent of (done differently with conditional inclusion) > > #if defined __GNUC__ > #define PS_ PTR_FMT %p > #else > #define PS_PTR_FMT 0x%p > #endif > > PS_PTR_FMT stands for Platform Specific PoinTeR ForMaT > > I'm not the least bit attached to the name, PSPF would make me just as (un)happy. > > This could not be used where the exact format of pointers matter, because > there is no width control, sometimes it might print "0xnull" (because C++ is Awe-Some) > and on windows it will print hex in uppercase. > > Here's my test program: > #include > int main(int argc, char ** argv) { > printf("%%p of NULL='%p'\n", (void *) 0); > printf("%%p of &main='%p'\n", &main); > printf("%%4p='%4p'\n", argv); > printf("%%16p='%16p'\n", argv); > printf("%%016p='%016p'\n", argv); > return 1; > } > > Mountain Lion: > (compilation warnings for %016p) > %p of NULL='0x0' > %p of &main='0x105783e60' > %4p='0x7fff5a47c9d0' > %16p=' 0x7fff5a47c9d0' > %016p='0x007fff5a47c9d0' > > Ubuntu32, 13.10: > (compilation warnings for %016p) > %p of NULL='(nil)' > %p of &main='0x804841d' > %4p='0xbfacd584' > %16p=' 0xbfacd584' > %016p='0x000000bfacd584' > > Ubuntu64, 14.04: > (compilation warnings for %016p) > %p of NULL='(nil)' > %p of &main='0x40052d' > %4p='0x7fff573e68d8' > %16p=' 0x7fff573e68d8' > %016p='0x007fff573e68d8' > > Solaris11: > %p of NULL='0' > %p of &main='80509a0' > %4p='feffeac0' > %16p=' feffeac0' > %016p='00000000feffeac0' > > I don't think we've got this to the point that I would call it a "good idea" yet. > It seems to me that we'd instead need to explore the route of doing our own formatting. > For that matter, do we have any assurance that the formatting of doubles on all C/C++ platforms is reproducible and clean? > > David > > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the source code. From dmitry.samersoff at oracle.com Sun May 11 19:12:35 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Sun, 11 May 2014 23:12:35 +0400 Subject: RFR (s) 8042661 : Enhance the fix for 8030011 to avoid warnings when using -Wstring-plus-int In-Reply-To: <536D1712.2090406@oracle.com> References: <536D1712.2090406@oracle.com> Message-ID: <536FCBA3.1090407@oracle.com> Alejandro, Looks good for me. (not an official reviewer) -Dmitry On 2014-05-09 21:57, Alejandro E Murillo wrote: > > Can I get a couple of reviews for this change? > > webrev: http://cr.openjdk.java.net/~amurillo/9/8042661/ > bug: https://bugs.openjdk.java.net/browse/JDK-8042661 > > Change is required to avoid warning when compiling with -Wstring-plus-int > tested with jprt and also on a standalone mac setup > > Thanks > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From david.holmes at oracle.com Mon May 12 07:20:38 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 May 2014 17:20:38 +1000 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <536C5498.80903@oracle.com> References: <531F05D3.6020008@oracle.com> <536C5498.80903@oracle.com> Message-ID: <53707646.9020805@oracle.com> For the Open record, the hotspot fix has an issue because FASTDEBUG_CFLAGS is unused on linux/bsd. David On 9/05/2014 2:07 PM, David Holmes wrote: > Hi Mike, > > This looks okay to me, but of course the proof is in the building and > testing. > > David > > On 4/05/2014 1:24 PM, Mike Duigou wrote: >> Hello; >> >> Finally getting back to this issue I have done some cleanup and >> adjusted the hotspot gcc.make files to use VARIANT rather than >> DEBUG_LEVEL. >> >> This version also add support for the "-fsanitize=undefined" undefined >> behaviour warnings feature when it is available (Clang and GCC 4.9). >> The code to emit the option has been added for Clang but I haven't yet >> added test for the option's availability under Clang so currently it >> will be enabled only for GCC. >> >> https://bugs.openjdk.java.net/browse/JDK-8032045 >> http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ >> >> Mike >> >> On Mar 11 2014, at 05:47 , Magnus Ihse Bursie >> wrote: >> >>> On 2014-03-11 00:49, Mike Duigou wrote: >>>> I have updated the patch to respond to Magnus's feedback and to >>>> accommodate intervening changes to the configure and hotspot make >>>> files. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>>> >>>> This version is, hopefully, almost ready to be pushed. >>> >>> I have only glanced at the hotspot build changes and can't really say >>> anything about them. The hotspot team still owns these; I'm cc:ing >>> them now. >>> >>> The top-level build changes looks fine. Thank you Mike for cleaning >>> things up! >>> >>> /Magnus >>> >>>> >>>> Mike >>>> >>>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>>> >>>>> Hello all; >>>>> >>>>> This issue is a followon to JDK-8030350 >>>>> (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced >>>>> the compiler warnings used for compiling native code. The proposed >>>>> changes principally impact the Linux platform. >>>>> >>>>> While 8030350 was focused on compiler warnings which did not impact >>>>> code generation, this changeset will, for some configurations, >>>>> change the native code generated and likely change performance. >>>>> These proposed option changes prevent specific types of relocation >>>>> table, stack and heap memory corruption in native code. Preventing >>>>> these types of memory corruption may be useful for finding certain >>>>> kinds of bugs though and do provide some minimal additional >>>>> protections against malicious attacks. They aren't, by any means, a >>>>> substitute for following appropriate secure coding guidelines. >>>>> >>>>> The rationale behind the implementation is as follows. For release >>>>> builds during the initial phase of JDK 9 I would like to enable >>>>> only compile time checks. This ends up being similar to the >>>>> warnings in JDK-8030350. These options have no runtime impact on >>>>> footprint or performance and very minimal additional compile time >>>>> cost while providing value. **Release builds are not expected to >>>>> see any performance or footprint change as a result of this >>>>> changeset** >>>>> >>>>> For fast debug builds we can enable linker protections (relro) and >>>>> static compile time bounds checks (FORTIFY_SOURCE=1). >>>>> FORTIFY_SOURCE=1 might be moved to the production builds as well >>>>> because it has no runtime cost or executable size cost. >>>>> >>>>> For slow debug builds we can enable full linker protection (at a >>>>> potential cost in startup time), runtime bounds checks and stack >>>>> protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely >>>>> enable -fprotect-stack-strong when available in GCC 4.9 >>>>> >>>>> The basis for enabling the additional protections in debug builds >>>>> is that it will help us find bugs in our native code and we aren't >>>>> as concerned in debug builds with footprint and performance. Since >>>>> many developers already do their personal builds in fastdebug or >>>>> slowdebug mode for testing this will provide good opportunity to >>>>> shake out any problems with the options while not impacting release >>>>> builds. Should we find that any of the options provide significant >>>>> value for their cost we can move them to fastdebug or release. If >>>>> any of the options prove too costly they can be demoted or removed >>>>> entirely. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>>> >>>>> Additional to enabling the various compiler options I attempted to >>>>> rationalize some of the skew between the various >>>>> hotspot/make/{platform}/makefiles/gcc.make files while avoiding >>>>> changing existing behaviour. I have also introduced the new -Og >>>>> "optimize for debugging" option and there are now an explicit >>>>> C{XX}_O_FLAG_DEBUG definitions to complement the >>>>> C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>>> >>>>> Thanks, >>>>> >>>>> Mike >>> >> From staffan.larsen at oracle.com Mon May 12 07:26:01 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 12 May 2014 09:26:01 +0200 Subject: Pointer formats, again... In-Reply-To: References: Message-ID: Just wondering if it is worth the time and energy to get the exact same formatting of pointers across all platforms. After all, they are details of the underlaying implementation and you wouldn?t normally compare output across platforms. Yes, I agree that ?0xnull? isn?t a pretty output, but we all understand what it means. To me at least, it is more important that the code looks good and is easy to read and write. /Staffan On 10 maj 2014, at 20:02, David Chase wrote: > Based on discussions for the recent port-to-mavericks+clang/format-fun patch, > it appears that wrapping every pointer in p2i(...) before printing is ugly. We'd like > to use %p, but cannot for customer-visible formatting because C++ was standardized > by a herd of cats. We can't use PTR_FORMAT for this (at least not yet) both because > the change would be reasonably large, and because there are places where PTR_FORMAT > is used to express an intent in the source code, and that are also customer-visible. > > Therefore, what we clearly need is ANOTHER FORMATTING MACRO. > > It would be the equivalent of (done differently with conditional inclusion) > > #if defined __GNUC__ > #define PS_ PTR_FMT %p > #else > #define PS_PTR_FMT 0x%p > #endif > > PS_PTR_FMT stands for Platform Specific PoinTeR ForMaT > > I'm not the least bit attached to the name, PSPF would make me just as (un)happy. > > This could not be used where the exact format of pointers matter, because > there is no width control, sometimes it might print "0xnull" (because C++ is Awe-Some) > and on windows it will print hex in uppercase. > > Here's my test program: > #include > int main(int argc, char ** argv) { > printf("%%p of NULL='%p'\n", (void *) 0); > printf("%%p of &main='%p'\n", &main); > printf("%%4p='%4p'\n", argv); > printf("%%16p='%16p'\n", argv); > printf("%%016p='%016p'\n", argv); > return 1; > } > > Mountain Lion: > (compilation warnings for %016p) > %p of NULL='0x0' > %p of &main='0x105783e60' > %4p='0x7fff5a47c9d0' > %16p=' 0x7fff5a47c9d0' > %016p='0x007fff5a47c9d0' > > Ubuntu32, 13.10: > (compilation warnings for %016p) > %p of NULL='(nil)' > %p of &main='0x804841d' > %4p='0xbfacd584' > %16p=' 0xbfacd584' > %016p='0x000000bfacd584' > > Ubuntu64, 14.04: > (compilation warnings for %016p) > %p of NULL='(nil)' > %p of &main='0x40052d' > %4p='0x7fff573e68d8' > %16p=' 0x7fff573e68d8' > %016p='0x007fff573e68d8' > > Solaris11: > %p of NULL='0' > %p of &main='80509a0' > %4p='feffeac0' > %16p=' feffeac0' > %016p='00000000feffeac0' > > I don't think we've got this to the point that I would call it a "good idea" yet. > It seems to me that we'd instead need to explore the route of doing our own formatting. > For that matter, do we have any assurance that the formatting of doubles on all C/C++ platforms is reproducible and clean? > > David > > > From david.holmes at oracle.com Mon May 12 07:35:15 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 May 2014 17:35:15 +1000 Subject: Pointer formats, again... In-Reply-To: References: Message-ID: <537079B3.6050103@oracle.com> Hi Staffan, On 12/05/2014 5:26 PM, Staffan Larsen wrote: > Just wondering if it is worth the time and energy to get the exact same formatting of pointers across all platforms. After all, they are details of the underlaying implementation and you wouldn?t normally compare output across platforms. Yes, I agree that ?0xnull? isn?t a pretty output, but we all understand what it means. To me at least, it is more important that the code looks good and is easy to read and write. I think the issue is with the use of printed pointer values in regular logging output. If that format changed in unexpected ways due to the use of %p then it could break a lot of external log parsers. The vicious circle here is that to get standardized output we can't use %p, but then we get the type mismatches that gcc warns about, which then requires the use of p2i and INTPTR_FORMAT which then raises the "why can't we just use %p?" question which ... :) The gcc warnings have forced action here (annoyingly so if you ask me) but we've known for years that this is a mess with no simple means of cleaning it up. David H. > /Staffan > > On 10 maj 2014, at 20:02, David Chase wrote: > >> Based on discussions for the recent port-to-mavericks+clang/format-fun patch, >> it appears that wrapping every pointer in p2i(...) before printing is ugly. We'd like >> to use %p, but cannot for customer-visible formatting because C++ was standardized >> by a herd of cats. We can't use PTR_FORMAT for this (at least not yet) both because >> the change would be reasonably large, and because there are places where PTR_FORMAT >> is used to express an intent in the source code, and that are also customer-visible. >> >> Therefore, what we clearly need is ANOTHER FORMATTING MACRO. >> >> It would be the equivalent of (done differently with conditional inclusion) >> >> #if defined __GNUC__ >> #define PS_ PTR_FMT %p >> #else >> #define PS_PTR_FMT 0x%p >> #endif >> >> PS_PTR_FMT stands for Platform Specific PoinTeR ForMaT >> >> I'm not the least bit attached to the name, PSPF would make me just as (un)happy. >> >> This could not be used where the exact format of pointers matter, because >> there is no width control, sometimes it might print "0xnull" (because C++ is Awe-Some) >> and on windows it will print hex in uppercase. >> >> Here's my test program: >> #include >> int main(int argc, char ** argv) { >> printf("%%p of NULL='%p'\n", (void *) 0); >> printf("%%p of &main='%p'\n", &main); >> printf("%%4p='%4p'\n", argv); >> printf("%%16p='%16p'\n", argv); >> printf("%%016p='%016p'\n", argv); >> return 1; >> } >> >> Mountain Lion: >> (compilation warnings for %016p) >> %p of NULL='0x0' >> %p of &main='0x105783e60' >> %4p='0x7fff5a47c9d0' >> %16p=' 0x7fff5a47c9d0' >> %016p='0x007fff5a47c9d0' >> >> Ubuntu32, 13.10: >> (compilation warnings for %016p) >> %p of NULL='(nil)' >> %p of &main='0x804841d' >> %4p='0xbfacd584' >> %16p=' 0xbfacd584' >> %016p='0x000000bfacd584' >> >> Ubuntu64, 14.04: >> (compilation warnings for %016p) >> %p of NULL='(nil)' >> %p of &main='0x40052d' >> %4p='0x7fff573e68d8' >> %16p=' 0x7fff573e68d8' >> %016p='0x007fff573e68d8' >> >> Solaris11: >> %p of NULL='0' >> %p of &main='80509a0' >> %4p='feffeac0' >> %16p=' feffeac0' >> %016p='00000000feffeac0' >> >> I don't think we've got this to the point that I would call it a "good idea" yet. >> It seems to me that we'd instead need to explore the route of doing our own formatting. >> For that matter, do we have any assurance that the formatting of doubles on all C/C++ platforms is reproducible and clean? >> >> David >> >> >> > From erik.osterlund at lnu.se Mon May 12 07:40:53 2014 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Mon, 12 May 2014 07:40:53 +0000 Subject: Return barriers Message-ID: <737189DB-F08C-429D-9C67-44F60921D333@lnu.se> Hello everyone, I'm wondering if it would be possible to add return barriers to hotspot and if so if it's a good idea? Coming from a GC background I see that response times could be decreased at no performance loss. I also have a wait-free GC implementation relying on them (however not without performance loss). Therefore, currently my implementation relies on JikesRVM since they have return barriers. Is there any good reason why this should not be implemented in hotspot? Regards, Erik ?sterlund From staffan.larsen at oracle.com Mon May 12 07:45:59 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 12 May 2014 09:45:59 +0200 Subject: Pointer formats, again... In-Reply-To: <537079B3.6050103@oracle.com> References: <537079B3.6050103@oracle.com> Message-ID: <976AC676-9EE2-45B2-87FB-7DB7276DE520@oracle.com> On 12 maj 2014, at 09:35, David Holmes wrote: > Hi Staffan, > > On 12/05/2014 5:26 PM, Staffan Larsen wrote: >> Just wondering if it is worth the time and energy to get the exact same formatting of pointers across all platforms. After all, they are details of the underlaying implementation and you wouldn?t normally compare output across platforms. Yes, I agree that ?0xnull? isn?t a pretty output, but we all understand what it means. To me at least, it is more important that the code looks good and is easy to read and write. > > I think the issue is with the use of printed pointer values in regular logging output. If that format changed in unexpected ways due to the use of %p then it could break a lot of external log parsers. I guess I don?t agree that this is a priority, I may be wrong. It?s just an inordinate amount of time spent. My $0.02, /Staffan > > The vicious circle here is that to get standardized output we can't use %p, but then we get the type mismatches that gcc warns about, which then requires the use of p2i and INTPTR_FORMAT which then raises the "why can't we just use %p?" question which ... :) > > The gcc warnings have forced action here (annoyingly so if you ask me) but we've known for years that this is a mess with no simple means of cleaning it up. > > David H. > >> /Staffan >> >> On 10 maj 2014, at 20:02, David Chase wrote: >> >>> Based on discussions for the recent port-to-mavericks+clang/format-fun patch, >>> it appears that wrapping every pointer in p2i(...) before printing is ugly. We'd like >>> to use %p, but cannot for customer-visible formatting because C++ was standardized >>> by a herd of cats. We can't use PTR_FORMAT for this (at least not yet) both because >>> the change would be reasonably large, and because there are places where PTR_FORMAT >>> is used to express an intent in the source code, and that are also customer-visible. >>> >>> Therefore, what we clearly need is ANOTHER FORMATTING MACRO. >>> >>> It would be the equivalent of (done differently with conditional inclusion) >>> >>> #if defined __GNUC__ >>> #define PS_ PTR_FMT %p >>> #else >>> #define PS_PTR_FMT 0x%p >>> #endif >>> >>> PS_PTR_FMT stands for Platform Specific PoinTeR ForMaT >>> >>> I'm not the least bit attached to the name, PSPF would make me just as (un)happy. >>> >>> This could not be used where the exact format of pointers matter, because >>> there is no width control, sometimes it might print "0xnull" (because C++ is Awe-Some) >>> and on windows it will print hex in uppercase. >>> >>> Here's my test program: >>> #include >>> int main(int argc, char ** argv) { >>> printf("%%p of NULL='%p'\n", (void *) 0); >>> printf("%%p of &main='%p'\n", &main); >>> printf("%%4p='%4p'\n", argv); >>> printf("%%16p='%16p'\n", argv); >>> printf("%%016p='%016p'\n", argv); >>> return 1; >>> } >>> >>> Mountain Lion: >>> (compilation warnings for %016p) >>> %p of NULL='0x0' >>> %p of &main='0x105783e60' >>> %4p='0x7fff5a47c9d0' >>> %16p=' 0x7fff5a47c9d0' >>> %016p='0x007fff5a47c9d0' >>> >>> Ubuntu32, 13.10: >>> (compilation warnings for %016p) >>> %p of NULL='(nil)' >>> %p of &main='0x804841d' >>> %4p='0xbfacd584' >>> %16p=' 0xbfacd584' >>> %016p='0x000000bfacd584' >>> >>> Ubuntu64, 14.04: >>> (compilation warnings for %016p) >>> %p of NULL='(nil)' >>> %p of &main='0x40052d' >>> %4p='0x7fff573e68d8' >>> %16p=' 0x7fff573e68d8' >>> %016p='0x007fff573e68d8' >>> >>> Solaris11: >>> %p of NULL='0' >>> %p of &main='80509a0' >>> %4p='feffeac0' >>> %16p=' feffeac0' >>> %016p='00000000feffeac0' >>> >>> I don't think we've got this to the point that I would call it a "good idea" yet. >>> It seems to me that we'd instead need to explore the route of doing our own formatting. >>> For that matter, do we have any assurance that the formatting of doubles on all C/C++ platforms is reproducible and clean? >>> >>> David >>> >>> >>> >> From vladimir.x.ivanov at oracle.com Mon May 12 11:19:04 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 12 May 2014 15:19:04 +0400 Subject: Lock-based vs lock-free MethodCounters initialization Message-ID: <5370AE28.90306@oracle.com> Hi, There's a possible memory leak when allocating MethodCounters concurrently [1], because there's no coordination between concurrent counters initialization actions. In constrast with MethodCounters, MethodData allocation is guarded by MethodData lock. I'm thinking about proper fix and considering 2 options: lock-based (reuse MethodData_lock or introduce MethodCounters_lock) and lock-free (do a CAS on _method_counters field) [3]. The problem with lock-based approach is that there's a possible deadlock with pending list lock (see JDK-6988439 [4] for details), so I need to check whether PLL is held and abort initialization in case it is (similar to what Method::build_interpreter_method_data does before acquiring the lock [2]). Lock-free solution doesn't suffer from deadlocks, but it can increase Metaspace fragmentation due to failed CAS updates and subsequent deallocations. I'm in favor of lock-free solution, but not sure about how serious fragmentation issue is in NPG world. What do you think? Thanks! Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l384 MethodCounters* Method::build_method_counters(Method* m, TRAPS) { methodHandle mh(m); ClassLoaderData* loader_data = mh->method_holder()->class_loader_data(); MethodCounters* counters = MethodCounters::allocate(loader_data, CHECK_NULL); if (mh->method_counters() == NULL) { mh->set_method_counters(counters); } else { MetadataFactory::free_metadata(loader_data, counters); } return mh->method_counters(); } [2] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l360 void Method::build_interpreter_method_data(methodHandle method, TRAPS) { // Do not profile method if current thread holds the pending list lock, // which avoids deadlock for acquiring the MethodData_lock. if (InstanceRefKlass::owns_pending_list_lock((JavaThread*)THREAD)) { return; } // Grab a lock here to prevent multiple // MethodData*s from being created. MutexLocker ml(MethodData_lock, THREAD); if (method->method_data() == NULL) { ClassLoaderData* loader_data = method->method_holder()->class_loader_data(); MethodData* method_data = MethodData::allocate(loader_data, method, CHECK); method->set_method_data(method_data); if (PrintMethodData && (Verbose || WizardMode)) { ResourceMark rm(THREAD); tty->print("build_interpreter_method_data for "); method->print_name(tty); tty->cr(); // At the end of the run, the MDO, full of data, will be dumped. } } } [3] Possible lock-free solution: MethodCounters* Method::build_method_counters(Method* m, TRAPS) { methodHandle mh(m); if (mh->method_counters() != NULL) return mh->method_counters(); ClassLoaderData* loader_data = mh->method_holder()->class_loader_data(); MethodCounters* counters = MethodCounters::allocate(loader_data, CHECK_NULL); MethodCounters* old = (MethodCounters*)Atomic::cmpxchg_ptr(counters, &_method_counters, NULL); if (old != NULL) { MetadataFactory::free_metadata(loader_data, counters); } return mh->method_counters(); } [4] https://bugs.openjdk.java.net/browse/JDK-6988439 From david.r.chase at oracle.com Mon May 12 11:42:12 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 12 May 2014 07:42:12 -0400 Subject: Lock-based vs lock-free MethodCounters initialization In-Reply-To: <5370AE28.90306@oracle.com> References: <5370AE28.90306@oracle.com> Message-ID: <62A6F221-1736-41D5-AB5C-8E00D0723F28@oracle.com> On 2014-05-12, at 7:19 AM, Vladimir Ivanov wrote: > Lock-free solution doesn't suffer from deadlocks, but it can increase Metaspace fragmentation due to failed CAS updates and subsequent deallocations. > > I'm in favor of lock-free solution, but not sure about how serious fragmentation issue is in NPG world. What do you think? I would try lock-free, because fragmentation is not as bad as deadlock. Would be interesting to define a benchmark/test that ought to thrash that lock to see if this turns out to matter anyhow. David From mikael.gerdin at oracle.com Mon May 12 13:09:04 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 12 May 2014 15:09:04 +0200 Subject: Lock-based vs lock-free MethodCounters initialization In-Reply-To: <5370AE28.90306@oracle.com> References: <5370AE28.90306@oracle.com> Message-ID: <11027150.GgWXGtHOP1@mgerdin03> Hi Vladimir, On Monday 12 May 2014 15.19.04 Vladimir Ivanov wrote: > Hi, > > There's a possible memory leak when allocating MethodCounters > concurrently [1], because there's no coordination between concurrent > counters initialization actions. In constrast with MethodCounters, > MethodData allocation is guarded by MethodData lock. > > I'm thinking about proper fix and considering 2 options: lock-based > (reuse MethodData_lock or introduce MethodCounters_lock) and lock-free > (do a CAS on _method_counters field) [3]. > > The problem with lock-based approach is that there's a possible deadlock > with pending list lock (see JDK-6988439 [4] for details), so I need to > check whether PLL is held and abort initialization in case it is > (similar to what Method::build_interpreter_method_data does before > acquiring the lock [2]). > > Lock-free solution doesn't suffer from deadlocks, but it can increase > Metaspace fragmentation due to failed CAS updates and subsequent > deallocations. > > I'm in favor of lock-free solution, but not sure about how serious > fragmentation issue is in NPG world. What do you think? 1. Are MethodCounters objects large enough to fit on the per-metaspace block free list? See: MetaWord* BlockFreelist::get_block(size_t word_size) { (...) if (word_size < TreeChunk >::min_size()) { // Dark matter. Too small for dictionary. return NULL; } 2. If the above is true, are all MethodCounters the same size? I know how you guys love your variable-sized objects :) If this is not true then we could encounter some severe fragmentation in Metaspace. There are already several issues with fragmentation and waste and I would prefer if we could avoid adding to the problem. /Mikael > > Thanks! > > Best regards, > Vladimir Ivanov > > [1] > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/m > ethod.cpp#l384 > > MethodCounters* Method::build_method_counters(Method* m, TRAPS) { > methodHandle mh(m); > ClassLoaderData* loader_data = mh->method_holder()->class_loader_data(); > MethodCounters* counters = MethodCounters::allocate(loader_data, > CHECK_NULL); > if (mh->method_counters() == NULL) { > mh->set_method_counters(counters); > } else { > MetadataFactory::free_metadata(loader_data, counters); > } > return mh->method_counters(); > } > > [2] > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/m > ethod.cpp#l360 > > void Method::build_interpreter_method_data(methodHandle method, TRAPS) { > // Do not profile method if current thread holds the pending list lock, > // which avoids deadlock for acquiring the MethodData_lock. > if (InstanceRefKlass::owns_pending_list_lock((JavaThread*)THREAD)) { > return; > } > > // Grab a lock here to prevent multiple > // MethodData*s from being created. > MutexLocker ml(MethodData_lock, THREAD); > if (method->method_data() == NULL) { > ClassLoaderData* loader_data = > method->method_holder()->class_loader_data(); > MethodData* method_data = MethodData::allocate(loader_data, method, > CHECK); > method->set_method_data(method_data); > if (PrintMethodData && (Verbose || WizardMode)) { > ResourceMark rm(THREAD); > tty->print("build_interpreter_method_data for "); > method->print_name(tty); > tty->cr(); > // At the end of the run, the MDO, full of data, will be dumped. > } > } > } > > [3] Possible lock-free solution: > MethodCounters* Method::build_method_counters(Method* m, TRAPS) { > methodHandle mh(m); > if (mh->method_counters() != NULL) return mh->method_counters(); > ClassLoaderData* loader_data = mh->method_holder()->class_loader_data(); > MethodCounters* counters = MethodCounters::allocate(loader_data, > CHECK_NULL); > MethodCounters* old = (MethodCounters*)Atomic::cmpxchg_ptr(counters, > &_method_counters, NULL); > if (old != NULL) { > MetadataFactory::free_metadata(loader_data, counters); > } > return mh->method_counters(); > } > > [4] https://bugs.openjdk.java.net/browse/JDK-6988439 From vladimir.x.ivanov at oracle.com Mon May 12 15:09:28 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 12 May 2014 19:09:28 +0400 Subject: Lock-based vs lock-free MethodCounters initialization In-Reply-To: <11027150.GgWXGtHOP1@mgerdin03> References: <5370AE28.90306@oracle.com> <11027150.GgWXGtHOP1@mgerdin03> Message-ID: <5370E428.7030404@oracle.com> On 5/12/14 5:09 PM, Mikael Gerdin wrote: >> I'm in favor of lock-free solution, but not sure about how serious >> fragmentation issue is in NPG world. What do you think? > > 1. Are MethodCounters objects large enough to fit on the per-metaspace block > free list? See: > MetaWord* BlockFreelist::get_block(size_t word_size) { > (...) > if (word_size < TreeChunk >::min_size()) { > // Dark matter. Too small for dictionary. > return NULL; > } MethodCounters have fixed size, but it's too small - MethodCounters::size() reports 4 words for Tiered in 64-bit VM and TreeChunk::min_size() is 12 words. > 2. If the above is true, are all MethodCounters the same size? I know how you > guys love your variable-sized objects :) > > > If this is not true then we could encounter some severe fragmentation in > Metaspace. There are already several issues with fragmentation and waste and I > would prefer if we could avoid adding to the problem. I'll choose lock-based solution then. Thanks for the info. Best regards, Vladimir Ivanov > > /Mikael > > >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov >> >> [1] >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/m >> ethod.cpp#l384 >> >> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { >> methodHandle mh(m); >> ClassLoaderData* loader_data = mh->method_holder()->class_loader_data(); >> MethodCounters* counters = MethodCounters::allocate(loader_data, >> CHECK_NULL); >> if (mh->method_counters() == NULL) { >> mh->set_method_counters(counters); >> } else { >> MetadataFactory::free_metadata(loader_data, counters); >> } >> return mh->method_counters(); >> } >> >> [2] >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/m >> ethod.cpp#l360 >> >> void Method::build_interpreter_method_data(methodHandle method, TRAPS) { >> // Do not profile method if current thread holds the pending list lock, >> // which avoids deadlock for acquiring the MethodData_lock. >> if (InstanceRefKlass::owns_pending_list_lock((JavaThread*)THREAD)) { >> return; >> } >> >> // Grab a lock here to prevent multiple >> // MethodData*s from being created. >> MutexLocker ml(MethodData_lock, THREAD); >> if (method->method_data() == NULL) { >> ClassLoaderData* loader_data = >> method->method_holder()->class_loader_data(); >> MethodData* method_data = MethodData::allocate(loader_data, method, >> CHECK); >> method->set_method_data(method_data); >> if (PrintMethodData && (Verbose || WizardMode)) { >> ResourceMark rm(THREAD); >> tty->print("build_interpreter_method_data for "); >> method->print_name(tty); >> tty->cr(); >> // At the end of the run, the MDO, full of data, will be dumped. >> } >> } >> } >> >> [3] Possible lock-free solution: >> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { >> methodHandle mh(m); >> if (mh->method_counters() != NULL) return mh->method_counters(); >> ClassLoaderData* loader_data = mh->method_holder()->class_loader_data(); >> MethodCounters* counters = MethodCounters::allocate(loader_data, >> CHECK_NULL); >> MethodCounters* old = (MethodCounters*)Atomic::cmpxchg_ptr(counters, >> &_method_counters, NULL); >> if (old != NULL) { >> MetadataFactory::free_metadata(loader_data, counters); >> } >> return mh->method_counters(); >> } >> >> [4] https://bugs.openjdk.java.net/browse/JDK-6988439 > From jiangli.zhou at oracle.com Mon May 12 16:51:41 2014 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 12 May 2014 09:51:41 -0700 Subject: Lock-based vs lock-free MethodCounters initialization In-Reply-To: <5370AE28.90306@oracle.com> References: <5370AE28.90306@oracle.com> Message-ID: <5370FC1D.4040806@oracle.com> Hi Vladimir, I worked on MethodCounters allocation and planned to look into the possible memory leak at some point, but haven't gotten to it. Thanks for looking into this! The possible contention of a MethodCounters allocation only happens when the method is being executed the first time. I did some quick profiling and determined that it was rare. If that was the case, metspace fragmentation due to deallocations of MethodCounters followed by failed CAS would also be uncommon. The lock-free based solution seems to be the safer and more efficient one to go for. I see David and Mikael already responded to your question already. Both of them provided very good information. Maybe it's worth to do some profiling first? Thanks, Jiangli On 05/12/2014 04:19 AM, Vladimir Ivanov wrote: > Hi, > > There's a possible memory leak when allocating MethodCounters > concurrently [1], because there's no coordination between concurrent > counters initialization actions. In constrast with MethodCounters, > MethodData allocation is guarded by MethodData lock. > > I'm thinking about proper fix and considering 2 options: lock-based > (reuse MethodData_lock or introduce MethodCounters_lock) and lock-free > (do a CAS on _method_counters field) [3]. > > The problem with lock-based approach is that there's a possible > deadlock with pending list lock (see JDK-6988439 [4] for details), so > I need to check whether PLL is held and abort initialization in case > it is (similar to what Method::build_interpreter_method_data does > before acquiring the lock [2]). > > Lock-free solution doesn't suffer from deadlocks, but it can increase > Metaspace fragmentation due to failed CAS updates and subsequent > deallocations. > > I'm in favor of lock-free solution, but not sure about how serious > fragmentation issue is in NPG world. What do you think? > > Thanks! > > Best regards, > Vladimir Ivanov > > [1] > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l384 > > MethodCounters* Method::build_method_counters(Method* m, TRAPS) { > methodHandle mh(m); > ClassLoaderData* loader_data = > mh->method_holder()->class_loader_data(); > MethodCounters* counters = MethodCounters::allocate(loader_data, > CHECK_NULL); > if (mh->method_counters() == NULL) { > mh->set_method_counters(counters); > } else { > MetadataFactory::free_metadata(loader_data, counters); > } > return mh->method_counters(); > } > > [2] > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l360 > > void Method::build_interpreter_method_data(methodHandle method, TRAPS) { > // Do not profile method if current thread holds the pending list lock, > // which avoids deadlock for acquiring the MethodData_lock. > if (InstanceRefKlass::owns_pending_list_lock((JavaThread*)THREAD)) { > return; > } > > // Grab a lock here to prevent multiple > // MethodData*s from being created. > MutexLocker ml(MethodData_lock, THREAD); > if (method->method_data() == NULL) { > ClassLoaderData* loader_data = > method->method_holder()->class_loader_data(); > MethodData* method_data = MethodData::allocate(loader_data, > method, CHECK); > method->set_method_data(method_data); > if (PrintMethodData && (Verbose || WizardMode)) { > ResourceMark rm(THREAD); > tty->print("build_interpreter_method_data for "); > method->print_name(tty); > tty->cr(); > // At the end of the run, the MDO, full of data, will be dumped. > } > } > } > > [3] Possible lock-free solution: > MethodCounters* Method::build_method_counters(Method* m, TRAPS) { > methodHandle mh(m); > if (mh->method_counters() != NULL) return mh->method_counters(); > ClassLoaderData* loader_data = > mh->method_holder()->class_loader_data(); > MethodCounters* counters = MethodCounters::allocate(loader_data, > CHECK_NULL); > MethodCounters* old = (MethodCounters*)Atomic::cmpxchg_ptr(counters, > &_method_counters, NULL); > if (old != NULL) { > MetadataFactory::free_metadata(loader_data, counters); > } > return mh->method_counters(); > } > > [4] https://bugs.openjdk.java.net/browse/JDK-6988439 From vladimir.x.ivanov at oracle.com Mon May 12 18:56:01 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 12 May 2014 22:56:01 +0400 Subject: Lock-based vs lock-free MethodCounters initialization In-Reply-To: <5370FC1D.4040806@oracle.com> References: <5370AE28.90306@oracle.com> <5370FC1D.4040806@oracle.com> Message-ID: <53711941.70500@oracle.com> Thanks, Jiangli & David. Yes, I agree that more understanding of the behavior will be very helpful. I'll try to come up with a stress test and run some benchmarks to see how common is the contention during allocation. Best regards, Vladimir Ivanov On 5/12/14 8:51 PM, Jiangli Zhou wrote: > Hi Vladimir, > > I worked on MethodCounters allocation and planned to look into the > possible memory leak at some point, but haven't gotten to it. Thanks for > looking into this! The possible contention of a MethodCounters > allocation only happens when the method is being executed the first > time. I did some quick profiling and determined that it was rare. If > that was the case, metspace fragmentation due to deallocations of > MethodCounters followed by failed CAS would also be uncommon. The > lock-free based solution seems to be the safer and more efficient one to > go for. > > I see David and Mikael already responded to your question already. Both > of them provided very good information. Maybe it's worth to do some > profiling first? > > Thanks, > > Jiangli > > On 05/12/2014 04:19 AM, Vladimir Ivanov wrote: >> Hi, >> >> There's a possible memory leak when allocating MethodCounters >> concurrently [1], because there's no coordination between concurrent >> counters initialization actions. In constrast with MethodCounters, >> MethodData allocation is guarded by MethodData lock. >> >> I'm thinking about proper fix and considering 2 options: lock-based >> (reuse MethodData_lock or introduce MethodCounters_lock) and lock-free >> (do a CAS on _method_counters field) [3]. >> >> The problem with lock-based approach is that there's a possible >> deadlock with pending list lock (see JDK-6988439 [4] for details), so >> I need to check whether PLL is held and abort initialization in case >> it is (similar to what Method::build_interpreter_method_data does >> before acquiring the lock [2]). >> >> Lock-free solution doesn't suffer from deadlocks, but it can increase >> Metaspace fragmentation due to failed CAS updates and subsequent >> deallocations. >> >> I'm in favor of lock-free solution, but not sure about how serious >> fragmentation issue is in NPG world. What do you think? >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov >> >> [1] >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l384 >> >> >> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { >> methodHandle mh(m); >> ClassLoaderData* loader_data = >> mh->method_holder()->class_loader_data(); >> MethodCounters* counters = MethodCounters::allocate(loader_data, >> CHECK_NULL); >> if (mh->method_counters() == NULL) { >> mh->set_method_counters(counters); >> } else { >> MetadataFactory::free_metadata(loader_data, counters); >> } >> return mh->method_counters(); >> } >> >> [2] >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l360 >> >> >> void Method::build_interpreter_method_data(methodHandle method, TRAPS) { >> // Do not profile method if current thread holds the pending list lock, >> // which avoids deadlock for acquiring the MethodData_lock. >> if (InstanceRefKlass::owns_pending_list_lock((JavaThread*)THREAD)) { >> return; >> } >> >> // Grab a lock here to prevent multiple >> // MethodData*s from being created. >> MutexLocker ml(MethodData_lock, THREAD); >> if (method->method_data() == NULL) { >> ClassLoaderData* loader_data = >> method->method_holder()->class_loader_data(); >> MethodData* method_data = MethodData::allocate(loader_data, >> method, CHECK); >> method->set_method_data(method_data); >> if (PrintMethodData && (Verbose || WizardMode)) { >> ResourceMark rm(THREAD); >> tty->print("build_interpreter_method_data for "); >> method->print_name(tty); >> tty->cr(); >> // At the end of the run, the MDO, full of data, will be dumped. >> } >> } >> } >> >> [3] Possible lock-free solution: >> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { >> methodHandle mh(m); >> if (mh->method_counters() != NULL) return mh->method_counters(); >> ClassLoaderData* loader_data = >> mh->method_holder()->class_loader_data(); >> MethodCounters* counters = MethodCounters::allocate(loader_data, >> CHECK_NULL); >> MethodCounters* old = (MethodCounters*)Atomic::cmpxchg_ptr(counters, >> &_method_counters, NULL); >> if (old != NULL) { >> MetadataFactory::free_metadata(loader_data, counters); >> } >> return mh->method_counters(); >> } >> >> [4] https://bugs.openjdk.java.net/browse/JDK-6988439 > From john.r.rose at oracle.com Mon May 12 19:17:24 2014 From: john.r.rose at oracle.com (John Rose) Date: Mon, 12 May 2014 12:17:24 -0700 Subject: Lock-based vs lock-free MethodCounters initialization In-Reply-To: <53711941.70500@oracle.com> References: <5370AE28.90306@oracle.com> <5370FC1D.4040806@oracle.com> <53711941.70500@oracle.com> Message-ID: <18082ADD-59E0-4916-BBB1-13AC9B4F37E5@oracle.com> Lock-free should be our default if we think contention is rare, for reasons already given. BUT, we should back up our intuition with a measurement, such as a pair of counters that track successful vs. failed CAS events. We can then detect unexpectedly frequent failed CAS events and treat them as a possible bug. Since we are noticing occasional fragmentation problems, we can also correlate them to those new counters. If fragmentation is a risk, we need to detect it routinely and mechanically. Mikael G., which metrics do we use to detect excessive holes in the metaspace heap? ? John On May 12, 2014, at 11:56 AM, Vladimir Ivanov wrote: > Thanks, Jiangli & David. > > Yes, I agree that more understanding of the behavior will be very helpful. I'll try to come up with a stress test and run some benchmarks to see how common is the contention during allocation. > > Best regards, > Vladimir Ivanov > > On 5/12/14 8:51 PM, Jiangli Zhou wrote: >> Hi Vladimir, >> >> I worked on MethodCounters allocation and planned to look into the >> possible memory leak at some point, but haven't gotten to it. Thanks for >> looking into this! The possible contention of a MethodCounters >> allocation only happens when the method is being executed the first >> time. I did some quick profiling and determined that it was rare. If >> that was the case, metspace fragmentation due to deallocations of >> MethodCounters followed by failed CAS would also be uncommon. The >> lock-free based solution seems to be the safer and more efficient one to >> go for. >> >> I see David and Mikael already responded to your question already. Both >> of them provided very good information. Maybe it's worth to do some >> profiling first? >> >> Thanks, >> >> Jiangli >> >> On 05/12/2014 04:19 AM, Vladimir Ivanov wrote: >>> Hi, >>> >>> There's a possible memory leak when allocating MethodCounters >>> concurrently [1], because there's no coordination between concurrent >>> counters initialization actions. In constrast with MethodCounters, >>> MethodData allocation is guarded by MethodData lock. >>> >>> I'm thinking about proper fix and considering 2 options: lock-based >>> (reuse MethodData_lock or introduce MethodCounters_lock) and lock-free >>> (do a CAS on _method_counters field) [3]. >>> >>> The problem with lock-based approach is that there's a possible >>> deadlock with pending list lock (see JDK-6988439 [4] for details), so >>> I need to check whether PLL is held and abort initialization in case >>> it is (similar to what Method::build_interpreter_method_data does >>> before acquiring the lock [2]). >>> >>> Lock-free solution doesn't suffer from deadlocks, but it can increase >>> Metaspace fragmentation due to failed CAS updates and subsequent >>> deallocations. >>> >>> I'm in favor of lock-free solution, but not sure about how serious >>> fragmentation issue is in NPG world. What do you think? >>> >>> Thanks! >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] >>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l384 >>> >>> >>> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { >>> methodHandle mh(m); >>> ClassLoaderData* loader_data = >>> mh->method_holder()->class_loader_data(); >>> MethodCounters* counters = MethodCounters::allocate(loader_data, >>> CHECK_NULL); >>> if (mh->method_counters() == NULL) { >>> mh->set_method_counters(counters); >>> } else { >>> MetadataFactory::free_metadata(loader_data, counters); >>> } >>> return mh->method_counters(); >>> } >>> >>> [2] >>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oops/method.cpp#l360 >>> >>> >>> void Method::build_interpreter_method_data(methodHandle method, TRAPS) { >>> // Do not profile method if current thread holds the pending list lock, >>> // which avoids deadlock for acquiring the MethodData_lock. >>> if (InstanceRefKlass::owns_pending_list_lock((JavaThread*)THREAD)) { >>> return; >>> } >>> >>> // Grab a lock here to prevent multiple >>> // MethodData*s from being created. >>> MutexLocker ml(MethodData_lock, THREAD); >>> if (method->method_data() == NULL) { >>> ClassLoaderData* loader_data = >>> method->method_holder()->class_loader_data(); >>> MethodData* method_data = MethodData::allocate(loader_data, >>> method, CHECK); >>> method->set_method_data(method_data); >>> if (PrintMethodData && (Verbose || WizardMode)) { >>> ResourceMark rm(THREAD); >>> tty->print("build_interpreter_method_data for "); >>> method->print_name(tty); >>> tty->cr(); >>> // At the end of the run, the MDO, full of data, will be dumped. >>> } >>> } >>> } >>> >>> [3] Possible lock-free solution: >>> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { >>> methodHandle mh(m); >>> if (mh->method_counters() != NULL) return mh->method_counters(); >>> ClassLoaderData* loader_data = >>> mh->method_holder()->class_loader_data(); >>> MethodCounters* counters = MethodCounters::allocate(loader_data, >>> CHECK_NULL); >>> MethodCounters* old = (MethodCounters*)Atomic::cmpxchg_ptr(counters, >>> &_method_counters, NULL); >>> if (old != NULL) { >>> MetadataFactory::free_metadata(loader_data, counters); >>> } >>> return mh->method_counters(); >>> } >>> >>> [4] https://bugs.openjdk.java.net/browse/JDK-6988439 >> From coleen.phillimore at oracle.com Tue May 13 00:19:15 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 12 May 2014 20:19:15 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking Message-ID: <53716503.7060503@oracle.com> Summary: Only prune metaspace virtual spaces at safepoint so walking them is safe outside a safepoint. Walking class loader data graph for contains is really slow for applications like nashorn that has a lot of class loader data. Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ bug link https://bugs.openjdk.java.net/browse/JDK-8038212 Marcus L. tested it with nashorn tests. Thanks, Coleen From mikael.gerdin at oracle.com Tue May 13 09:37:52 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 13 May 2014 11:37:52 +0200 Subject: Lock-based vs lock-free MethodCounters initialization In-Reply-To: <18082ADD-59E0-4916-BBB1-13AC9B4F37E5@oracle.com> References: <5370AE28.90306@oracle.com> <53711941.70500@oracle.com> <18082ADD-59E0-4916-BBB1-13AC9B4F37E5@oracle.com> Message-ID: <1783189.DkMKGRp4Aq@mgerdin03> On Monday 12 May 2014 12.17.24 John Rose wrote: > Lock-free should be our default if we think contention is rare, for reasons > already given. > > BUT, we should back up our intuition with a measurement, such as a pair of > counters that track successful vs. failed CAS events. We can then detect > unexpectedly frequent failed CAS events and treat them as a possible bug. > > Since we are noticing occasional fragmentation problems, we can also > correlate them to those new counters. If fragmentation is a risk, we need > to detect it routinely and mechanically. Mikael G., which metrics do we > use to detect excessive holes in the metaspace heap? Unfortunately it's a bit hard to measure fragmentation and waste in metaspace. We do have some trace events for reporting the amount of memory used, committed and reserved for metaspace. Currently we don't keep track of the amount of dark matter memory (deallocations which do not fit in the free list) and we don't have any good way to expose information about the contents block freelists (which is where memory returned by explicit deallocations end up). /Mikael > > ? John > > On May 12, 2014, at 11:56 AM, Vladimir Ivanov wrote: > > Thanks, Jiangli & David. > > > > Yes, I agree that more understanding of the behavior will be very helpful. > > I'll try to come up with a stress test and run some benchmarks to see how > > common is the contention during allocation. > > > > Best regards, > > Vladimir Ivanov > > > > On 5/12/14 8:51 PM, Jiangli Zhou wrote: > >> Hi Vladimir, > >> > >> I worked on MethodCounters allocation and planned to look into the > >> possible memory leak at some point, but haven't gotten to it. Thanks for > >> looking into this! The possible contention of a MethodCounters > >> allocation only happens when the method is being executed the first > >> time. I did some quick profiling and determined that it was rare. If > >> that was the case, metspace fragmentation due to deallocations of > >> MethodCounters followed by failed CAS would also be uncommon. The > >> lock-free based solution seems to be the safer and more efficient one to > >> go for. > >> > >> I see David and Mikael already responded to your question already. Both > >> of them provided very good information. Maybe it's worth to do some > >> profiling first? > >> > >> Thanks, > >> > >> Jiangli > >> > >> On 05/12/2014 04:19 AM, Vladimir Ivanov wrote: > >>> Hi, > >>> > >>> There's a possible memory leak when allocating MethodCounters > >>> concurrently [1], because there's no coordination between concurrent > >>> counters initialization actions. In constrast with MethodCounters, > >>> MethodData allocation is guarded by MethodData lock. > >>> > >>> I'm thinking about proper fix and considering 2 options: lock-based > >>> (reuse MethodData_lock or introduce MethodCounters_lock) and lock-free > >>> (do a CAS on _method_counters field) [3]. > >>> > >>> The problem with lock-based approach is that there's a possible > >>> deadlock with pending list lock (see JDK-6988439 [4] for details), so > >>> I need to check whether PLL is held and abort initialization in case > >>> it is (similar to what Method::build_interpreter_method_data does > >>> before acquiring the lock [2]). > >>> > >>> Lock-free solution doesn't suffer from deadlocks, but it can increase > >>> Metaspace fragmentation due to failed CAS updates and subsequent > >>> deallocations. > >>> > >>> I'm in favor of lock-free solution, but not sure about how serious > >>> fragmentation issue is in NPG world. What do you think? > >>> > >>> Thanks! > >>> > >>> Best regards, > >>> Vladimir Ivanov > >>> > >>> [1] > >>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oo > >>> ps/method.cpp#l384 > >>> > >>> > >>> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { > >>> > >>> methodHandle mh(m); > >>> ClassLoaderData* loader_data = > >>> > >>> mh->method_holder()->class_loader_data(); > >>> > >>> MethodCounters* counters = MethodCounters::allocate(loader_data, > >>> > >>> CHECK_NULL); > >>> > >>> if (mh->method_counters() == NULL) { > >>> > >>> mh->set_method_counters(counters); > >>> > >>> } else { > >>> > >>> MetadataFactory::free_metadata(loader_data, counters); > >>> > >>> } > >>> return mh->method_counters(); > >>> > >>> } > >>> > >>> [2] > >>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/tip/src/share/vm/oo > >>> ps/method.cpp#l360 > >>> > >>> > >>> void Method::build_interpreter_method_data(methodHandle method, TRAPS) { > >>> > >>> // Do not profile method if current thread holds the pending list lock, > >>> // which avoids deadlock for acquiring the MethodData_lock. > >>> if (InstanceRefKlass::owns_pending_list_lock((JavaThread*)THREAD)) { > >>> > >>> return; > >>> > >>> } > >>> > >>> // Grab a lock here to prevent multiple > >>> // MethodData*s from being created. > >>> MutexLocker ml(MethodData_lock, THREAD); > >>> if (method->method_data() == NULL) { > >>> > >>> ClassLoaderData* loader_data = > >>> > >>> method->method_holder()->class_loader_data(); > >>> > >>> MethodData* method_data = MethodData::allocate(loader_data, > >>> > >>> method, CHECK); > >>> > >>> method->set_method_data(method_data); > >>> if (PrintMethodData && (Verbose || WizardMode)) { > >>> > >>> ResourceMark rm(THREAD); > >>> tty->print("build_interpreter_method_data for "); > >>> method->print_name(tty); > >>> tty->cr(); > >>> // At the end of the run, the MDO, full of data, will be dumped. > >>> > >>> } > >>> > >>> } > >>> > >>> } > >>> > >>> [3] Possible lock-free solution: > >>> MethodCounters* Method::build_method_counters(Method* m, TRAPS) { > >>> > >>> methodHandle mh(m); > >>> if (mh->method_counters() != NULL) return mh->method_counters(); > >>> ClassLoaderData* loader_data = > >>> > >>> mh->method_holder()->class_loader_data(); > >>> > >>> MethodCounters* counters = MethodCounters::allocate(loader_data, > >>> > >>> CHECK_NULL); > >>> > >>> MethodCounters* old = (MethodCounters*)Atomic::cmpxchg_ptr(counters, > >>> > >>> &_method_counters, NULL); > >>> > >>> if (old != NULL) { > >>> > >>> MetadataFactory::free_metadata(loader_data, counters); > >>> > >>> } > >>> return mh->method_counters(); > >>> > >>> } > >>> > >>> [4] https://bugs.openjdk.java.net/browse/JDK-6988439 From mikael.gerdin at oracle.com Tue May 13 10:33:59 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 13 May 2014 12:33:59 +0200 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <53716503.7060503@oracle.com> References: <53716503.7060503@oracle.com> Message-ID: <8288203.XBo0QC8Y2f@mgerdin03> Hi Coleen, On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: > Summary: Only prune metaspace virtual spaces at safepoint so walking > them is safe outside a safepoint. > > Walking class loader data graph for contains is really slow for > applications like nashorn that has a lot of class loader data. > > Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran > > open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ > bug link https://bugs.openjdk.java.net/browse/JDK-8038212 Thanks for taking care of this fix. I think the idea of deferring CLDG::purge to the next safepoint cleanup is sound. I like the removal of CLDG::contains since it makes more sense to ask Metaspace straight up. I have a small question about the test you added: 3798 static void test_contains() { 3799 // Test that all the methods in the CLDG are contained 3800 ClassLoaderDataGraph::methods_do(assert_contains); 3801 } Have we always allocated methods at this point in the VM startup? Should we also test CLDG::classes_do to make sure it works for the compressed class space? /Mikael > > Marcus L. tested it with nashorn tests. > > Thanks, > Coleen From stefan.johansson at oracle.com Tue May 13 12:31:10 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 13 May 2014 14:31:10 +0200 Subject: RFR: 8027915: TestParallelHeapSizeFlags fails with unexpected heap size on sparcv9 In-Reply-To: <536BA420.5000408@oracle.com> References: <5368A0D6.5050609@oracle.com> <5368C396.4020206@oracle.com> <536BA420.5000408@oracle.com> Message-ID: <5372108E.7070508@oracle.com> On 2014-05-08 17:34, Erik Helin wrote: > Hi Stefan, > > thanks for your thorough review! Please see new webrev at: > http://cr.openjdk.java.net/~ehelin/8027915/webrev.01/ > > and incremental webrev at: > http://cr.openjdk.java.net/~ehelin/8027915/webrev.00-01/ > > and comments inline! > > On 2014-05-06 13:12, Stefan Johansson wrote: >> Hi Erik, >> >> Thanks for making page_size_for_region more strict. A few comments: >> >> src/share/vm/runtime/os.cpp >> >> With this fix page_size_for_region doesn't care about alignment at all >> (old version wasn't much better) and to me that feels wrong. I would >> expect to get a page size that the given region sizes is aligned to, but >> I might be missing some use case where that isn't needed. > > Agree, page_size_for_region should check alignment for large pages. > Added fix and regression tests. Good fix. Regarding the testing, I looked at the code and the _page_sizes array is public. You could start of the testing with recording all "real" values and then set up an array that allows more testing. It would be nice to have a case in the alignment test that makes sure for example that for a 10M region you can get 2M pages but not 4M pages. If doing this you need to make sure the values in the _page_sizes array are set back to normal after the test. This would also allow the test to run even if UseLargePages isn't set. > > On 2014-05-06 13:12, Stefan Johansson wrote: >> src/share/vm/gc_implementation/parallelScavenge/generationSizer.cpp >> >> For the heap we want both min and max to be a multiple of the page size >> so we shoud call page_size_for_region for both _min_heap_byte_size and >> _max_heap_byte_size and use the smallest of the two returned page sizes. > > Agree, fixed. Great! > > On 2014-05-06 13:12, Stefan Johansson wrote: >> src/share/vm/memory/heap.cpp >> >> The current patch will not change the default behavior so I'm fine with >> just calling page_size_for_region once using reserved_size, but someone >> with more insight might know if we should handle it the same way as the >> generationSizer and use both committed and reserved. > > Since I'm not sure how the compiler guys want to handle this, I opted > for the safe alternative and updated the patch to be backwards > compatible with the old code (except slightly more strict). I agree that this is probably the correct solution, but it would be nice to hear what the compiler guys think. As you say, this is a little more strict and there might be cases where we now stop getting large pages where we used to before. To summarize, I think the changes are good but wouldn't mind some improved tests for it. Stefan > > Thanks, > Erik > >> Thanks, >> Stefan >> >> >> On 2014-05-06 10:44, Erik Helin wrote: >>> Hi all, >>> >>> this patch changes >>> os::page_size_for_region(min_size, max_size, min_pages) to always >>> guarantee that the returned page size <= max_size / min_pages. Prior >>> to this patch, page_size_for_region could return a page size larger >>> than max_size / min_pages. This can cause some unexpected behavior >>> for code using page_size_for_region, such as the following regression >>> tests: >>> >>> http://cr.openjdk.java.net/~ehelin/8027915/regression-test/ >>> >>> As the test shows, if you have a 2 MB region and you must use at least >>> 2 pages, you would still get back 2 MB as the page size on a Linux >>> machine that supports 2 MB large pages. The reason for this is >>> explained in a comment above the function in os.hpp: >>> >>> // The current implementation ignores min_pages if a larger page >>> // size is an exact multiple of both region_min_size and >>> // region_max_size. This allows larger pages to be used when doing >>> // so would not cause fragmentation; in particular, a single page can >>> // be used when region_min_size == region_max_size == a supported page >>> // size. >>> >>> Reducing fragmentation of large pages is good but the varying return >>> value makes it very hard to depend on os::page_size_for_region in >>> calculations (for example when sizing the heap). >>> >>> This patch removes the fragmentation prevention mechanism, simplifies >>> the function and adds a regression tests: >>> >>> http://cr.openjdk.java.net/~ehelin/8027915/webrev.00/ >>> >>> Testing: >>> - JPRT >>> >>> Thanks, >>> Erik >> From coleen.phillimore at oracle.com Tue May 13 13:51:59 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 13 May 2014 09:51:59 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <8288203.XBo0QC8Y2f@mgerdin03> References: <53716503.7060503@oracle.com> <8288203.XBo0QC8Y2f@mgerdin03> Message-ID: <5372237F.1050005@oracle.com> Hi Mikael, On 5/13/14, 6:33 AM, Mikael Gerdin wrote: > Hi Coleen, > > On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >> Summary: Only prune metaspace virtual spaces at safepoint so walking >> them is safe outside a safepoint. >> >> Walking class loader data graph for contains is really slow for >> applications like nashorn that has a lot of class loader data. >> >> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 > Thanks for taking care of this fix. > I think the idea of deferring CLDG::purge to the next safepoint cleanup is > sound. Oh great! I'm glad you agree. > > I like the removal of CLDG::contains since it makes more sense to ask > Metaspace straight up. > > I have a small question about the test you added: > > 3798 static void test_contains() { > 3799 // Test that all the methods in the CLDG are contained > 3800 ClassLoaderDataGraph::methods_do(assert_contains); > 3801 } > > Have we always allocated methods at this point in the VM startup? I thought this ExecuteVMInternalTests (or whatever the option is called) is at shutdown? I'll check. > Should we also test CLDG::classes_do to make sure it works for the compressed > class space? I had that same thought this morning. Yes, I'll add it. Coleen > > /Mikael > >> Marcus L. tested it with nashorn tests. >> >> Thanks, >> Coleen From niclas.adlertz at oracle.com Tue May 13 15:35:42 2014 From: niclas.adlertz at oracle.com (Niclas Adlertz) Date: Tue, 13 May 2014 17:35:42 +0200 Subject: [8u20] RFR: 8029302: Performance regression in Math.pow intrinsic In-Reply-To: <536D20E0.3060106@oracle.com> References: <536D1F47.1090104@oracle.com> <536D20E0.3060106@oracle.com> Message-ID: <53723BCE.1010200@oracle.com> Thanks Vladimir. Kind Regards, Niclas Adlertz On 05/09/2014 08:39 PM, Vladimir Kozlov wrote: > Good. > > Note, 8u20 requests should be sent to hotspot-dev. > > Thanks, > Vladimir > > On 5/9/14 11:32 AM, Niclas Adlertz wrote: >> Hi all, >> >> Please review the backport of 8029302 together with 8042052, where >> 8042052 fixed an issue introduced by 8029302. >> >> The changesets apply cleanly. I intend push 8029302 and 8042052 as >> separate changesets. >> >> 8029302: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6314 >> http://cr.openjdk.java.net/~adlertz/JDK-8029302/webrev01/ >> >> 8042052: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6385 >> http://cr.openjdk.java.net/~anoll/8042052/webrev.01/ >> >> Kind Regards, >> Niclas Adlertz From daniel.daugherty at oracle.com Tue May 13 17:42:56 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 13 May 2014 11:42:56 -0600 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <536D1C8A.4080503@oracle.com> References: <536D1C8A.4080503@oracle.com> Message-ID: <537259A0.10005@oracle.com> > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 src/share/vm/oops/method.hpp No comments. src/share/vm/utilities/accessFlags.hpp line 57: JVM_ACC_ON_STACK = 0x00080000, // RedefinedClasses() is used on the stack Typos: 'RedefinedClasses() is' -> 'RedefineClasses() was' Not your typos, but would you mind fixing them? src/share/vm/oops/cpCache.cpp line 501: (!f1_as_method()->is_old() && !f1_as_method()->is_obsolete() || line 502 f1_as_method()->is_deleted())); Perhaps this would a little more clear (added parens and reformatted): line 501: ((!f1_as_method()->is_old() && !f1_as_method()->is_obsolete()) || line 502 f1_as_method()->is_deleted())); Or this one: line 501: (f1_as_method()->is_deleted() || line 502: (!f1_as_method()->is_old() && !f1_as_method()->is_obsolete()))); src/share/vm/prims/jvmtiRedefineClasses.cpp No comments Thumbs up even if you don't want to juggle the logic in cpCache.cpp... Minor clarification: > - A regression introduced by the fix of: > https://bugs.openjdk.java.net/browse/JDK-7182152 The above fix modified the existing VM_RedefineClasses::check_class() code to use guarantee() instead of assert() and also modified the logic to catch "obsolete" in addition to "old" entries. I strongly suspect that the code before JDK-7182152 would still have fired an assert in the presence of deleted methods so the regression is probably older than JDK-7182152. Don't know if it is worth fixing that far back though... Dan On 5/9/14 12:20 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-8042796 > > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 > > > Summary: > > This is a Nightly Stabilization issue that was caused by a > combination of two problems: > - A regression introduced by the fix of: > https://bugs.openjdk.java.net/browse/JDK-7182152 > - An SQE testbase infra regression: > https://bugs.openjdk.java.net/browse/INTJDK-7611018 > > A number of the vm.mlvm tests hits this guarantee taht was added by > 7182152 (must be relaxed). > The issue is with the deleted static private methods that are still > present in the CP cache. > The fix is to mark the deleted methods with the flag > JVM_ACC_IS_DELETED and > then use it to relax the guarantee condition. > > > Testing: > Running the failing tests: vm.mlvm.indy.func.jvmti > In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs on > sparcv9 and amd64. > > > Thanks, > Serguei From coleen.phillimore at oracle.com Tue May 13 18:16:30 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 13 May 2014 14:16:30 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <5372237F.1050005@oracle.com> References: <53716503.7060503@oracle.com> <8288203.XBo0QC8Y2f@mgerdin03> <5372237F.1050005@oracle.com> Message-ID: <5372617E.40706@oracle.com> Hi again, I have moved the metaspace vm internal test to the end of execution, and also test if klasses are in metaspace, which works fine. Thank you for the suggestion! Here is a new webrev: http://cr.openjdk.java.net/~coleenp/8038212_2/ Thanks, Coleen On 5/13/14, 9:51 AM, Coleen Phillimore wrote: > > Hi Mikael, > > On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >> Hi Coleen, >> >> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>> Summary: Only prune metaspace virtual spaces at safepoint so walking >>> them is safe outside a safepoint. >>> >>> Walking class loader data graph for contains is really slow for >>> applications like nashorn that has a lot of class loader data. >>> >>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >> Thanks for taking care of this fix. >> I think the idea of deferring CLDG::purge to the next safepoint >> cleanup is >> sound. > > Oh great! I'm glad you agree. >> >> I like the removal of CLDG::contains since it makes more sense to ask >> Metaspace straight up. >> >> I have a small question about the test you added: >> >> 3798 static void test_contains() { >> 3799 // Test that all the methods in the CLDG are contained >> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >> 3801 } >> >> Have we always allocated methods at this point in the VM startup? > > I thought this ExecuteVMInternalTests (or whatever the option is > called) is at shutdown? I'll check. >> Should we also test CLDG::classes_do to make sure it works for the >> compressed >> class space? > > I had that same thought this morning. Yes, I'll add it. > > Coleen > >> >> /Mikael >> >>> Marcus L. tested it with nashorn tests. >>> >>> Thanks, >>> Coleen > From mikael.gerdin at oracle.com Tue May 13 18:41:47 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 13 May 2014 20:41:47 +0200 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <5372617E.40706@oracle.com> References: <53716503.7060503@oracle.com> <5372237F.1050005@oracle.com> <5372617E.40706@oracle.com> Message-ID: <2511461.tAbzLMKlmA@vboxbuntu> On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: > Hi again, > > I have moved the metaspace vm internal test to the end of execution, and > also test if klasses are in metaspace, which works fine. Thank you for > the suggestion! Here is a new webrev: I'm not too happy about moving the test to the before_exit handler. Also, how can using the run_unit_test-macro (defined in jni.cpp) even work in java.cpp? About methods and classes being available I think we may have preloaded some classes in Threads::create_vm, right? /Mikael > > http://cr.openjdk.java.net/~coleenp/8038212_2/ > > Thanks, > Coleen > > On 5/13/14, 9:51 AM, Coleen Phillimore wrote: > > Hi Mikael, > > > > On 5/13/14, 6:33 AM, Mikael Gerdin wrote: > >> Hi Coleen, > >> > >> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: > >>> Summary: Only prune metaspace virtual spaces at safepoint so walking > >>> them is safe outside a safepoint. > >>> > >>> Walking class loader data graph for contains is really slow for > >>> applications like nashorn that has a lot of class loader data. > >>> > >>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran > >>> > >>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ > >>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 > >> > >> Thanks for taking care of this fix. > >> I think the idea of deferring CLDG::purge to the next safepoint > >> cleanup is > >> sound. > > > > Oh great! I'm glad you agree. > > > >> I like the removal of CLDG::contains since it makes more sense to ask > >> Metaspace straight up. > >> > >> I have a small question about the test you added: > >> > >> 3798 static void test_contains() { > >> 3799 // Test that all the methods in the CLDG are contained > >> 3800 ClassLoaderDataGraph::methods_do(assert_contains); > >> 3801 } > >> > >> Have we always allocated methods at this point in the VM startup? > > > > I thought this ExecuteVMInternalTests (or whatever the option is > > called) is at shutdown? I'll check. > > > >> Should we also test CLDG::classes_do to make sure it works for the > >> compressed > >> class space? > > > > I had that same thought this morning. Yes, I'll add it. > > > > Coleen > > > >> /Mikael > >> > >>> Marcus L. tested it with nashorn tests. > >>> > >>> Thanks, > >>> Coleen From coleen.phillimore at oracle.com Tue May 13 18:49:03 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 13 May 2014 14:49:03 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <2511461.tAbzLMKlmA@vboxbuntu> References: <53716503.7060503@oracle.com> <5372237F.1050005@oracle.com> <5372617E.40706@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> Message-ID: <5372691F.2030107@oracle.com> On 5/13/14, 2:41 PM, Mikael Gerdin wrote: > On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: >> Hi again, >> >> I have moved the metaspace vm internal test to the end of execution, and >> also test if klasses are in metaspace, which works fine. Thank you for >> the suggestion! Here is a new webrev: > I'm not too happy about moving the test to the before_exit handler. Why not? > > Also, how can using the run_unit_test-macro (defined in jni.cpp) even work in > java.cpp? It's moved to globalDefinitions.hpp. > > About methods and classes being available I think we may have preloaded some > classes in Threads::create_vm, right? Yes, we have loaded classes before the initial ExecuteVMInternalTests but they are only in the boot class loader, which is only in one metaspace. So it's not much of a test. At least at the end, we will have classes in the application class loader that we are testing whether they are contained in metaspace. I am not motivated to add the test for just for the sake of having a test. This change fixes the nashorn performance problem. Maybe that's test enough. before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 Coleen > > /Mikael > > >> http://cr.openjdk.java.net/~coleenp/8038212_2/ >> >> Thanks, >> Coleen >> >> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: >>> Hi Mikael, >>> >>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >>>> Hi Coleen, >>>> >>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>>>> Summary: Only prune metaspace virtual spaces at safepoint so walking >>>>> them is safe outside a safepoint. >>>>> >>>>> Walking class loader data graph for contains is really slow for >>>>> applications like nashorn that has a lot of class loader data. >>>>> >>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >>>> Thanks for taking care of this fix. >>>> I think the idea of deferring CLDG::purge to the next safepoint >>>> cleanup is >>>> sound. >>> Oh great! I'm glad you agree. >>> >>>> I like the removal of CLDG::contains since it makes more sense to ask >>>> Metaspace straight up. >>>> >>>> I have a small question about the test you added: >>>> >>>> 3798 static void test_contains() { >>>> 3799 // Test that all the methods in the CLDG are contained >>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >>>> 3801 } >>>> >>>> Have we always allocated methods at this point in the VM startup? >>> I thought this ExecuteVMInternalTests (or whatever the option is >>> called) is at shutdown? I'll check. >>> >>>> Should we also test CLDG::classes_do to make sure it works for the >>>> compressed >>>> class space? >>> I had that same thought this morning. Yes, I'll add it. >>> >>> Coleen >>> >>>> /Mikael >>>> >>>>> Marcus L. tested it with nashorn tests. >>>>> >>>>> Thanks, >>>>> Coleen From volker.simonis at gmail.com Tue May 13 19:32:09 2014 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 13 May 2014 21:32:09 +0200 Subject: RFR(XS): 8043029: Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas Message-ID: Hi, Change 8037816 introduced extra format checking by GCC for custom HS printf functions by annotating them with __attribute__((format(printf,...)). These checks have been selectively disabled with the help of diagnostic pragmas (i.e. #pragma GCC diagnostic ignored "-Wformat-nonliteral") in places where these functions are called with non-constant format strings. Unfortunately, diagnostic pragmas were only introduced in GCC 4.2 (see http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gcc/Diagnostic-Pragmas.html#Diagnostic-Pragmas). Older GCC versions simply ignore these pragmas and issue a warning. The fix to this problem is trivial - only enable '__attribute__((format(printf,...))'-annotations for GCC 4.2 and above: http://cr.openjdk.java.net/~simonis/webrevs/8043029/ https://bugs.openjdk.java.net/browse/JDK-8043029 Additionally to the before-mentioned fix, I have also improved the version checking for the definition of the various diagnostic pragmas. In the original version they would have only been enabled for __GNUC_MINOR__ versions greater and equal '6' but nor for GCC 5.0 for example. And I've also tried to break some of the overly long lines to make them more readable. Thank you and best regards, Volker PS: as always, I kindly ask for a sponsor to push the change once it is reviewed :) From mikael.gerdin at oracle.com Tue May 13 19:38:53 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 13 May 2014 21:38:53 +0200 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <5372691F.2030107@oracle.com> References: <53716503.7060503@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> <5372691F.2030107@oracle.com> Message-ID: <3786299.dyoyOuaSQ9@vboxbuntu> On Tuesday 13 May 2014 14.49.03 Coleen Phillimore wrote: > On 5/13/14, 2:41 PM, Mikael Gerdin wrote: > > On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: > >> Hi again, > >> > >> I have moved the metaspace vm internal test to the end of execution, and > >> also test if klasses are in metaspace, which works fine. Thank you for > > > >> the suggestion! Here is a new webrev: > > I'm not too happy about moving the test to the before_exit handler. > > Why not? I guess that most of the other unit tests are (and in my opinion should be) as independent as possible of JVM state such as loaded classes. I was fine with the test before since it's a reasonably small amount of code to get a test and it adds some test coverage. When the change involves adding additional hooks for unit tests in the exit path I'm just slightly more bothered by it. If we want a proper unit test for Metaspace::contains which tests all sorts of alllocations/classloaders and such then it needs to be a bit more explicit with setting up a known test environment. > > > Also, how can using the run_unit_test-macro (defined in jni.cpp) even work > > in java.cpp? > > It's moved to globalDefinitions.hpp. I see. > > > About methods and classes being available I think we may have preloaded > > some classes in Threads::create_vm, right? > > Yes, we have loaded classes before the initial ExecuteVMInternalTests > but they are only in the boot class loader, which is only in one > metaspace. So it's not much of a test. At least at the end, we will > have classes in the application class loader that we are testing whether > they are contained in metaspace. Ok. > > I am not motivated to add the test for just for the sake of having a > test. This change fixes the nashorn performance problem. Maybe that's > test enough. > > before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 > after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 Well, I guess the test isn't really for the correctness of the ::contains function anyway? And we don't really have a good way to do a measured test to make sure that ::contains isn't "too slow". /Mikael > > Coleen > > > /Mikael > > > >> http://cr.openjdk.java.net/~coleenp/8038212_2/ > >> > >> Thanks, > >> Coleen > >> > >> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: > >>> Hi Mikael, > >>> > >>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: > >>>> Hi Coleen, > >>>> > >>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: > >>>>> Summary: Only prune metaspace virtual spaces at safepoint so walking > >>>>> them is safe outside a safepoint. > >>>>> > >>>>> Walking class loader data graph for contains is really slow for > >>>>> applications like nashorn that has a lot of class loader data. > >>>>> > >>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran > >>>>> > >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ > >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 > >>>> > >>>> Thanks for taking care of this fix. > >>>> I think the idea of deferring CLDG::purge to the next safepoint > >>>> cleanup is > >>>> sound. > >>> > >>> Oh great! I'm glad you agree. > >>> > >>>> I like the removal of CLDG::contains since it makes more sense to ask > >>>> Metaspace straight up. > >>>> > >>>> I have a small question about the test you added: > >>>> > >>>> 3798 static void test_contains() { > >>>> 3799 // Test that all the methods in the CLDG are contained > >>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); > >>>> 3801 } > >>>> > >>>> Have we always allocated methods at this point in the VM startup? > >>> > >>> I thought this ExecuteVMInternalTests (or whatever the option is > >>> called) is at shutdown? I'll check. > >>> > >>>> Should we also test CLDG::classes_do to make sure it works for the > >>>> compressed > >>>> class space? > >>> > >>> I had that same thought this morning. Yes, I'll add it. > >>> > >>> Coleen > >>> > >>>> /Mikael > >>>> > >>>>> Marcus L. tested it with nashorn tests. > >>>>> > >>>>> Thanks, > >>>>> Coleen From john.r.rose at oracle.com Tue May 13 19:44:18 2014 From: john.r.rose at oracle.com (John Rose) Date: Tue, 13 May 2014 12:44:18 -0700 Subject: Pointer formats, again... In-Reply-To: <537079B3.6050103@oracle.com> References: <537079B3.6050103@oracle.com> Message-ID: <30222405-69FC-4030-AF5B-9C5A00CC94CC@oracle.com> On May 12, 2014, at 12:35 AM, David Holmes wrote: > The vicious circle here is that to get standardized output we can't use %p, but then we get the type mismatches that gcc warns about, which then requires the use of p2i and INTPTR_FORMAT which then raises the "why can't we just use %p?" question which ... :) If you look very carefully at this: http://melissadupreephotography.com/blogshow-data/storage/3842/will-o-the-wisp4-3846.jpg you might see at the heart of each wisp a tiny glowing '%p'. (Or not.) ? John From serguei.spitsyn at oracle.com Tue May 13 20:19:16 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 13 May 2014 13:19:16 -0700 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <537259A0.10005@oracle.com> References: <536D1C8A.4080503@oracle.com> <537259A0.10005@oracle.com> Message-ID: <53727E44.4010308@oracle.com> Dan, Thank you a lot for reviewing! On 5/13/14 10:42 AM, Daniel D. Daugherty wrote: > > > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 > > > src/share/vm/oops/method.hpp > No comments. > > src/share/vm/utilities/accessFlags.hpp > line 57: JVM_ACC_ON_STACK = 0x00080000, // > RedefinedClasses() is used on the stack > Typos: 'RedefinedClasses() is' -> 'RedefineClasses() was' > Not your typos, but would you mind fixing them? Thanks, I'll fix it. > > src/share/vm/oops/cpCache.cpp > line 501: (!f1_as_method()->is_old() && > !f1_as_method()->is_obsolete() || > line 502 f1_as_method()->is_deleted())); > > Perhaps this would a little more clear (added parens and > reformatted): > > line 501: ((!f1_as_method()->is_old() && > !f1_as_method()->is_obsolete()) || > line 502 f1_as_method()->is_deleted())); > > Or this one: > > line 501: (f1_as_method()->is_deleted() || > line 502: (!f1_as_method()->is_old() && > !f1_as_method()->is_obsolete()))); I agree and will fix it as you suggested. > > src/share/vm/prims/jvmtiRedefineClasses.cpp > No comments > > Thumbs up even if you don't want to juggle the logic > in cpCache.cpp... > > Minor clarification: > > > - A regression introduced by the fix of: > > https://bugs.openjdk.java.net/browse/JDK-7182152 > > The above fix modified the existing VM_RedefineClasses::check_class() > code to use guarantee() instead of assert() and also modified the > logic to catch "obsolete" in addition to "old" entries. I strongly > suspect that the code before JDK-7182152 would still have fired an > assert in the presence of deleted methods so the regression is > probably older than JDK-7182152. Don't know if it is worth fixing > that far back though... Most likely, you are right. In fact, it is not that easy to track it back in time so far. :) Thanks, Serguei > > Dan > > > On 5/9/14 12:20 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-8042796 >> >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >> >> >> Summary: >> >> This is a Nightly Stabilization issue that was caused by a >> combination of two problems: >> - A regression introduced by the fix of: >> https://bugs.openjdk.java.net/browse/JDK-7182152 >> - An SQE testbase infra regression: >> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >> >> A number of the vm.mlvm tests hits this guarantee taht was added by >> 7182152 (must be relaxed). >> The issue is with the deleted static private methods that are still >> present in the CP cache. >> The fix is to mark the deleted methods with the flag >> JVM_ACC_IS_DELETED and >> then use it to relax the guarantee condition. >> >> >> Testing: >> Running the failing tests: vm.mlvm.indy.func.jvmti >> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs on >> sparcv9 and amd64. >> >> >> Thanks, >> Serguei > From coleen.phillimore at oracle.com Tue May 13 20:20:08 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 13 May 2014 16:20:08 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <3786299.dyoyOuaSQ9@vboxbuntu> References: <53716503.7060503@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> <5372691F.2030107@oracle.com> <3786299.dyoyOuaSQ9@vboxbuntu> Message-ID: <53727E78.7030307@oracle.com> On 5/13/14, 3:38 PM, Mikael Gerdin wrote: > On Tuesday 13 May 2014 14.49.03 Coleen Phillimore wrote: >> On 5/13/14, 2:41 PM, Mikael Gerdin wrote: >>> On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: >>>> Hi again, >>>> >>>> I have moved the metaspace vm internal test to the end of execution, and >>>> also test if klasses are in metaspace, which works fine. Thank you for >>>> the suggestion! Here is a new webrev: >>> I'm not too happy about moving the test to the before_exit handler. >> Why not? > I guess that most of the other unit tests are (and in my opinion should be) as > independent as possible of JVM state such as loaded classes. > I was fine with the test before since it's a reasonably small amount of code to > get a test and it adds some test coverage. > When the change involves adding additional hooks for unit tests in the exit > path I'm just slightly more bothered by it. > > If we want a proper unit test for Metaspace::contains which tests all sorts of > alllocations/classloaders and such then it needs to be a bit more explicit > with setting up a known test environment. > >>> Also, how can using the run_unit_test-macro (defined in jni.cpp) even work >>> in java.cpp? >> It's moved to globalDefinitions.hpp. > I see. > >>> About methods and classes being available I think we may have preloaded >>> some classes in Threads::create_vm, right? >> Yes, we have loaded classes before the initial ExecuteVMInternalTests >> but they are only in the boot class loader, which is only in one >> metaspace. So it's not much of a test. At least at the end, we will >> have classes in the application class loader that we are testing whether >> they are contained in metaspace. > Ok. > >> I am not motivated to add the test for just for the sake of having a >> test. This change fixes the nashorn performance problem. Maybe that's >> test enough. >> >> before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 >> after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 > Well, I guess the test isn't really for the correctness of the ::contains > function anyway? And we don't really have a good way to do a measured test to > make sure that ::contains isn't "too slow". We actually already test that classes are Metaspace::contains() in the system dictionary verification at the end of the debug build (Dictionary::verify => instanceKlass::verify => Metaspace::contains) I think this test adds nothing now and given your concerns, I'm going to revert it. Thanks, Coleen > > /Mikael > >> Coleen >> >>> /Mikael >>> >>>> http://cr.openjdk.java.net/~coleenp/8038212_2/ >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: >>>>> Hi Mikael, >>>>> >>>>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>>>>>> Summary: Only prune metaspace virtual spaces at safepoint so walking >>>>>>> them is safe outside a safepoint. >>>>>>> >>>>>>> Walking class loader data graph for contains is really slow for >>>>>>> applications like nashorn that has a lot of class loader data. >>>>>>> >>>>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran >>>>>>> >>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >>>>>> Thanks for taking care of this fix. >>>>>> I think the idea of deferring CLDG::purge to the next safepoint >>>>>> cleanup is >>>>>> sound. >>>>> Oh great! I'm glad you agree. >>>>> >>>>>> I like the removal of CLDG::contains since it makes more sense to ask >>>>>> Metaspace straight up. >>>>>> >>>>>> I have a small question about the test you added: >>>>>> >>>>>> 3798 static void test_contains() { >>>>>> 3799 // Test that all the methods in the CLDG are contained >>>>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >>>>>> 3801 } >>>>>> >>>>>> Have we always allocated methods at this point in the VM startup? >>>>> I thought this ExecuteVMInternalTests (or whatever the option is >>>>> called) is at shutdown? I'll check. >>>>> >>>>>> Should we also test CLDG::classes_do to make sure it works for the >>>>>> compressed >>>>>> class space? >>>>> I had that same thought this morning. Yes, I'll add it. >>>>> >>>>> Coleen >>>>> >>>>>> /Mikael >>>>>> >>>>>>> Marcus L. tested it with nashorn tests. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen From david.r.chase at oracle.com Tue May 13 20:59:40 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 13 May 2014 16:59:40 -0400 Subject: RFR(XS): 8043029: Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas In-Reply-To: References: Message-ID: <0F944B5B-748C-49B3-AA11-179D62D04C6B@oracle.com> I'm not a Reviewer, but I've become rather familiar with this code recently, and your changes look entirely correct. And I will sponsor the fix as soon as we do get a Reviewer to comment. On 2014-05-13, at 3:32 PM, Volker Simonis wrote: > Hi, > > Change 8037816 introduced extra format checking by GCC for custom HS > printf functions by annotating them with > __attribute__((format(printf,...)). These checks have been selectively > disabled with the help of diagnostic pragmas (i.e. #pragma GCC > diagnostic ignored "-Wformat-nonliteral") in places where these > functions are called with non-constant format strings. > > Unfortunately, diagnostic pragmas were only introduced in GCC 4.2 (see > http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gcc/Diagnostic-Pragmas.html#Diagnostic-Pragmas). > Older GCC versions simply ignore these pragmas and issue a warning. > > The fix to this problem is trivial - only enable > '__attribute__((format(printf,...))'-annotations for GCC 4.2 and > above: > > http://cr.openjdk.java.net/~simonis/webrevs/8043029/ > https://bugs.openjdk.java.net/browse/JDK-8043029 > > Additionally to the before-mentioned fix, I have also improved the > version checking for the definition of the various diagnostic pragmas. > In the original version they would have only been enabled for > __GNUC_MINOR__ versions greater and equal '6' but nor for GCC 5.0 for > example. And I've also tried to break some of the overly long lines to > make them more readable. > > Thank you and best regards, > Volker > > PS: as always, I kindly ask for a sponsor to push the change once it > is reviewed :) From vladimir.kozlov at oracle.com Tue May 13 21:37:09 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 13 May 2014 14:37:09 -0700 Subject: RFR(XS): 8043029: Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas In-Reply-To: <0F944B5B-748C-49B3-AA11-179D62D04C6B@oracle.com> References: <0F944B5B-748C-49B3-AA11-179D62D04C6B@oracle.com> Message-ID: <53729085.4000205@oracle.com> Looks fine to me too. Thanks, Vladimir On 5/13/14 1:59 PM, David Chase wrote: > I'm not a Reviewer, but I've become rather familiar with this code recently, > and your changes look entirely correct. > > And I will sponsor the fix as soon as we do get a Reviewer to comment. > > On 2014-05-13, at 3:32 PM, Volker Simonis wrote: > >> Hi, >> >> Change 8037816 introduced extra format checking by GCC for custom HS >> printf functions by annotating them with >> __attribute__((format(printf,...)). These checks have been selectively >> disabled with the help of diagnostic pragmas (i.e. #pragma GCC >> diagnostic ignored "-Wformat-nonliteral") in places where these >> functions are called with non-constant format strings. >> >> Unfortunately, diagnostic pragmas were only introduced in GCC 4.2 (see >> http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gcc/Diagnostic-Pragmas.html#Diagnostic-Pragmas). >> Older GCC versions simply ignore these pragmas and issue a warning. >> >> The fix to this problem is trivial - only enable >> '__attribute__((format(printf,...))'-annotations for GCC 4.2 and >> above: >> >> http://cr.openjdk.java.net/~simonis/webrevs/8043029/ >> https://bugs.openjdk.java.net/browse/JDK-8043029 >> >> Additionally to the before-mentioned fix, I have also improved the >> version checking for the definition of the various diagnostic pragmas. >> In the original version they would have only been enabled for >> __GNUC_MINOR__ versions greater and equal '6' but nor for GCC 5.0 for >> example. And I've also tried to break some of the overly long lines to >> make them more readable. >> >> Thank you and best regards, >> Volker >> >> PS: as always, I kindly ask for a sponsor to push the change once it >> is reviewed :) > From coleen.phillimore at oracle.com Tue May 13 23:17:02 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 13 May 2014 19:17:02 -0400 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <536D1C8A.4080503@oracle.com> References: <536D1C8A.4080503@oracle.com> Message-ID: <5372A7EE.2080109@oracle.com> Serguei, I don't know if this is the right fix. We shouldn't have deleted methods in the cpCache because there's no way to keep these Method*s from being deallocated. I thought AdjustCpoolCacheAndVtable => ConstantPoolCache::adjust_method_entries should fix these entries. For deleted methods, shouldn't their cpCache entries be set to unresolved? Which class had the deleted methods? The one being redefined? It looks like the code never did this. I think this is missing functionality. Do we not have any tests that delete a method? Your change should add a test anyway - there are class redefinition tests in jdk/test/java/lang/instrument that can be adapted and we should probably keep them all together there. Thanks, Coleen On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-8042796 > > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 > > > Summary: > > This is a Nightly Stabilization issue that was caused by a > combination of two problems: > - A regression introduced by the fix of: > https://bugs.openjdk.java.net/browse/JDK-7182152 > - An SQE testbase infra regression: > https://bugs.openjdk.java.net/browse/INTJDK-7611018 > > A number of the vm.mlvm tests hits this guarantee taht was added by > 7182152 (must be relaxed). > The issue is with the deleted static private methods that are still > present in the CP cache. > The fix is to mark the deleted methods with the flag > JVM_ACC_IS_DELETED and > then use it to relax the guarantee condition. > > > Testing: > Running the failing tests: vm.mlvm.indy.func.jvmti > In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs on > sparcv9 and amd64. > > > Thanks, > Serguei From serguei.spitsyn at oracle.com Wed May 14 00:31:08 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 13 May 2014 17:31:08 -0700 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <5372A7EE.2080109@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> Message-ID: <5372B94C.4060905@oracle.com> Coleen, On 5/13/14 4:17 PM, Coleen Phillimore wrote: > > Serguei, > > I don't know if this is the right fix. We shouldn't have deleted > methods in the cpCache because there's no way to keep these Method*s > from being deallocated. I thought AdjustCpoolCacheAndVtable => > ConstantPoolCache::adjust_method_entries should fix these entries. For > deleted methods, shouldn't their cpCache entries be set to > unresolved? Which class had the deleted methods? The one being > redefined? Thank you for reviewing this and for the questions. Let's check if my understanding is right as it is relatively easy to miss something. The original class has been redefined so that the new version does not have several private static methods anymore. These methods are called "deleted" but they are still present in the original version of the class. Even if we set their cpCache entries to unresolved these entries can be set to resolved again. So that I've decided this approach is not going to work in general case. The interesting question is: Q: What is going to happen with these entries when the original version of the class goes away? I'll try to answer this question by looking at the code. It won't be a surprise if there is nothing in place to cover this case. > > It looks like the code never did this. I think this is missing > functionality. Yes, that was my guess too. > Do we not have any tests that delete a method? Most likely, we don't. > Your change should add a test anyway - there are class redefinition > tests in jdk/test/java/lang/instrument that can be adapted and we > should probably keep them all together there. Right, it is better to have some test coverage here. I'll check if any of the JLI or JDI tests can be extended to cover this. I'm not sure the JLI is good for this task though as deleting the methods is not instrumentation. :) Thanks! Serguei > > Thanks, > Coleen > > On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-8042796 >> >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >> >> >> Summary: >> >> This is a Nightly Stabilization issue that was caused by a >> combination of two problems: >> - A regression introduced by the fix of: >> https://bugs.openjdk.java.net/browse/JDK-7182152 >> - An SQE testbase infra regression: >> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >> >> A number of the vm.mlvm tests hits this guarantee taht was added by >> 7182152 (must be relaxed). >> The issue is with the deleted static private methods that are still >> present in the CP cache. >> The fix is to mark the deleted methods with the flag >> JVM_ACC_IS_DELETED and >> then use it to relax the guarantee condition. >> >> >> Testing: >> Running the failing tests: vm.mlvm.indy.func.jvmti >> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs on >> sparcv9 and amd64. >> >> >> Thanks, >> Serguei > From david.simms at oracle.com Wed May 14 08:46:01 2014 From: david.simms at oracle.com (David Simms) Date: Wed, 14 May 2014 10:46:01 +0200 Subject: RFR (XS) JNI Specification Issue: JDK-6681965 Strings and NULL termination Message-ID: <53732D49.2060602@oracle.com> Bug/Enhancement: https://bugs.openjdk.java.net/browse/JDK-6681965 Web review: http://cr.openjdk.java.net/~dsimms/jnispec/6681965/ HTML version: http://cr.openjdk.java.net/~dsimms/jnispec/6681965/JDK-6681965.html There are number of mail threads asking for clarification surrounding string operations, and whether the results should be NULL terminated. Functions affected: * const jchar * *GetStringChars*(JNIEnv *env, jstring string, jboolean *isCopy); * const char * *GetStringUTFChars*(JNIEnv *env, jstring string, jboolean *isCopy); * void *GetStringRegion*(JNIEnv *env, jstring str, jsize start, jsize len, jchar *buf); * void *GetStringUTFRegion*(JNIEnv *env, jstring str, jsize start, jsize len, char *buf); * const jchar * *GetStringCritical*(JNIEnv *env, jstring string, jboolean *isCopy); There are two stories here, how the particular JVM implements Java Strings internally, and the JNI Specification. I believe these two things are separate: * From a HotSpot JVM perspective, null-terminating strings might be seen as "safety first". * There may be other JVM implementations whom, for argument sake, that never garbage collect and always use direct heap pointers. Such implementations have no reason to null-terminate... * ...indeed HotSpot's "GetStringCritical()" implementation does no such thing. Currently the JNI Specification does not mention null-termination, and given we are not dealing with c-strings, but Java Strings, that is because it's not required. "GetStringUTFRegion()" as currently implemented by HotSpot has an issue, it is possible to null-terminate past the user specified length (which in itself a bit of an issue, its related to the input, not output, being UTF-16 length != UT8 encoded length). I'm not advocating removing the current null-termination where HotSpot supplies the buffers. In fact I'm worried will break too much existing code. There is no harm in null-termination where the JVM has control over the buffer. Clarify NULL-termination, or rather lack of, for string operations. Additional text at the beginning of the "String Operations" section: ---------------------------------------------------------------------- String Operations This specification makes no assumptions on how a JVM represents Java strings internally. Strings returned from these operations: * GetStringChars() * GetStringUTFChars() * GetStringRegion() * GetStringUTFRegion() * GetStringCritical() are therefore not required to be NULL terminated. Programmers are expected to determine buffer capacity requirements via"|GetStringLength()| " or "|GetStringUTFLength()| ". ---------------------------------------------------------------------- /David Simms From tobias.hartmann at oracle.com Wed May 14 08:57:31 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 14 May 2014 10:57:31 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache Message-ID: <53732FFB.7050407@oracle.com> Hi, please review the following patch for bug 8005873. *Problem* Bug: https://bugs.openjdk.java.net/browse/JDK-8005873 The test creates multiple threads (~500) that repeatedly execute invokeExact on a method handle referencing a static method. The call to invokeExact is performed through an optimized inline cache (CompiledStaticCall) that points to the LambdaForm generated for the method handle. The IC is first set to interpreted code by CompiledStaticCall::set_to_interpreted(...) but soon updated to refer to the compiled version of the lambda form (-Xcomp). Because tiered compilation is enabled, the VM decides to compile the LambdaForm at a different level and sets the old version to not-entrant. The next call through the IC therefore triggers a re-resolving of the call site finally performing a Java upcall to java.lang.invoke.MethodHandleNatives::linkMethod(...) (see call sequence [1]). A *new* LambdaForm is returned and CompiledStaticCall::set_to_interpreted(...) is executed again to update the IC. The assert is hit because the callee differs. The problem is that there are multiple LambdaForms created for the same invokeExact instruction. Caching in the Java runtime code should guarantee that only one LambdaForm is created and reused. Instrumenting Invokers::invokeHandleForm(...) shows that almost all requests result in a cache miss and return a new LambdaForm. This behaviour is caused by a race condition in Invokers::invokeHandleForm(...). Threads may initially see a cache miss when invoking MethodTypeForm::cachedLambdaForm(...), then create a new LambdaForm and call MethodTypeForm::setCachedLambdaForm(...) to update the cache without checking it again. In a high concurrency setting, this leads to multiple LambdaForms being created for the same invokeExact instruction because the cache entry is overwritten by multiple threads. *Solution* Webrev: http://cr.openjdk.java.net/~anoll/8005873/webrev.00/ An AtomicReferenceArray is used to cache the LambdaForms and .get(...) and .compareAndSet(...) are used to retrieve and update the cache entries. This allows only one thread to set the LambdaForm that is then being used by all instances of the invokeExact. *Testing* - Failing test (vm/mlvm/meth/stress/jni/nativeAndMH) - Nashorn + Octane - JPRT Many thanks to Christian Thalinger and Vladimir Ivanov! Best, Tobias [1] Call sequence of reresolving the IC target SharedRuntime::handle_wrong_method(...) -> SharedRuntime::reresolve_call_site(...) -> SharedRuntime::find_callee_method(...) -> SharedRuntime::find_callee_info_helper(...) -> LinkResolver::resolve_invoke(...) -> LinkResolver::resolve_invokehandle(...) -> LinkResolver::resolve_invokehandle(...) -> LinkResolver::lookup_polymorphic_method(...) -> SystemDictionary::find_method_handle_invoker(...) -> Java upcall to java.lang.invoke.MethodHandleNatives::linkMethod(...) -> Invokers::methodHandleInvokeLinkerMethod(...) -> Invokers::invokeHandleForm(...) Backport? From markus.gronlund at oracle.com Wed May 14 09:10:14 2014 From: markus.gronlund at oracle.com (=?iso-8859-1?B?TWFya3VzIEdy9m5sdW5k?=) Date: Wed, 14 May 2014 02:10:14 -0700 (PDT) Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <53727E78.7030307@oracle.com> References: <53716503.7060503@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> <5372691F.2030107@oracle.com> <3786299.dyoyOuaSQ9@vboxbuntu> <53727E78.7030307@oracle.com> Message-ID: <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> Hi Coleen, I have taken a look as well as running the nashorn crypto tests with the patches - these changes looks good. Thanks for addressing this. /Markus -----Original Message----- From: Coleen Phillimore Sent: den 13 maj 2014 22:20 To: Mikael Gerdin Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking On 5/13/14, 3:38 PM, Mikael Gerdin wrote: > On Tuesday 13 May 2014 14.49.03 Coleen Phillimore wrote: >> On 5/13/14, 2:41 PM, Mikael Gerdin wrote: >>> On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: >>>> Hi again, >>>> >>>> I have moved the metaspace vm internal test to the end of >>>> execution, and also test if klasses are in metaspace, which works >>>> fine. Thank you for the suggestion! Here is a new webrev: >>> I'm not too happy about moving the test to the before_exit handler. >> Why not? > I guess that most of the other unit tests are (and in my opinion > should be) as independent as possible of JVM state such as loaded classes. > I was fine with the test before since it's a reasonably small amount > of code to get a test and it adds some test coverage. > When the change involves adding additional hooks for unit tests in the > exit path I'm just slightly more bothered by it. > > If we want a proper unit test for Metaspace::contains which tests all > sorts of alllocations/classloaders and such then it needs to be a bit > more explicit with setting up a known test environment. > >>> Also, how can using the run_unit_test-macro (defined in jni.cpp) >>> even work in java.cpp? >> It's moved to globalDefinitions.hpp. > I see. > >>> About methods and classes being available I think we may have >>> preloaded some classes in Threads::create_vm, right? >> Yes, we have loaded classes before the initial ExecuteVMInternalTests >> but they are only in the boot class loader, which is only in one >> metaspace. So it's not much of a test. At least at the end, we will >> have classes in the application class loader that we are testing >> whether they are contained in metaspace. > Ok. > >> I am not motivated to add the test for just for the sake of having a >> test. This change fixes the nashorn performance problem. Maybe that's >> test enough. >> >> before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 >> after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 > Well, I guess the test isn't really for the correctness of the > ::contains function anyway? And we don't really have a good way to do > a measured test to make sure that ::contains isn't "too slow". We actually already test that classes are Metaspace::contains() in the system dictionary verification at the end of the debug build (Dictionary::verify => instanceKlass::verify => Metaspace::contains) I think this test adds nothing now and given your concerns, I'm going to revert it. Thanks, Coleen > > /Mikael > >> Coleen >> >>> /Mikael >>> >>>> http://cr.openjdk.java.net/~coleenp/8038212_2/ >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: >>>>> Hi Mikael, >>>>> >>>>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>>>>>> Summary: Only prune metaspace virtual spaces at safepoint so >>>>>>> walking them is safe outside a safepoint. >>>>>>> >>>>>>> Walking class loader data graph for contains is really slow for >>>>>>> applications like nashorn that has a lot of class loader data. >>>>>>> >>>>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also >>>>>>> ran >>>>>>> >>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >>>>>> Thanks for taking care of this fix. >>>>>> I think the idea of deferring CLDG::purge to the next safepoint >>>>>> cleanup is sound. >>>>> Oh great! I'm glad you agree. >>>>> >>>>>> I like the removal of CLDG::contains since it makes more sense to ask >>>>>> Metaspace straight up. >>>>>> >>>>>> I have a small question about the test you added: >>>>>> >>>>>> 3798 static void test_contains() { >>>>>> 3799 // Test that all the methods in the CLDG are contained >>>>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >>>>>> 3801 } >>>>>> >>>>>> Have we always allocated methods at this point in the VM startup? >>>>> I thought this ExecuteVMInternalTests (or whatever the option is >>>>> called) is at shutdown? I'll check. >>>>> >>>>>> Should we also test CLDG::classes_do to make sure it works for the >>>>>> compressed >>>>>> class space? >>>>> I had that same thought this morning. Yes, I'll add it. >>>>> >>>>> Coleen >>>>> >>>>>> /Mikael >>>>>> >>>>>>> Marcus L. tested it with nashorn tests. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen From forax at univ-mlv.fr Wed May 14 10:04:26 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 14 May 2014 12:04:26 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <53732FFB.7050407@oracle.com> References: <53732FFB.7050407@oracle.com> Message-ID: <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> your patch doesn't work ! the array is still allocated as an classical array in the constructor. cheers, Remi Envoy? avec AquaMail pour Android http://www.aqua-mail.com Le 14 mai 2014 11:04:41 Tobias Hartmann a ?crit : > Hi, > > please review the following patch for bug 8005873. > > *Problem* > Bug: https://bugs.openjdk.java.net/browse/JDK-8005873 > > The test creates multiple threads (~500) that repeatedly execute > invokeExact on a method handle referencing a static method. The call to > invokeExact is performed through an optimized inline cache > (CompiledStaticCall) that points to the LambdaForm generated for the method > handle. The IC is first set to interpreted code by > CompiledStaticCall::set_to_interpreted(...) but soon updated to refer to > the compiled version of the lambda form (-Xcomp). > > Because tiered compilation is enabled, the VM decides to compile the > LambdaForm at a different level and sets the old version to not-entrant. > The next call through the IC therefore triggers a re-resolving of the call > site finally performing a Java upcall to > java.lang.invoke.MethodHandleNatives::linkMethod(...) (see call sequence > [1]). A *new* LambdaForm is returned and > CompiledStaticCall::set_to_interpreted(...) is executed again to update the > IC. The assert is hit because the callee differs. > > The problem is that there are multiple LambdaForms created for the same > invokeExact instruction. Caching in the Java runtime code should guarantee > that only one LambdaForm is created and reused. Instrumenting > Invokers::invokeHandleForm(...) shows that almost all requests result in a > cache miss and return a new LambdaForm. > > This behaviour is caused by a race condition in > Invokers::invokeHandleForm(...). Threads may initially see a cache miss > when invoking MethodTypeForm::cachedLambdaForm(...), then create a new > LambdaForm and call MethodTypeForm::setCachedLambdaForm(...) to update the > cache without checking it again. In a high concurrency setting, this leads > to multiple LambdaForms being created for the same invokeExact instruction > because the cache entry is overwritten by multiple threads. > > *Solution* > Webrev: http://cr.openjdk.java.net/~anoll/8005873/webrev.00/ > > An AtomicReferenceArray is used to cache the LambdaForms and .get(...) and > .compareAndSet(...) are used to retrieve and update the cache entries. This > allows only one thread to set the LambdaForm that is then being used by all > instances of the invokeExact. > > *Testing* > - Failing test (vm/mlvm/meth/stress/jni/nativeAndMH) > - Nashorn + Octane > - JPRT > > Many thanks to Christian Thalinger and Vladimir Ivanov! > > Best, > Tobias > > [1] Call sequence of reresolving the IC target > SharedRuntime::handle_wrong_method(...) > -> SharedRuntime::reresolve_call_site(...) > -> SharedRuntime::find_callee_method(...) > -> SharedRuntime::find_callee_info_helper(...) > -> LinkResolver::resolve_invoke(...) > -> LinkResolver::resolve_invokehandle(...) > -> LinkResolver::resolve_invokehandle(...) > -> LinkResolver::lookup_polymorphic_method(...) > -> SystemDictionary::find_method_handle_invoker(...) > -> Java upcall to java.lang.invoke.MethodHandleNatives::linkMethod(...) > -> Invokers::methodHandleInvokeLinkerMethod(...) > -> Invokers::invokeHandleForm(...) > > > > > > > > > > Backport? From tobias.hartmann at oracle.com Wed May 14 10:33:54 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 14 May 2014 12:33:54 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> Message-ID: <53734692.6080409@oracle.com> Hi Remi, sorry, I accidentally reverted that part.. Here is the correct webrev: http://cr.openjdk.java.net/~anoll/8005873/webrev.01/ Thanks, Tobias On 14.05.2014 12:04, Remi Forax wrote: > your patch doesn't work ! > > the array is still allocated as an classical array in the constructor. > > cheers, > Remi > > Envoy? avec AquaMail pour Android > http://www.aqua-mail.com > > > Le 14 mai 2014 11:04:41 Tobias Hartmann a > ?crit : > >> Hi, >> >> please review the following patch for bug 8005873. >> >> *Problem* >> Bug: https://bugs.openjdk.java.net/browse/JDK-8005873 >> >> The test creates multiple threads (~500) that repeatedly execute >> invokeExact on a method handle referencing a static method. The call >> to invokeExact is performed through an optimized inline cache >> (CompiledStaticCall) that points to the LambdaForm generated for the >> method handle. The IC is first set to interpreted code by >> CompiledStaticCall::set_to_interpreted(...) but soon updated to refer >> to the compiled version of the lambda form (-Xcomp). >> >> Because tiered compilation is enabled, the VM decides to compile the >> LambdaForm at a different level and sets the old version to >> not-entrant. The next call through the IC therefore triggers a >> re-resolving of the call site finally performing a Java upcall to >> java.lang.invoke.MethodHandleNatives::linkMethod(...) (see call >> sequence [1]). A *new* LambdaForm is returned and >> CompiledStaticCall::set_to_interpreted(...) is executed again to >> update the IC. The assert is hit because the callee differs. >> >> The problem is that there are multiple LambdaForms created for the >> same invokeExact instruction. Caching in the Java runtime code should >> guarantee that only one LambdaForm is created and reused. >> Instrumenting Invokers::invokeHandleForm(...) shows that almost all >> requests result in a cache miss and return a new LambdaForm. >> >> This behaviour is caused by a race condition in >> Invokers::invokeHandleForm(...). Threads may initially see a cache >> miss when invoking MethodTypeForm::cachedLambdaForm(...), then create >> a new LambdaForm and call MethodTypeForm::setCachedLambdaForm(...) to >> update the cache without checking it again. In a high concurrency >> setting, this leads to multiple LambdaForms being created for the >> same invokeExact instruction because the cache entry is overwritten >> by multiple threads. >> >> *Solution* >> Webrev: http://cr.openjdk.java.net/~anoll/8005873/webrev.00/ >> >> An AtomicReferenceArray is used to cache the LambdaForms and >> .get(...) and .compareAndSet(...) are used to retrieve and update the >> cache entries. This allows only one thread to set the LambdaForm that >> is then being used by all instances of the invokeExact. >> >> *Testing* >> - Failing test (vm/mlvm/meth/stress/jni/nativeAndMH) >> - Nashorn + Octane >> - JPRT >> >> Many thanks to Christian Thalinger and Vladimir Ivanov! >> >> Best, >> Tobias >> >> [1] Call sequence of reresolving the IC target >> SharedRuntime::handle_wrong_method(...) >> -> SharedRuntime::reresolve_call_site(...) >> -> SharedRuntime::find_callee_method(...) >> -> SharedRuntime::find_callee_info_helper(...) >> -> LinkResolver::resolve_invoke(...) >> -> LinkResolver::resolve_invokehandle(...) >> -> LinkResolver::resolve_invokehandle(...) >> -> LinkResolver::lookup_polymorphic_method(...) >> -> SystemDictionary::find_method_handle_invoker(...) >> -> Java upcall to java.lang.invoke.MethodHandleNatives::linkMethod(...) >> -> Invokers::methodHandleInvokeLinkerMethod(...) >> -> Invokers::invokeHandleForm(...) >> >> >> >> >> >> >> >> >> >> Backport? > > From vladimir.x.ivanov at oracle.com Wed May 14 10:47:38 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 14 May 2014 14:47:38 +0400 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <53734692.6080409@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> Message-ID: <537349CA.1090904@oracle.com> Tobias, I agree with your evaluation. My only concern is that @Stable doesn't work for AtomicReferenceArray, so JIT doesn't see what is stored in the array. Maybe use a lock instead? Best regards, Vladimir Ivanov On 5/14/14 2:33 PM, Tobias Hartmann wrote: > Hi Remi, > > sorry, I accidentally reverted that part.. Here is the correct webrev: > > http://cr.openjdk.java.net/~anoll/8005873/webrev.01/ > > Thanks, > Tobias > > On 14.05.2014 12:04, Remi Forax wrote: >> your patch doesn't work ! >> >> the array is still allocated as an classical array in the constructor. >> >> cheers, >> Remi >> >> Envoy? avec AquaMail pour Android >> http://www.aqua-mail.com >> >> >> Le 14 mai 2014 11:04:41 Tobias Hartmann a >> ?crit : >> >>> Hi, >>> >>> please review the following patch for bug 8005873. >>> >>> *Problem* >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8005873 >>> >>> The test creates multiple threads (~500) that repeatedly execute >>> invokeExact on a method handle referencing a static method. The call >>> to invokeExact is performed through an optimized inline cache >>> (CompiledStaticCall) that points to the LambdaForm generated for the >>> method handle. The IC is first set to interpreted code by >>> CompiledStaticCall::set_to_interpreted(...) but soon updated to refer >>> to the compiled version of the lambda form (-Xcomp). >>> >>> Because tiered compilation is enabled, the VM decides to compile the >>> LambdaForm at a different level and sets the old version to >>> not-entrant. The next call through the IC therefore triggers a >>> re-resolving of the call site finally performing a Java upcall to >>> java.lang.invoke.MethodHandleNatives::linkMethod(...) (see call >>> sequence [1]). A *new* LambdaForm is returned and >>> CompiledStaticCall::set_to_interpreted(...) is executed again to >>> update the IC. The assert is hit because the callee differs. >>> >>> The problem is that there are multiple LambdaForms created for the >>> same invokeExact instruction. Caching in the Java runtime code should >>> guarantee that only one LambdaForm is created and reused. >>> Instrumenting Invokers::invokeHandleForm(...) shows that almost all >>> requests result in a cache miss and return a new LambdaForm. >>> >>> This behaviour is caused by a race condition in >>> Invokers::invokeHandleForm(...). Threads may initially see a cache >>> miss when invoking MethodTypeForm::cachedLambdaForm(...), then create >>> a new LambdaForm and call MethodTypeForm::setCachedLambdaForm(...) to >>> update the cache without checking it again. In a high concurrency >>> setting, this leads to multiple LambdaForms being created for the >>> same invokeExact instruction because the cache entry is overwritten >>> by multiple threads. >>> >>> *Solution* >>> Webrev: http://cr.openjdk.java.net/~anoll/8005873/webrev.00/ >>> >>> An AtomicReferenceArray is used to cache the LambdaForms and >>> .get(...) and .compareAndSet(...) are used to retrieve and update the >>> cache entries. This allows only one thread to set the LambdaForm that >>> is then being used by all instances of the invokeExact. >>> >>> *Testing* >>> - Failing test (vm/mlvm/meth/stress/jni/nativeAndMH) >>> - Nashorn + Octane >>> - JPRT >>> >>> Many thanks to Christian Thalinger and Vladimir Ivanov! >>> >>> Best, >>> Tobias >>> >>> [1] Call sequence of reresolving the IC target >>> SharedRuntime::handle_wrong_method(...) >>> -> SharedRuntime::reresolve_call_site(...) >>> -> SharedRuntime::find_callee_method(...) >>> -> SharedRuntime::find_callee_info_helper(...) >>> -> LinkResolver::resolve_invoke(...) >>> -> LinkResolver::resolve_invokehandle(...) >>> -> LinkResolver::resolve_invokehandle(...) >>> -> LinkResolver::lookup_polymorphic_method(...) >>> -> SystemDictionary::find_method_handle_invoker(...) >>> -> Java upcall to java.lang.invoke.MethodHandleNatives::linkMethod(...) >>> -> Invokers::methodHandleInvokeLinkerMethod(...) >>> -> Invokers::invokeHandleForm(...) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Backport? >> >> > From paul.sandoz at oracle.com Wed May 14 12:12:00 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 14 May 2014 14:12:00 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <537349CA.1090904@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> Message-ID: <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> On May 14, 2014, at 12:47 PM, Vladimir Ivanov wrote: > Tobias, I agree with your evaluation. > V. tricky one to track down! From @Stable: * It is (currently) undefined what happens if a field annotated as stable * is given a third value. In practice, if the JVM relies on this annotation * to promote a field reference to a constant, it may be that the Java memory * model would appear to be broken, if such a constant (the second value of the field) * is used as the value of the field even after the field value has changed. I dunno if that was a contributing factor in this case. > My only concern is that @Stable doesn't work for AtomicReferenceArray, so JIT doesn't see what is stored in the array. Yes, stability needs to be associated with the array elements. > Maybe use a lock instead? > Or perhaps use Unsafe.CAS directly within setCachedLambdaForm? Also, as a consequence of using AtomicReferenceArray the following change may result in a memory barrier on some architectures: public LambdaForm cachedLambdaForm(int which) { - return lambdaForms[which]; + return lambdaForms.get(which); } since lambdaForms.get will call Unsafe.getObjectVolatile. Separately, i think code that calls setCachedLambdaForm needs to be double checked to ensure that the return value is used. For example, in MethodHandleImpl.makeGuardWithCatchForm i see: basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); return lform; which i think needs to be: return basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); Paul. From coleen.phillimore at oracle.com Wed May 14 12:21:44 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 14 May 2014 08:21:44 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> References: <53716503.7060503@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> <5372691F.2030107@oracle.com> <3786299.dyoyOuaSQ9@vboxbuntu> <53727E78.7030307@oracle.com> <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> Message-ID: <53735FD8.6080908@oracle.com> Thank you Markus, Coleen On 5/14/14, 5:10 AM, Markus Gr?nlund wrote: > Hi Coleen, > > I have taken a look as well as running the nashorn crypto tests with the patches - these changes looks good. > > Thanks for addressing this. > > /Markus > > > -----Original Message----- > From: Coleen Phillimore > Sent: den 13 maj 2014 22:20 > To: Mikael Gerdin > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking > > > On 5/13/14, 3:38 PM, Mikael Gerdin wrote: >> On Tuesday 13 May 2014 14.49.03 Coleen Phillimore wrote: >>> On 5/13/14, 2:41 PM, Mikael Gerdin wrote: >>>> On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: >>>>> Hi again, >>>>> >>>>> I have moved the metaspace vm internal test to the end of >>>>> execution, and also test if klasses are in metaspace, which works >>>>> fine. Thank you for the suggestion! Here is a new webrev: >>>> I'm not too happy about moving the test to the before_exit handler. >>> Why not? >> I guess that most of the other unit tests are (and in my opinion >> should be) as independent as possible of JVM state such as loaded classes. >> I was fine with the test before since it's a reasonably small amount >> of code to get a test and it adds some test coverage. >> When the change involves adding additional hooks for unit tests in the >> exit path I'm just slightly more bothered by it. >> >> If we want a proper unit test for Metaspace::contains which tests all >> sorts of alllocations/classloaders and such then it needs to be a bit >> more explicit with setting up a known test environment. >> >>>> Also, how can using the run_unit_test-macro (defined in jni.cpp) >>>> even work in java.cpp? >>> It's moved to globalDefinitions.hpp. >> I see. >> >>>> About methods and classes being available I think we may have >>>> preloaded some classes in Threads::create_vm, right? >>> Yes, we have loaded classes before the initial ExecuteVMInternalTests >>> but they are only in the boot class loader, which is only in one >>> metaspace. So it's not much of a test. At least at the end, we will >>> have classes in the application class loader that we are testing >>> whether they are contained in metaspace. >> Ok. >> >>> I am not motivated to add the test for just for the sake of having a >>> test. This change fixes the nashorn performance problem. Maybe that's >>> test enough. >>> >>> before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 >>> after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 >> Well, I guess the test isn't really for the correctness of the >> ::contains function anyway? And we don't really have a good way to do >> a measured test to make sure that ::contains isn't "too slow". > We actually already test that classes are Metaspace::contains() in the system dictionary verification at the end of the debug build (Dictionary::verify => instanceKlass::verify => Metaspace::contains) > > I think this test adds nothing now and given your concerns, I'm going to revert it. > > Thanks, > Coleen > >> /Mikael >> >>> Coleen >>> >>>> /Mikael >>>> >>>>> http://cr.openjdk.java.net/~coleenp/8038212_2/ >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: >>>>>> Hi Mikael, >>>>>> >>>>>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>>>>>>> Summary: Only prune metaspace virtual spaces at safepoint so >>>>>>>> walking them is safe outside a safepoint. >>>>>>>> >>>>>>>> Walking class loader data graph for contains is really slow for >>>>>>>> applications like nashorn that has a lot of class loader data. >>>>>>>> >>>>>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also >>>>>>>> ran >>>>>>>> >>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >>>>>>> Thanks for taking care of this fix. >>>>>>> I think the idea of deferring CLDG::purge to the next safepoint >>>>>>> cleanup is sound. >>>>>> Oh great! I'm glad you agree. >>>>>> >>>>>>> I like the removal of CLDG::contains since it makes more sense to ask >>>>>>> Metaspace straight up. >>>>>>> >>>>>>> I have a small question about the test you added: >>>>>>> >>>>>>> 3798 static void test_contains() { >>>>>>> 3799 // Test that all the methods in the CLDG are contained >>>>>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >>>>>>> 3801 } >>>>>>> >>>>>>> Have we always allocated methods at this point in the VM startup? >>>>>> I thought this ExecuteVMInternalTests (or whatever the option is >>>>>> called) is at shutdown? I'll check. >>>>>> >>>>>>> Should we also test CLDG::classes_do to make sure it works for the >>>>>>> compressed >>>>>>> class space? >>>>>> I had that same thought this morning. Yes, I'll add it. >>>>>> >>>>>> Coleen >>>>>> >>>>>>> /Mikael >>>>>>> >>>>>>>> Marcus L. tested it with nashorn tests. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen From harold.seigel at oracle.com Wed May 14 12:50:53 2014 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 14 May 2014 08:50:53 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> References: <53716503.7060503@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> <5372691F.2030107@oracle.com> <3786299.dyoyOuaSQ9@vboxbuntu> <53727E78.7030307@oracle.com> <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> Message-ID: <537366AD.3030602@oracle.com> Hi Coleen, Your changes look good to me, also. Harold On 5/14/2014 5:10 AM, Markus Gr?nlund wrote: > Hi Coleen, > > I have taken a look as well as running the nashorn crypto tests with the patches - these changes looks good. > > Thanks for addressing this. > > /Markus > > > -----Original Message----- > From: Coleen Phillimore > Sent: den 13 maj 2014 22:20 > To: Mikael Gerdin > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking > > > On 5/13/14, 3:38 PM, Mikael Gerdin wrote: >> On Tuesday 13 May 2014 14.49.03 Coleen Phillimore wrote: >>> On 5/13/14, 2:41 PM, Mikael Gerdin wrote: >>>> On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: >>>>> Hi again, >>>>> >>>>> I have moved the metaspace vm internal test to the end of >>>>> execution, and also test if klasses are in metaspace, which works >>>>> fine. Thank you for the suggestion! Here is a new webrev: >>>> I'm not too happy about moving the test to the before_exit handler. >>> Why not? >> I guess that most of the other unit tests are (and in my opinion >> should be) as independent as possible of JVM state such as loaded classes. >> I was fine with the test before since it's a reasonably small amount >> of code to get a test and it adds some test coverage. >> When the change involves adding additional hooks for unit tests in the >> exit path I'm just slightly more bothered by it. >> >> If we want a proper unit test for Metaspace::contains which tests all >> sorts of alllocations/classloaders and such then it needs to be a bit >> more explicit with setting up a known test environment. >> >>>> Also, how can using the run_unit_test-macro (defined in jni.cpp) >>>> even work in java.cpp? >>> It's moved to globalDefinitions.hpp. >> I see. >> >>>> About methods and classes being available I think we may have >>>> preloaded some classes in Threads::create_vm, right? >>> Yes, we have loaded classes before the initial ExecuteVMInternalTests >>> but they are only in the boot class loader, which is only in one >>> metaspace. So it's not much of a test. At least at the end, we will >>> have classes in the application class loader that we are testing >>> whether they are contained in metaspace. >> Ok. >> >>> I am not motivated to add the test for just for the sake of having a >>> test. This change fixes the nashorn performance problem. Maybe that's >>> test enough. >>> >>> before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 >>> after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 >> Well, I guess the test isn't really for the correctness of the >> ::contains function anyway? And we don't really have a good way to do >> a measured test to make sure that ::contains isn't "too slow". > We actually already test that classes are Metaspace::contains() in the system dictionary verification at the end of the debug build (Dictionary::verify => instanceKlass::verify => Metaspace::contains) > > I think this test adds nothing now and given your concerns, I'm going to revert it. > > Thanks, > Coleen > >> /Mikael >> >>> Coleen >>> >>>> /Mikael >>>> >>>>> http://cr.openjdk.java.net/~coleenp/8038212_2/ >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: >>>>>> Hi Mikael, >>>>>> >>>>>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>>>>>>> Summary: Only prune metaspace virtual spaces at safepoint so >>>>>>>> walking them is safe outside a safepoint. >>>>>>>> >>>>>>>> Walking class loader data graph for contains is really slow for >>>>>>>> applications like nashorn that has a lot of class loader data. >>>>>>>> >>>>>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also >>>>>>>> ran >>>>>>>> >>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >>>>>>> Thanks for taking care of this fix. >>>>>>> I think the idea of deferring CLDG::purge to the next safepoint >>>>>>> cleanup is sound. >>>>>> Oh great! I'm glad you agree. >>>>>> >>>>>>> I like the removal of CLDG::contains since it makes more sense to ask >>>>>>> Metaspace straight up. >>>>>>> >>>>>>> I have a small question about the test you added: >>>>>>> >>>>>>> 3798 static void test_contains() { >>>>>>> 3799 // Test that all the methods in the CLDG are contained >>>>>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >>>>>>> 3801 } >>>>>>> >>>>>>> Have we always allocated methods at this point in the VM startup? >>>>>> I thought this ExecuteVMInternalTests (or whatever the option is >>>>>> called) is at shutdown? I'll check. >>>>>> >>>>>>> Should we also test CLDG::classes_do to make sure it works for the >>>>>>> compressed >>>>>>> class space? >>>>>> I had that same thought this morning. Yes, I'll add it. >>>>>> >>>>>> Coleen >>>>>> >>>>>>> /Mikael >>>>>>> >>>>>>>> Marcus L. tested it with nashorn tests. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen From vladimir.x.ivanov at oracle.com Wed May 14 13:15:39 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 14 May 2014 17:15:39 +0400 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> Message-ID: <53736C7B.2040303@oracle.com> On 5/14/14 4:12 PM, Paul Sandoz wrote: > > On May 14, 2014, at 12:47 PM, Vladimir Ivanov wrote: > >> Tobias, I agree with your evaluation. >> > > V. tricky one to track down! > > From @Stable: > > * It is (currently) undefined what happens if a field annotated as stable > * is given a third value. In practice, if the JVM relies on this annotation > * to promote a field reference to a constant, it may be that the Java memory > * model would appear to be broken, if such a constant (the second value of the field) > * is used as the value of the field even after the field value has changed. > > I dunno if that was a contributing factor in this case. No, @Stable doesn't contribute to the problem. > >> My only concern is that @Stable doesn't work for AtomicReferenceArray, so JIT doesn't see what is stored in the array. > > Yes, stability needs to be associated with the array elements. > > >> Maybe use a lock instead? >> > > Or perhaps use Unsafe.CAS directly within setCachedLambdaForm? Yes, Unsafe is another option here. But since cache updates should be rare, Unsafe is an overkill here IMO - locking should be fine. > > Also, as a consequence of using AtomicReferenceArray the following change may result in a memory barrier on some architectures: > > public LambdaForm cachedLambdaForm(int which) { > - return lambdaForms[which]; > > + return lambdaForms.get(which); > } > > since lambdaForms.get will call Unsafe.getObjectVolatile. MTF.cachedLambaForm isn't on a fast path, so it shouldn't be a problem. > Separately, i think code that calls setCachedLambdaForm needs to be double checked to ensure that the return value is used. For example, in MethodHandleImpl.makeGuardWithCatchForm i see: > > basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); > return lform; > > which i think needs to be: > > return basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); Good catch! MTF.setCachedLambdaForm usages should be fixed as well. Best regards, Vladimir Ivanov > > Paul. > > > > From dmitry.fazunenko at oracle.com Wed May 14 14:28:19 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 14 May 2014 18:28:19 +0400 Subject: RFR: 8039489: Refactor test framework for dynamic VM options In-Reply-To: <535A524D.1000701@oracle.com> References: <535A524D.1000701@oracle.com> Message-ID: <53737D83.2050205@oracle.com> Hi, Would you please look at the new version of a common library for testing dynamic VM options. Description: Some time ago a library for testing VM flags which could be changed dynamically was integrated into the hs repository: test/testlibrary/com/oracle/java/testlibrary/DynamicVMOptionChecker.java test/testlibrary/com/oracle/java/testlibrary/TestDynamicVMOption.java The library was designed in not the most optimal way. This change is an attempt to redesign it: test/testlibrary/com/oracle/java/testlibrary/DynamicVMOption.java is a new version which provides a convenient access to the HS API on changing VM flags. The tests based on the library are also refactored to meet new API. You comments will be much appreciated. http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8039489 I want to thank Erik Helin for his incredible assistance with working on this change. Thanks, Dima From jesper.wilhelmsson at oracle.com Wed May 14 14:50:21 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 14 May 2014 16:50:21 +0200 Subject: RFR: 8039489: Refactor test framework for dynamic VM options In-Reply-To: <53737D83.2050205@oracle.com> References: <535A524D.1000701@oracle.com> <53737D83.2050205@oracle.com> Message-ID: <537382AD.6090107@oracle.com> Looks good! /Jesper Dmitry Fazunenko skrev 14/5/14 16:28: > Hi, > > Would you please look at the new version of a common library for testing dynamic > VM options. > > Description: > Some time ago a library for testing VM flags which could be changed dynamically > was integrated into the hs repository: > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOptionChecker.java > test/testlibrary/com/oracle/java/testlibrary/TestDynamicVMOption.java > > The library was designed in not the most optimal way. > This change is an attempt to redesign it: > > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOption.java > > is a new version which provides a convenient access to the HS API on changing VM > flags. > The tests based on the library are also refactored to meet new API. > > You comments will be much appreciated. > > http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8039489 > > I want to thank Erik Helin for his incredible assistance with working on this > change. > > Thanks, > Dima From christian.thalinger at oracle.com Wed May 14 16:02:35 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 14 May 2014 09:02:35 -0700 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <537349CA.1090904@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> Message-ID: <195D7C4C-58E0-44ED-A57C-D51159E60BA6@oracle.com> On May 14, 2014, at 3:47 AM, Vladimir Ivanov wrote: > Tobias, I agree with your evaluation. > > My only concern is that @Stable doesn't work for AtomicReferenceArray, so JIT doesn't see what is stored in the array. Now that is really unfortunate. > Maybe use a lock instead? > > Best regards, > Vladimir Ivanov > > On 5/14/14 2:33 PM, Tobias Hartmann wrote: >> Hi Remi, >> >> sorry, I accidentally reverted that part.. Here is the correct webrev: >> >> http://cr.openjdk.java.net/~anoll/8005873/webrev.01/ >> >> Thanks, >> Tobias >> >> On 14.05.2014 12:04, Remi Forax wrote: >>> your patch doesn't work ! >>> >>> the array is still allocated as an classical array in the constructor. >>> >>> cheers, >>> Remi >>> >>> Envoy? avec AquaMail pour Android >>> http://www.aqua-mail.com >>> >>> >>> Le 14 mai 2014 11:04:41 Tobias Hartmann a >>> ?crit : >>> >>>> Hi, >>>> >>>> please review the following patch for bug 8005873. >>>> >>>> *Problem* >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8005873 >>>> >>>> The test creates multiple threads (~500) that repeatedly execute >>>> invokeExact on a method handle referencing a static method. The call >>>> to invokeExact is performed through an optimized inline cache >>>> (CompiledStaticCall) that points to the LambdaForm generated for the >>>> method handle. The IC is first set to interpreted code by >>>> CompiledStaticCall::set_to_interpreted(...) but soon updated to refer >>>> to the compiled version of the lambda form (-Xcomp). >>>> >>>> Because tiered compilation is enabled, the VM decides to compile the >>>> LambdaForm at a different level and sets the old version to >>>> not-entrant. The next call through the IC therefore triggers a >>>> re-resolving of the call site finally performing a Java upcall to >>>> java.lang.invoke.MethodHandleNatives::linkMethod(...) (see call >>>> sequence [1]). A *new* LambdaForm is returned and >>>> CompiledStaticCall::set_to_interpreted(...) is executed again to >>>> update the IC. The assert is hit because the callee differs. >>>> >>>> The problem is that there are multiple LambdaForms created for the >>>> same invokeExact instruction. Caching in the Java runtime code should >>>> guarantee that only one LambdaForm is created and reused. >>>> Instrumenting Invokers::invokeHandleForm(...) shows that almost all >>>> requests result in a cache miss and return a new LambdaForm. >>>> >>>> This behaviour is caused by a race condition in >>>> Invokers::invokeHandleForm(...). Threads may initially see a cache >>>> miss when invoking MethodTypeForm::cachedLambdaForm(...), then create >>>> a new LambdaForm and call MethodTypeForm::setCachedLambdaForm(...) to >>>> update the cache without checking it again. In a high concurrency >>>> setting, this leads to multiple LambdaForms being created for the >>>> same invokeExact instruction because the cache entry is overwritten >>>> by multiple threads. >>>> >>>> *Solution* >>>> Webrev: http://cr.openjdk.java.net/~anoll/8005873/webrev.00/ >>>> >>>> An AtomicReferenceArray is used to cache the LambdaForms and >>>> .get(...) and .compareAndSet(...) are used to retrieve and update the >>>> cache entries. This allows only one thread to set the LambdaForm that >>>> is then being used by all instances of the invokeExact. >>>> >>>> *Testing* >>>> - Failing test (vm/mlvm/meth/stress/jni/nativeAndMH) >>>> - Nashorn + Octane >>>> - JPRT >>>> >>>> Many thanks to Christian Thalinger and Vladimir Ivanov! >>>> >>>> Best, >>>> Tobias >>>> >>>> [1] Call sequence of reresolving the IC target >>>> SharedRuntime::handle_wrong_method(...) >>>> -> SharedRuntime::reresolve_call_site(...) >>>> -> SharedRuntime::find_callee_method(...) >>>> -> SharedRuntime::find_callee_info_helper(...) >>>> -> LinkResolver::resolve_invoke(...) >>>> -> LinkResolver::resolve_invokehandle(...) >>>> -> LinkResolver::resolve_invokehandle(...) >>>> -> LinkResolver::lookup_polymorphic_method(...) >>>> -> SystemDictionary::find_method_handle_invoker(...) >>>> -> Java upcall to java.lang.invoke.MethodHandleNatives::linkMethod(...) >>>> -> Invokers::methodHandleInvokeLinkerMethod(...) >>>> -> Invokers::invokeHandleForm(...) >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Backport? >>> >>> >> From coleen.phillimore at oracle.com Wed May 14 16:47:26 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 14 May 2014 12:47:26 -0400 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <5372B94C.4060905@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> <5372B94C.4060905@oracle.com> Message-ID: <53739E1E.1080002@oracle.com> Hi Serguei, On 5/13/14, 8:31 PM, serguei.spitsyn at oracle.com wrote: > Coleen, > > > On 5/13/14 4:17 PM, Coleen Phillimore wrote: >> >> Serguei, >> >> I don't know if this is the right fix. We shouldn't have deleted >> methods in the cpCache because there's no way to keep these Method*s >> from being deallocated. I thought AdjustCpoolCacheAndVtable => >> ConstantPoolCache::adjust_method_entries should fix these entries. >> For deleted methods, shouldn't their cpCache entries be set to >> unresolved? Which class had the deleted methods? The one being >> redefined? > > Thank you for reviewing this and for the questions. > > Let's check if my understanding is right as it is relatively easy to > miss something. > > The original class has been redefined so that the new version does not > have several private static methods anymore. > These methods are called "deleted" but they are still present in the > original version of the class. > Even if we set their cpCache entries to unresolved these entries can > be set to resolved again. > So that I've decided this approach is not going to work in general case. Okay, so you answered my question - since the deleted methods are private, they can only be found in the class being redefined's constant pool cache. Since the class being redefined's cpCache hasn't been deallocated, neither will the Method* that it's pointing to. Phew! Thank you for the explanation. > > The interesting question is: > Q: What is going to happen with these entries when the original > version of the class goes away? > Everything will get deallocated, which is fine. > I'll try to answer this question by looking at the code. > It won't be a surprise if there is nothing in place to cover this case. > >> >> It looks like the code never did this. I think this is missing >> functionality. > > Yes, that was my guess too. > >> Do we not have any tests that delete a method? > > Most likely, we don't. > >> Your change should add a test anyway - there are class redefinition >> tests in jdk/test/java/lang/instrument that can be adapted and we >> should probably keep them all together there. > > Right, it is better to have some test coverage here. > I'll check if any of the JLI or JDI tests can be extended to cover this. > I'm not sure the JLI is good for this task though as deleting the > methods is not instrumentation. :) > So the change looks good pending a test case. There might be one already but why didn't it fail this assert? So, we need a test for this case somewhere. Thanks, Coleen > > Thanks! > Serguei > >> >> Thanks, >> Coleen >> >> On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8042796 >>> >>> >>> Open webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >>> >>> >>> Summary: >>> >>> This is a Nightly Stabilization issue that was caused by a >>> combination of two problems: >>> - A regression introduced by the fix of: >>> https://bugs.openjdk.java.net/browse/JDK-7182152 >>> - An SQE testbase infra regression: >>> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >>> >>> A number of the vm.mlvm tests hits this guarantee taht was added >>> by 7182152 (must be relaxed). >>> The issue is with the deleted static private methods that are >>> still present in the CP cache. >>> The fix is to mark the deleted methods with the flag >>> JVM_ACC_IS_DELETED and >>> then use it to relax the guarantee condition. >>> >>> >>> Testing: >>> Running the failing tests: vm.mlvm.indy.func.jvmti >>> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs on >>> sparcv9 and amd64. >>> >>> >>> Thanks, >>> Serguei >> > From daniel.daugherty at oracle.com Wed May 14 17:04:33 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 14 May 2014 11:04:33 -0600 Subject: RFR (XS) JNI Specification Issue: JDK-6681965 Strings and NULL termination In-Reply-To: <53732D49.2060602@oracle.com> References: <53732D49.2060602@oracle.com> Message-ID: <5373A221.20909@oracle.com> Thumbs up on the new wording. Dan On 5/14/14 2:46 AM, David Simms wrote: > Bug/Enhancement: https://bugs.openjdk.java.net/browse/JDK-6681965 > Web review: http://cr.openjdk.java.net/~dsimms/jnispec/6681965/ > HTML version: > http://cr.openjdk.java.net/~dsimms/jnispec/6681965/JDK-6681965.html > > There are number of mail threads asking for clarification surrounding > string operations, and whether the results should be NULL terminated. > Functions affected: > > * const jchar * *GetStringChars*(JNIEnv *env, jstring string, jboolean > *isCopy); > * const char * *GetStringUTFChars*(JNIEnv *env, jstring string, > jboolean *isCopy); > * void *GetStringRegion*(JNIEnv *env, jstring str, jsize start, jsize > len, jchar *buf); > * void *GetStringUTFRegion*(JNIEnv *env, jstring str, jsize start, > jsize len, char *buf); > * const jchar * *GetStringCritical*(JNIEnv *env, jstring string, > jboolean *isCopy); > > There are two stories here, how the particular JVM implements Java > Strings internally, and the JNI Specification. I believe these two > things are separate: > > * From a HotSpot JVM perspective, null-terminating strings might be > seen as "safety first". > * There may be other JVM implementations whom, for argument sake, that > never garbage collect and always use direct heap pointers. Such > implementations have no reason to null-terminate... > * ...indeed HotSpot's "GetStringCritical()" implementation does no > such thing. > > Currently the JNI Specification does not mention null-termination, and > given we are not dealing with c-strings, but Java Strings, that is > because it's not required. > > "GetStringUTFRegion()" as currently implemented by HotSpot has an > issue, it is possible to null-terminate past the user specified length > (which in itself a bit of an issue, its related to the input, not > output, being UTF-16 length != UT8 encoded length). > > I'm not advocating removing the current null-termination where HotSpot > supplies the buffers. In fact I'm worried will break too much existing > code. There is no harm in null-termination where the JVM has control > over the buffer. > > Clarify NULL-termination, or rather lack of, for string operations. > Additional text at the beginning of the "String Operations" section: > > ---------------------------------------------------------------------- > String Operations > > This specification makes no assumptions on how a JVM represents Java > strings internally. Strings returned from these operations: > > * GetStringChars() > * GetStringUTFChars() > * GetStringRegion() > * GetStringUTFRegion() > * GetStringCritical() > > are therefore not required to be NULL terminated. Programmers are > expected to determine buffer capacity requirements > via"|GetStringLength()| > " > or "|GetStringUTFLength()| > ". > > ---------------------------------------------------------------------- > > /David Simms > > From serguei.spitsyn at oracle.com Wed May 14 17:30:37 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 14 May 2014 10:30:37 -0700 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <53739E1E.1080002@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> <5372B94C.4060905@oracle.com> <53739E1E.1080002@oracle.com> Message-ID: <5373A83D.10909@oracle.com> Hi Coleen, Thank you for the review and analysis! I'm working on adding a test coverage for this issue. Thanks, Serguei On 5/14/14 9:47 AM, Coleen Phillimore wrote: > > Hi Serguei, > > On 5/13/14, 8:31 PM, serguei.spitsyn at oracle.com wrote: >> Coleen, >> >> >> On 5/13/14 4:17 PM, Coleen Phillimore wrote: >>> >>> Serguei, >>> >>> I don't know if this is the right fix. We shouldn't have deleted >>> methods in the cpCache because there's no way to keep these Method*s >>> from being deallocated. I thought AdjustCpoolCacheAndVtable => >>> ConstantPoolCache::adjust_method_entries should fix these entries. >>> For deleted methods, shouldn't their cpCache entries be set to >>> unresolved? Which class had the deleted methods? The one being >>> redefined? >> >> Thank you for reviewing this and for the questions. >> >> Let's check if my understanding is right as it is relatively easy to >> miss something. >> >> The original class has been redefined so that the new version does >> not have several private static methods anymore. >> These methods are called "deleted" but they are still present in the >> original version of the class. >> Even if we set their cpCache entries to unresolved these entries can >> be set to resolved again. >> So that I've decided this approach is not going to work in general case. > > Okay, so you answered my question - since the deleted methods are > private, they can only be found in the class being redefined's > constant pool cache. Since the class being redefined's cpCache > hasn't been deallocated, neither will the Method* that it's pointing > to. Phew! Thank you for the explanation. > >> >> The interesting question is: >> Q: What is going to happen with these entries when the original >> version of the class goes away? >> > > Everything will get deallocated, which is fine. > >> I'll try to answer this question by looking at the code. >> It won't be a surprise if there is nothing in place to cover this case. >> >>> >>> It looks like the code never did this. I think this is missing >>> functionality. >> >> Yes, that was my guess too. >> >>> Do we not have any tests that delete a method? >> >> Most likely, we don't. >> >>> Your change should add a test anyway - there are class redefinition >>> tests in jdk/test/java/lang/instrument that can be adapted and we >>> should probably keep them all together there. >> >> Right, it is better to have some test coverage here. >> I'll check if any of the JLI or JDI tests can be extended to cover this. >> I'm not sure the JLI is good for this task though as deleting the >> methods is not instrumentation. :) >> > > So the change looks good pending a test case. There might be one > already but why didn't it fail this assert? So, we need a test for > this case somewhere. > > Thanks, > Coleen > >> >> Thanks! >> Serguei >> >>> >>> Thanks, >>> Coleen >>> >>> On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8042796 >>>> >>>> >>>> Open webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >>>> >>>> >>>> Summary: >>>> >>>> This is a Nightly Stabilization issue that was caused by a >>>> combination of two problems: >>>> - A regression introduced by the fix of: >>>> https://bugs.openjdk.java.net/browse/JDK-7182152 >>>> - An SQE testbase infra regression: >>>> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >>>> >>>> A number of the vm.mlvm tests hits this guarantee taht was added >>>> by 7182152 (must be relaxed). >>>> The issue is with the deleted static private methods that are >>>> still present in the CP cache. >>>> The fix is to mark the deleted methods with the flag >>>> JVM_ACC_IS_DELETED and >>>> then use it to relax the guarantee condition. >>>> >>>> >>>> Testing: >>>> Running the failing tests: vm.mlvm.indy.func.jvmti >>>> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs >>>> on sparcv9 and amd64. >>>> >>>> >>>> Thanks, >>>> Serguei >>> >> > From daniel.daugherty at oracle.com Wed May 14 17:52:21 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 14 May 2014 11:52:21 -0600 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <5373A83D.10909@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> <5372B94C.4060905@oracle.com> <53739E1E.1080002@oracle.com> <5373A83D.10909@oracle.com> Message-ID: <5373AD55.7070904@oracle.com> I found an existing test that exercises the add side of the house, but haven't found one that exercises the delete side (yet): $ pwd /work/shared/mirrors/src_clones/jdk9/dev_baseline di$ ls jdk/test/java/lang/instrument/RedefineMethodAddInvoke* jdk/test/java/lang/instrument/RedefineMethodAddInvoke.sh jdk/test/java/lang/instrument/RedefineMethodAddInvokeAgent.java jdk/test/java/lang/instrument/RedefineMethodAddInvokeApp.java jdk/test/java/lang/instrument/RedefineMethodAddInvokeTarget.java jdk/test/java/lang/instrument/RedefineMethodAddInvokeTarget_1.java jdk/test/java/lang/instrument/RedefineMethodAddInvokeTarget_2.java Dan On 5/14/14 11:30 AM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > Thank you for the review and analysis! > I'm working on adding a test coverage for this issue. > > Thanks, > Serguei > > > On 5/14/14 9:47 AM, Coleen Phillimore wrote: >> >> Hi Serguei, >> >> On 5/13/14, 8:31 PM, serguei.spitsyn at oracle.com wrote: >>> Coleen, >>> >>> >>> On 5/13/14 4:17 PM, Coleen Phillimore wrote: >>>> >>>> Serguei, >>>> >>>> I don't know if this is the right fix. We shouldn't have deleted >>>> methods in the cpCache because there's no way to keep these >>>> Method*s from being deallocated. I thought >>>> AdjustCpoolCacheAndVtable => >>>> ConstantPoolCache::adjust_method_entries should fix these entries. >>>> For deleted methods, shouldn't their cpCache entries be set to >>>> unresolved? Which class had the deleted methods? The one being >>>> redefined? >>> >>> Thank you for reviewing this and for the questions. >>> >>> Let's check if my understanding is right as it is relatively easy to >>> miss something. >>> >>> The original class has been redefined so that the new version does >>> not have several private static methods anymore. >>> These methods are called "deleted" but they are still present in the >>> original version of the class. >>> Even if we set their cpCache entries to unresolved these entries can >>> be set to resolved again. >>> So that I've decided this approach is not going to work in general >>> case. >> >> Okay, so you answered my question - since the deleted methods are >> private, they can only be found in the class being redefined's >> constant pool cache. Since the class being redefined's cpCache >> hasn't been deallocated, neither will the Method* that it's pointing >> to. Phew! Thank you for the explanation. >> >>> >>> The interesting question is: >>> Q: What is going to happen with these entries when the original >>> version of the class goes away? >>> >> >> Everything will get deallocated, which is fine. >> >>> I'll try to answer this question by looking at the code. >>> It won't be a surprise if there is nothing in place to cover this case. >>> >>>> >>>> It looks like the code never did this. I think this is missing >>>> functionality. >>> >>> Yes, that was my guess too. >>> >>>> Do we not have any tests that delete a method? >>> >>> Most likely, we don't. >>> >>>> Your change should add a test anyway - there are class redefinition >>>> tests in jdk/test/java/lang/instrument that can be adapted and we >>>> should probably keep them all together there. >>> >>> Right, it is better to have some test coverage here. >>> I'll check if any of the JLI or JDI tests can be extended to cover >>> this. >>> I'm not sure the JLI is good for this task though as deleting the >>> methods is not instrumentation. :) >>> >> >> So the change looks good pending a test case. There might be one >> already but why didn't it fail this assert? So, we need a test for >> this case somewhere. >> >> Thanks, >> Coleen >> >>> >>> Thanks! >>> Serguei >>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8042796 >>>>> >>>>> >>>>> Open webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >>>>> >>>>> >>>>> Summary: >>>>> >>>>> This is a Nightly Stabilization issue that was caused by a >>>>> combination of two problems: >>>>> - A regression introduced by the fix of: >>>>> https://bugs.openjdk.java.net/browse/JDK-7182152 >>>>> - An SQE testbase infra regression: >>>>> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >>>>> >>>>> A number of the vm.mlvm tests hits this guarantee taht was added >>>>> by 7182152 (must be relaxed). >>>>> The issue is with the deleted static private methods that are >>>>> still present in the CP cache. >>>>> The fix is to mark the deleted methods with the flag >>>>> JVM_ACC_IS_DELETED and >>>>> then use it to relax the guarantee condition. >>>>> >>>>> >>>>> Testing: >>>>> Running the failing tests: vm.mlvm.indy.func.jvmti >>>>> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs >>>>> on sparcv9 and amd64. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>> >>> >> > From serguei.spitsyn at oracle.com Wed May 14 18:05:20 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 14 May 2014 11:05:20 -0700 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <5373AD55.7070904@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> <5372B94C.4060905@oracle.com> <53739E1E.1080002@oracle.com> <5373A83D.10909@oracle.com> <5373AD55.7070904@oracle.com> Message-ID: <5373B060.6000206@oracle.com> Thanks, Dan! This test has a good base for what is needed in my case. Thanks, Serguei On 5/14/14 10:52 AM, Daniel D. Daugherty wrote: > I found an existing test that exercises the add side of the > house, but haven't found one that exercises the delete side (yet): > > $ pwd > /work/shared/mirrors/src_clones/jdk9/dev_baseline > > di$ ls jdk/test/java/lang/instrument/RedefineMethodAddInvoke* > jdk/test/java/lang/instrument/RedefineMethodAddInvoke.sh > jdk/test/java/lang/instrument/RedefineMethodAddInvokeAgent.java > jdk/test/java/lang/instrument/RedefineMethodAddInvokeApp.java > jdk/test/java/lang/instrument/RedefineMethodAddInvokeTarget.java > jdk/test/java/lang/instrument/RedefineMethodAddInvokeTarget_1.java > jdk/test/java/lang/instrument/RedefineMethodAddInvokeTarget_2.java > > Dan > > > On 5/14/14 11:30 AM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> Thank you for the review and analysis! >> I'm working on adding a test coverage for this issue. >> >> Thanks, >> Serguei >> >> >> On 5/14/14 9:47 AM, Coleen Phillimore wrote: >>> >>> Hi Serguei, >>> >>> On 5/13/14, 8:31 PM, serguei.spitsyn at oracle.com wrote: >>>> Coleen, >>>> >>>> >>>> On 5/13/14 4:17 PM, Coleen Phillimore wrote: >>>>> >>>>> Serguei, >>>>> >>>>> I don't know if this is the right fix. We shouldn't have deleted >>>>> methods in the cpCache because there's no way to keep these >>>>> Method*s from being deallocated. I thought >>>>> AdjustCpoolCacheAndVtable => >>>>> ConstantPoolCache::adjust_method_entries should fix these entries. >>>>> For deleted methods, shouldn't their cpCache entries be set to >>>>> unresolved? Which class had the deleted methods? The one being >>>>> redefined? >>>> >>>> Thank you for reviewing this and for the questions. >>>> >>>> Let's check if my understanding is right as it is relatively easy >>>> to miss something. >>>> >>>> The original class has been redefined so that the new version does >>>> not have several private static methods anymore. >>>> These methods are called "deleted" but they are still present in >>>> the original version of the class. >>>> Even if we set their cpCache entries to unresolved these entries >>>> can be set to resolved again. >>>> So that I've decided this approach is not going to work in general >>>> case. >>> >>> Okay, so you answered my question - since the deleted methods are >>> private, they can only be found in the class being redefined's >>> constant pool cache. Since the class being redefined's cpCache >>> hasn't been deallocated, neither will the Method* that it's pointing >>> to. Phew! Thank you for the explanation. >>> >>>> >>>> The interesting question is: >>>> Q: What is going to happen with these entries when the original >>>> version of the class goes away? >>>> >>> >>> Everything will get deallocated, which is fine. >>> >>>> I'll try to answer this question by looking at the code. >>>> It won't be a surprise if there is nothing in place to cover this >>>> case. >>>> >>>>> >>>>> It looks like the code never did this. I think this is missing >>>>> functionality. >>>> >>>> Yes, that was my guess too. >>>> >>>>> Do we not have any tests that delete a method? >>>> >>>> Most likely, we don't. >>>> >>>>> Your change should add a test anyway - there are class >>>>> redefinition tests in jdk/test/java/lang/instrument that can be >>>>> adapted and we should probably keep them all together there. >>>> >>>> Right, it is better to have some test coverage here. >>>> I'll check if any of the JLI or JDI tests can be extended to cover >>>> this. >>>> I'm not sure the JLI is good for this task though as deleting the >>>> methods is not instrumentation. :) >>>> >>> >>> So the change looks good pending a test case. There might be one >>> already but why didn't it fail this assert? So, we need a test for >>> this case somewhere. >>> >>> Thanks, >>> Coleen >>> >>>> >>>> Thanks! >>>> Serguei >>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8042796 >>>>>> >>>>>> >>>>>> Open webrev: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> This is a Nightly Stabilization issue that was caused by a >>>>>> combination of two problems: >>>>>> - A regression introduced by the fix of: >>>>>> https://bugs.openjdk.java.net/browse/JDK-7182152 >>>>>> - An SQE testbase infra regression: >>>>>> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >>>>>> >>>>>> A number of the vm.mlvm tests hits this guarantee taht was >>>>>> added by 7182152 (must be relaxed). >>>>>> The issue is with the deleted static private methods that are >>>>>> still present in the CP cache. >>>>>> The fix is to mark the deleted methods with the flag >>>>>> JVM_ACC_IS_DELETED and >>>>>> then use it to relax the guarantee condition. >>>>>> >>>>>> >>>>>> Testing: >>>>>> Running the failing tests: vm.mlvm.indy.func.jvmti >>>>>> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs >>>>>> on sparcv9 and amd64. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>> >>>> >>> >> > From david.r.chase at oracle.com Wed May 14 19:51:00 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 14 May 2014 15:51:00 -0400 Subject: RFR (XS) 8043164: Format warning in traceStream.hpp Message-ID: bug: https://bugs.openjdk.java.net/browse/JDK-8043164 webrev: http://cr.openjdk.java.net/~drchase/8043164/webrev.00/ fix: add appropriate cast to int64_t for INT64_FORMAT testing: Compiled with bug-tickling configuration on MacOS Mountain Lion (10.8) XCode 4.6.3 Mavericks (10.9) XCode (4.6.3 and 5.1). Verified that there were not other warnings tickled by this particular build configuration on either platform. Also retested with fastdebug build on Mavericks. From zhengyu.gu at oracle.com Wed May 14 20:02:02 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Wed, 14 May 2014 16:02:02 -0400 Subject: RFR (XS) 8043164: Format warning in traceStream.hpp In-Reply-To: References: Message-ID: <5373CBBA.7040006@oracle.com> Looks good. -Zhengyu On 5/14/2014 3:51 PM, David Chase wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8043164 > webrev: http://cr.openjdk.java.net/~drchase/8043164/webrev.00/ > fix: add appropriate cast to int64_t for INT64_FORMAT > > testing: > Compiled with bug-tickling configuration on > MacOS Mountain Lion (10.8) XCode 4.6.3 > Mavericks (10.9) XCode (4.6.3 and 5.1). > Verified that there were not other warnings tickled by this particular build configuration on either platform. > Also retested with fastdebug build on Mavericks. > From forax at univ-mlv.fr Wed May 14 20:23:51 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 14 May 2014 22:23:51 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <53736C7B.2040303@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> Message-ID: <5373D0D7.40407@univ-mlv.fr> On 05/14/2014 03:15 PM, Vladimir Ivanov wrote: > > On 5/14/14 4:12 PM, Paul Sandoz wrote: >> >> On May 14, 2014, at 12:47 PM, Vladimir Ivanov >> wrote: >> >>> Tobias, I agree with your evaluation. >>> >> >> V. tricky one to track down! >> >> From @Stable: >> >> * It is (currently) undefined what happens if a field annotated as >> stable >> * is given a third value. In practice, if the JVM relies on this >> annotation >> * to promote a field reference to a constant, it may be that the >> Java memory >> * model would appear to be broken, if such a constant (the second >> value of the field) >> * is used as the value of the field even after the field value has >> changed. >> >> I dunno if that was a contributing factor in this case. > No, @Stable doesn't contribute to the problem. > >> >>> My only concern is that @Stable doesn't work for >>> AtomicReferenceArray, so JIT doesn't see what is stored in the array. >> >> Yes, stability needs to be associated with the array elements. >> >> >>> Maybe use a lock instead? >>> >> >> Or perhaps use Unsafe.CAS directly within setCachedLambdaForm? > Yes, Unsafe is another option here. But since cache updates should be > rare, Unsafe is an overkill here IMO - locking should be fine. I don't get it, AtomicReferenceArray.CAS implementation uses Unsafe.CAS > >> >> Also, as a consequence of using AtomicReferenceArray the following >> change may result in a memory barrier on some architectures: >> >> public LambdaForm cachedLambdaForm(int which) { >> - return lambdaForms[which]; >> >> + return lambdaForms.get(which); >> } >> >> since lambdaForms.get will call Unsafe.getObjectVolatile. > MTF.cachedLambaForm isn't on a fast path, so it shouldn't be a problem. > >> Separately, i think code that calls setCachedLambdaForm needs to be >> double checked to ensure that the return value is used. For example, >> in MethodHandleImpl.makeGuardWithCatchForm i see: >> >> basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); >> return lform; >> >> which i think needs to be: >> >> return >> basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); > Good catch! MTF.setCachedLambdaForm usages should be fixed as well. IMO, forms can be a classical Java array, * final @Stable_LambdaForm[]_ lambdaForms;* setCachedLambdaForm can use Unsafe.compareAndSwapObject * // Make sure cache entry is set only once* * if (UNSAFE.compareAndSwap(***lambdaForms*, which, null, form)) {* * return form;* * } **return cachedLambdaForm(which);* and cachedLambdaForm can try to use the stable value, if the stable value is not null, it's ok, otherwise we can use getVolatileObject public LambdaForm cachedLambdaForm(int which) { LambdaForm form = lambdaForms[which]; if (form != null) { return form; } return UNSAFE.getObjectVolatile(lambdaForms, which); } cheers, R?mi > > Best regards, > Vladimir Ivanov > >> >> Paul. >> >> >> >> From vladimir.x.ivanov at oracle.com Wed May 14 20:50:46 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 15 May 2014 00:50:46 +0400 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5373D0D7.40407@univ-mlv.fr> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> Message-ID: <5373D726.10008@oracle.com> >>> Or perhaps use Unsafe.CAS directly within setCachedLambdaForm? >> Yes, Unsafe is another option here. But since cache updates should be >> rare, Unsafe is an overkill here IMO - locking should be fine. > > I don't get it, AtomicReferenceArray.CAS implementation uses Unsafe.CAS By overkill I meant code complexity - it increases for no benefit. If we use Unsafe directly, we need to duplicate offset calculation logic from AtomicReferenceArray. AtomicReferenceArray doesn't fit, because to declare an array as @Stable we need to work with it directly, but AtomicReferenceArray manages it's private copy we don't have access to. Best regards, Vladimir Ivanov > >> >>> >>> Also, as a consequence of using AtomicReferenceArray the following >>> change may result in a memory barrier on some architectures: >>> >>> public LambdaForm cachedLambdaForm(int which) { >>> - return lambdaForms[which]; >>> >>> + return lambdaForms.get(which); >>> } >>> >>> since lambdaForms.get will call Unsafe.getObjectVolatile. >> MTF.cachedLambaForm isn't on a fast path, so it shouldn't be a problem. >> >>> Separately, i think code that calls setCachedLambdaForm needs to be >>> double checked to ensure that the return value is used. For example, >>> in MethodHandleImpl.makeGuardWithCatchForm i see: >>> >>> basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); >>> return lform; >>> >>> which i think needs to be: >>> >>> return >>> basicType.form().setCachedLambdaForm(MethodTypeForm.LF_GWC, lform); >> Good catch! MTF.setCachedLambdaForm usages should be fixed as well. > > IMO, forms can be a classical Java array, > > * final @Stable_LambdaForm[]_ lambdaForms;* > > > setCachedLambdaForm can use Unsafe.compareAndSwapObject > > * // Make sure cache entry is set only once* > * if (UNSAFE.compareAndSwap(***lambdaForms*, which, null, form)) {* > * return form;* > * } > **return cachedLambdaForm(which);* > > > and cachedLambdaForm can try to use the stable value, > if the stable value is not null, it's ok, otherwise we can use > getVolatileObject > > public LambdaForm cachedLambdaForm(int which) { > LambdaForm form = lambdaForms[which]; > if (form != null) { > return form; > } > return UNSAFE.getObjectVolatile(lambdaForms, which); > } > > > cheers, > R?mi > >> >> Best regards, >> Vladimir Ivanov >> >>> >>> Paul. >>> >>> >>> >>> > From vladimir.kozlov at oracle.com Wed May 14 21:56:37 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 14 May 2014 14:56:37 -0700 Subject: RFR (XS) 8043164: Format warning in traceStream.hpp In-Reply-To: References: Message-ID: <5373E695.2090507@oracle.com> Good. Vladimir On 5/14/14 12:51 PM, David Chase wrote: > > bug: https://bugs.openjdk.java.net/browse/JDK-8043164 > webrev: http://cr.openjdk.java.net/~drchase/8043164/webrev.00/ > fix: add appropriate cast to int64_t for INT64_FORMAT > > testing: > Compiled with bug-tickling configuration on > MacOS Mountain Lion (10.8) XCode 4.6.3 > Mavericks (10.9) XCode (4.6.3 and 5.1). > Verified that there were not other warnings tickled by this particular build configuration on either platform. > Also retested with fastdebug build on Mavericks. > From david.holmes at oracle.com Thu May 15 04:20:17 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 May 2014 14:20:17 +1000 Subject: RFR (XS) JNI Specification Issue: JDK-6681965 Strings and NULL termination In-Reply-To: <53732D49.2060602@oracle.com> References: <53732D49.2060602@oracle.com> Message-ID: <53744081.2010002@oracle.com> Hi David, This clarification seems fine to me. Thanks, David H. On 14/05/2014 6:46 PM, David Simms wrote: > Bug/Enhancement: https://bugs.openjdk.java.net/browse/JDK-6681965 > Web review: http://cr.openjdk.java.net/~dsimms/jnispec/6681965/ > HTML version: > http://cr.openjdk.java.net/~dsimms/jnispec/6681965/JDK-6681965.html > > There are number of mail threads asking for clarification surrounding > string operations, and whether the results should be NULL terminated. > Functions affected: > > * const jchar * *GetStringChars*(JNIEnv *env, jstring string, jboolean > *isCopy); > * const char * *GetStringUTFChars*(JNIEnv *env, jstring string, > jboolean *isCopy); > * void *GetStringRegion*(JNIEnv *env, jstring str, jsize start, jsize > len, jchar *buf); > * void *GetStringUTFRegion*(JNIEnv *env, jstring str, jsize start, > jsize len, char *buf); > * const jchar * *GetStringCritical*(JNIEnv *env, jstring string, > jboolean *isCopy); > > There are two stories here, how the particular JVM implements Java > Strings internally, and the JNI Specification. I believe these two > things are separate: > > * From a HotSpot JVM perspective, null-terminating strings might be > seen as "safety first". > * There may be other JVM implementations whom, for argument sake, that > never garbage collect and always use direct heap pointers. Such > implementations have no reason to null-terminate... > * ...indeed HotSpot's "GetStringCritical()" implementation does no > such thing. > > Currently the JNI Specification does not mention null-termination, and > given we are not dealing with c-strings, but Java Strings, that is > because it's not required. > > "GetStringUTFRegion()" as currently implemented by HotSpot has an issue, > it is possible to null-terminate past the user specified length (which > in itself a bit of an issue, its related to the input, not output, being > UTF-16 length != UT8 encoded length). > > I'm not advocating removing the current null-termination where HotSpot > supplies the buffers. In fact I'm worried will break too much existing > code. There is no harm in null-termination where the JVM has control > over the buffer. > > Clarify NULL-termination, or rather lack of, for string operations. > Additional text at the beginning of the "String Operations" section: > > ---------------------------------------------------------------------- > String Operations > > This specification makes no assumptions on how a JVM represents Java > strings internally. Strings returned from these operations: > > * GetStringChars() > * GetStringUTFChars() > * GetStringRegion() > * GetStringUTFRegion() > * GetStringCritical() > > are therefore not required to be NULL terminated. Programmers are > expected to determine buffer capacity requirements > via"|GetStringLength()| > " > or "|GetStringUTFLength()| > ". > > > ---------------------------------------------------------------------- > > /David Simms From stefan.karlsson at oracle.com Thu May 15 06:24:44 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 15 May 2014 08:24:44 +0200 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <53716503.7060503@oracle.com> References: <53716503.7060503@oracle.com> Message-ID: <53745DAC.9080105@oracle.com> Hi Coleen, On 2014-05-13 02:19, Coleen Phillimore wrote: > Summary: Only prune metaspace virtual spaces at safepoint so walking > them is safe outside a safepoint. > > Walking class loader data graph for contains is really slow for > applications like nashorn that has a lot of class loader data. > > Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran > > open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ > bug link https://bugs.openjdk.java.net/browse/JDK-8038212 Thanks for fixing this. I looked at the version at: http://cr.openjdk.java.net/~coleenp/8038212_3 Could you explain why you added the storestore here? Shouldn't it be placed after the node has been initialized but before it is linked into the list? *** 1219,1228 **** --- 1237,1248 ---- if (virtual_space_list() == NULL) { set_virtual_space_list(new_entry); } else { current_virtual_space()->set_next(new_entry); } + // ensure lock-free iteration sees fully initialized node + OrderAccess::storestore(); set_current_virtual_space(new_entry); inc_reserved_words(new_entry->reserved_words()); inc_committed_words(new_entry->committed_words()); inc_virtual_space_count(); #ifdef ASSERT Otherwise, this looks good. thanks, StefanK > > Marcus L. tested it with nashorn tests. > > Thanks, > Coleen From erik.helin at oracle.com Thu May 15 07:09:15 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 15 May 2014 09:09:15 +0200 Subject: RFR: 8039489: Refactor test framework for dynamic VM options In-Reply-To: <53737D83.2050205@oracle.com> References: <535A524D.1000701@oracle.com> <53737D83.2050205@oracle.com> Message-ID: <2853847.YKHQLIsHKd@ehelin-laptop> Hi Dmitry, the patch looks good to me, thanks for taking care of these tests and also updating the testlibrary! Thanks, Erik On Wednesday 14 May 2014 18:28:19 PM Dmitry Fazunenko wrote: > Hi, > > Would you please look at the new version of a common library for testing > dynamic VM options. > > Description: > Some time ago a library for testing VM flags which could be changed > dynamically was integrated into the hs repository: > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOptionChecker.java > test/testlibrary/com/oracle/java/testlibrary/TestDynamicVMOption.java > > The library was designed in not the most optimal way. > This change is an attempt to redesign it: > > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOption.java > > is a new version which provides a convenient access to the HS API on > changing VM flags. > The tests based on the library are also refactored to meet new API. > > You comments will be much appreciated. > > http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8039489 > > I want to thank Erik Helin for his incredible assistance with working on > this change. > > Thanks, > Dima From goetz.lindenmaier at sap.com Thu May 15 08:16:01 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 15 May 2014 08:16:01 +0000 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> Hi, This change introduces an umbrella header for prefetch__.inline.hpp files: runtime/prefetch.inline.hpp as proposed in http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html This follows the scheme applied to atomic.inline.hpp. Further this change adds includes of prefetch.inline.hpp in all .cpp and .inline.hpp files where a method of Prefetch declared 'inline' is called. Finally it moves macros calling inline methods of Prefetch from space.hpp to the corresponding .inline.hpp file and adds the necessary includes in files using the moved macros. Please review and test this change. I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ Thanks and best regards, Goetz. From forax at univ-mlv.fr Thu May 15 08:36:35 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 15 May 2014 10:36:35 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5373D726.10008@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> Message-ID: <53747C93.3030508@univ-mlv.fr> On 05/14/2014 10:50 PM, Vladimir Ivanov wrote: >>>> Or perhaps use Unsafe.CAS directly within setCachedLambdaForm? >>> Yes, Unsafe is another option here. But since cache updates should be >>> rare, Unsafe is an overkill here IMO - locking should be fine. >> >> I don't get it, AtomicReferenceArray.CAS implementation uses Unsafe.CAS > By overkill I meant code complexity - it increases for no benefit. > > If we use Unsafe directly, we need to duplicate offset calculation > logic from AtomicReferenceArray. yes > > AtomicReferenceArray doesn't fit, because to declare an array as > @Stable we need to work with it directly, but AtomicReferenceArray > manages it's private copy we don't have access to. yes, I may be wrong but for me cachedLambdaForm() is used in a fast path. If it's not the case, I agree that a lock is enough. > > Best regards, > Vladimir Ivanov regards, R?mi From paul.sandoz at oracle.com Thu May 15 08:53:37 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 15 May 2014 10:53:37 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5373D726.10008@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> Message-ID: On May 14, 2014, at 10:50 PM, Vladimir Ivanov wrote: >>>> Or perhaps use Unsafe.CAS directly within setCachedLambdaForm? >>> Yes, Unsafe is another option here. But since cache updates should be >>> rare, Unsafe is an overkill here IMO - locking should be fine. >> >> I don't get it, AtomicReferenceArray.CAS implementation uses Unsafe.CAS > By overkill I meant code complexity - it increases for no benefit. > > If we use Unsafe directly, we need to duplicate offset calculation logic from AtomicReferenceArray. > While i find that simple [*], i tend to agree that a lock in setCachedLambdaForm is simpler for this case (and cachedLambdaForm could be left as is). Paul. [*] There are constants on Unsafe, Unsafe.ARRAY_OBJECT_BASE_OFFSET and Unsafe.ARRAY_OBJECT_INDEX_SCALE, although commonly one would prefer to do address = (i << shift) + base, rather than address = i * scale + base: boolean r = Unsafe.compareAndSwapObject(lambdaForms, which * Unsafe.ARRAY_OBJECT_INDEX_SCALE + Unsafe.ARRAY_OBJECT_BASE_OFFSET, null, form); return r ? form : lambdaForms[which]; // always returns non-null value From vladimir.x.ivanov at oracle.com Thu May 15 09:04:47 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 15 May 2014 13:04:47 +0400 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <53747C93.3030508@univ-mlv.fr> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> Message-ID: <5374832F.5000800@oracle.com> > > I may be wrong but for me cachedLambdaForm() is used in a fast path. > If it's not the case, I agree that a lock is enough. I hold the opinion that we only need to synchronize writers and not readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). Current usage pattern is the following: MethodType type = ...; LambdaForm lform = type.form().cachedLambdaForm(idx); if (lform == NULL) { // construct new LambaForm lform = type.form().setCachedLambdaForm(idx, lform); } return lform; Cache is written only once, so it has only 2 states (null and non-null value) during it's lifecycle. The only stale value cachedLambdaForm() can see is null, but then the caller tries to initialize the cache and after acquiring the lock in setCachedLambdaForm() sees actual value. So, the worst thing can happen if readers aren't synchronized is creation of unnecessary LF (which go dead right away) in rare cases. Best regards, Vladimir Ivanov > >> >> Best regards, >> Vladimir Ivanov > > regards, > R?mi > From roland.westrelin at oracle.com Thu May 15 09:39:35 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Thu, 15 May 2014 11:39:35 +0200 Subject: [8u20] 8041481: JVM crashes with collect_args_for_profiling Message-ID: <99063C7A-C6FD-407C-9302-9D0698BB2441@oracle.com> 8u20 backport request. The change was pushed to jdk9 almost 3 weeks ago and nightly testing hasn?t shown any problem. The change applies cleanly to 8u. http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/a35e424a76a4 https://bugs.openjdk.java.net/browse/JDK-8041481 Roland. From david.holmes at oracle.com Thu May 15 11:10:32 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 May 2014 21:10:32 +1000 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5374832F.5000800@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> Message-ID: <5374A0A8.1000805@oracle.com> On 15/05/2014 7:04 PM, Vladimir Ivanov wrote: > > >> I may be wrong but for me cachedLambdaForm() is used in a fast path. >> If it's not the case, I agree that a lock is enough. > I hold the opinion that we only need to synchronize writers and not > readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). > > Current usage pattern is the following: > MethodType type = ...; > LambdaForm lform = type.form().cachedLambdaForm(idx); > if (lform == NULL) { > // construct new LambaForm > lform = type.form().setCachedLambdaForm(idx, lform); > } > return lform; > > Cache is written only once, so it has only 2 states (null and non-null > value) during it's lifecycle. > > The only stale value cachedLambdaForm() can see is null, but then the > caller tries to initialize the cache and after acquiring the lock in > setCachedLambdaForm() sees actual value. > > So, the worst thing can happen if readers aren't synchronized is > creation of unnecessary LF (which go dead right away) in rare cases. Unless the cache is volatile this would be unsafe publication and the reader may see a non-null reference to the LF but the LF may not be properly initialized. David ----- > Best regards, > Vladimir Ivanov >> >>> >>> Best regards, >>> Vladimir Ivanov >> >> regards, >> R?mi >> From vladimir.x.ivanov at oracle.com Thu May 15 11:29:17 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 15 May 2014 15:29:17 +0400 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5374A0A8.1000805@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> Message-ID: <5374A50D.3080704@oracle.com> >>> I may be wrong but for me cachedLambdaForm() is used in a fast path. >>> If it's not the case, I agree that a lock is enough. >> I hold the opinion that we only need to synchronize writers and not >> readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). >> >> Current usage pattern is the following: >> MethodType type = ...; >> LambdaForm lform = type.form().cachedLambdaForm(idx); >> if (lform == NULL) { >> // construct new LambaForm >> lform = type.form().setCachedLambdaForm(idx, lform); >> } >> return lform; >> >> Cache is written only once, so it has only 2 states (null and non-null >> value) during it's lifecycle. >> >> The only stale value cachedLambdaForm() can see is null, but then the >> caller tries to initialize the cache and after acquiring the lock in >> setCachedLambdaForm() sees actual value. >> >> So, the worst thing can happen if readers aren't synchronized is >> creation of unnecessary LF (which go dead right away) in rare cases. > > Unless the cache is volatile this would be unsafe publication and the > reader may see a non-null reference to the LF but the LF may not be > properly initialized. Good point. The cache is an array element. How does safe publication of array element look like? Is accessing the element through volatile reference to the array enough or additional synchronization actions are needed? Best regards, Vladimir Ivanov > > David > ----- > > >> Best regards, >> Vladimir Ivanov >>> >>>> >>>> Best regards, >>>> Vladimir Ivanov >>> >>> regards, >>> R?mi >>> From david.holmes at oracle.com Thu May 15 12:54:51 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 May 2014 22:54:51 +1000 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5374A50D.3080704@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> Message-ID: <5374B91B.2000400@oracle.com> On 15/05/2014 9:29 PM, Vladimir Ivanov wrote: > >>> I may be wrong but for me cachedLambdaForm() is used in a fast path. >>>> If it's not the case, I agree that a lock is enough. >>> I hold the opinion that we only need to synchronize writers and not >>> readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). >>> >>> Current usage pattern is the following: >>> MethodType type = ...; >>> LambdaForm lform = type.form().cachedLambdaForm(idx); >>> if (lform == NULL) { >>> // construct new LambaForm >>> lform = type.form().setCachedLambdaForm(idx, lform); >>> } >>> return lform; >>> >>> Cache is written only once, so it has only 2 states (null and non-null >>> value) during it's lifecycle. >>> >>> The only stale value cachedLambdaForm() can see is null, but then the >>> caller tries to initialize the cache and after acquiring the lock in >>> setCachedLambdaForm() sees actual value. >>> >>> So, the worst thing can happen if readers aren't synchronized is >>> creation of unnecessary LF (which go dead right away) in rare cases. >> >> Unless the cache is volatile this would be unsafe publication and the >> reader may see a non-null reference to the LF but the LF may not be >> properly initialized. > Good point. The cache is an array element. How does safe publication of > array element look like? Is accessing the element through volatile > reference to the array enough or additional synchronization actions are > needed? Additional sync would be needed, such as using an AtomicReferenceArray. Otherwise a volatile array reference doesn't make the array elements volatile. David > Best regards, > Vladimir Ivanov > >> >> David >> ----- >> >> >>> Best regards, >>> Vladimir Ivanov >>>> >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>> >>>> regards, >>>> R?mi >>>> From paul.sandoz at oracle.com Thu May 15 14:26:28 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 15 May 2014 16:26:28 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5374A50D.3080704@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> Message-ID: <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> On May 15, 2014, at 1:29 PM, Vladimir Ivanov wrote: > >>> I may be wrong but for me cachedLambdaForm() is used in a fast path. >>>> If it's not the case, I agree that a lock is enough. >>> I hold the opinion that we only need to synchronize writers and not >>> readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). >>> >>> Current usage pattern is the following: >>> MethodType type = ...; >>> LambdaForm lform = type.form().cachedLambdaForm(idx); >>> if (lform == NULL) { >>> // construct new LambaForm >>> lform = type.form().setCachedLambdaForm(idx, lform); >>> } >>> return lform; >>> >>> Cache is written only once, so it has only 2 states (null and non-null >>> value) during it's lifecycle. >>> >>> The only stale value cachedLambdaForm() can see is null, but then the >>> caller tries to initialize the cache and after acquiring the lock in >>> setCachedLambdaForm() sees actual value. >>> >>> So, the worst thing can happen if readers aren't synchronized is >>> creation of unnecessary LF (which go dead right away) in rare cases. >> >> Unless the cache is volatile this would be unsafe publication and the >> reader may see a non-null reference to the LF but the LF may not be >> properly initialized. > Good point. The cache is an array element. How does safe publication of array element look like? Is accessing the element through volatile reference to the array enough or additional synchronization actions are needed? > I thought that since the only fields explicitly set when constructing a LambdaForm are final fields it should be safe to publish the form reference as an element in the array? Paul. From christian.thalinger at oracle.com Thu May 15 15:13:47 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 15 May 2014 08:13:47 -0700 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> Message-ID: Looks good to me. I thought that prefetch is used more often but apparently not. On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: > Hi, > > This change introduces an umbrella header for prefetch__.inline.hpp > files: runtime/prefetch.inline.hpp as proposed in > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html > This follows the scheme applied to atomic.inline.hpp. > > Further this change adds includes of prefetch.inline.hpp in all .cpp > and .inline.hpp files where a method of Prefetch declared 'inline' is > called. > > Finally it moves macros calling inline methods of Prefetch from > space.hpp to the corresponding .inline.hpp file and adds the necessary > includes in files using the moved macros. > > Please review and test this change. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ > > Thanks and best regards, > Goetz. From vladimir.kozlov at oracle.com Thu May 15 17:01:03 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 15 May 2014 10:01:03 -0700 Subject: [8u20] 8041481: JVM crashes with collect_args_for_profiling In-Reply-To: <99063C7A-C6FD-407C-9302-9D0698BB2441@oracle.com> References: <99063C7A-C6FD-407C-9302-9D0698BB2441@oracle.com> Message-ID: <5374F2CF.407@oracle.com> Good. Thanks, Vladimir On 5/15/14 2:39 AM, Roland Westrelin wrote: > 8u20 backport request. The change was pushed to jdk9 almost 3 weeks ago and nightly testing hasn?t shown any problem. > > The change applies cleanly to 8u. > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/a35e424a76a4 > https://bugs.openjdk.java.net/browse/JDK-8041481 > > Roland. > From serguei.spitsyn at oracle.com Thu May 15 17:21:59 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 15 May 2014 10:21:59 -0700 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <53739E1E.1080002@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> <5372B94C.4060905@oracle.com> <53739E1E.1080002@oracle.com> Message-ID: <5374F7B7.8030103@oracle.com> Hi Coleen and Dan, I've created a j.l.i test covering this issue. The jdk webrev is: http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/jdk/8042796-JVMTI-del-tests.1 The updated hotspot webrev is: http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-del.1 I guess, it is Ok to push the hotspot fix while the test is reviewed? Thanks, Serguei On 5/14/14 9:47 AM, Coleen Phillimore wrote: > > Hi Serguei, > > On 5/13/14, 8:31 PM, serguei.spitsyn at oracle.com wrote: >> Coleen, >> >> >> On 5/13/14 4:17 PM, Coleen Phillimore wrote: >>> >>> Serguei, >>> >>> I don't know if this is the right fix. We shouldn't have deleted >>> methods in the cpCache because there's no way to keep these Method*s >>> from being deallocated. I thought AdjustCpoolCacheAndVtable => >>> ConstantPoolCache::adjust_method_entries should fix these entries. >>> For deleted methods, shouldn't their cpCache entries be set to >>> unresolved? Which class had the deleted methods? The one being >>> redefined? >> >> Thank you for reviewing this and for the questions. >> >> Let's check if my understanding is right as it is relatively easy to >> miss something. >> >> The original class has been redefined so that the new version does >> not have several private static methods anymore. >> These methods are called "deleted" but they are still present in the >> original version of the class. >> Even if we set their cpCache entries to unresolved these entries can >> be set to resolved again. >> So that I've decided this approach is not going to work in general case. > > Okay, so you answered my question - since the deleted methods are > private, they can only be found in the class being redefined's > constant pool cache. Since the class being redefined's cpCache > hasn't been deallocated, neither will the Method* that it's pointing > to. Phew! Thank you for the explanation. > >> >> The interesting question is: >> Q: What is going to happen with these entries when the original >> version of the class goes away? >> > > Everything will get deallocated, which is fine. > >> I'll try to answer this question by looking at the code. >> It won't be a surprise if there is nothing in place to cover this case. >> >>> >>> It looks like the code never did this. I think this is missing >>> functionality. >> >> Yes, that was my guess too. >> >>> Do we not have any tests that delete a method? >> >> Most likely, we don't. >> >>> Your change should add a test anyway - there are class redefinition >>> tests in jdk/test/java/lang/instrument that can be adapted and we >>> should probably keep them all together there. >> >> Right, it is better to have some test coverage here. >> I'll check if any of the JLI or JDI tests can be extended to cover this. >> I'm not sure the JLI is good for this task though as deleting the >> methods is not instrumentation. :) >> > > So the change looks good pending a test case. There might be one > already but why didn't it fail this assert? So, we need a test for > this case somewhere. > > Thanks, > Coleen > >> >> Thanks! >> Serguei >> >>> >>> Thanks, >>> Coleen >>> >>> On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8042796 >>>> >>>> >>>> Open webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >>>> >>>> >>>> Summary: >>>> >>>> This is a Nightly Stabilization issue that was caused by a >>>> combination of two problems: >>>> - A regression introduced by the fix of: >>>> https://bugs.openjdk.java.net/browse/JDK-7182152 >>>> - An SQE testbase infra regression: >>>> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >>>> >>>> A number of the vm.mlvm tests hits this guarantee taht was added >>>> by 7182152 (must be relaxed). >>>> The issue is with the deleted static private methods that are >>>> still present in the CP cache. >>>> The fix is to mark the deleted methods with the flag >>>> JVM_ACC_IS_DELETED and >>>> then use it to relax the guarantee condition. >>>> >>>> >>>> Testing: >>>> Running the failing tests: vm.mlvm.indy.func.jvmti >>>> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs >>>> on sparcv9 and amd64. >>>> >>>> >>>> Thanks, >>>> Serguei >>> >> > From forax at univ-mlv.fr Thu May 15 18:25:24 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 15 May 2014 20:25:24 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> Message-ID: <53750694.3080702@univ-mlv.fr> On 05/15/2014 04:26 PM, Paul Sandoz wrote: > On May 15, 2014, at 1:29 PM, Vladimir Ivanov wrote: > >>>>> I may be wrong but for me cachedLambdaForm() is used in a fast path. >>>>> If it's not the case, I agree that a lock is enough. >>>> I hold the opinion that we only need to synchronize writers and not >>>> readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). >>>> >>>> Current usage pattern is the following: >>>> MethodType type = ...; >>>> LambdaForm lform = type.form().cachedLambdaForm(idx); >>>> if (lform == NULL) { >>>> // construct new LambaForm >>>> lform = type.form().setCachedLambdaForm(idx, lform); >>>> } >>>> return lform; >>>> >>>> Cache is written only once, so it has only 2 states (null and non-null >>>> value) during it's lifecycle. >>>> >>>> The only stale value cachedLambdaForm() can see is null, but then the >>>> caller tries to initialize the cache and after acquiring the lock in >>>> setCachedLambdaForm() sees actual value. >>>> >>>> So, the worst thing can happen if readers aren't synchronized is >>>> creation of unnecessary LF (which go dead right away) in rare cases. >>> Unless the cache is volatile this would be unsafe publication and the >>> reader may see a non-null reference to the LF but the LF may not be >>> properly initialized. >> Good point. The cache is an array element. How does safe publication of array element look like? Is accessing the element through volatile reference to the array enough or additional synchronization actions are needed? >> > I thought that since the only fields explicitly set when constructing a LambdaForm are final fields it should be safe to publish the form reference as an element in the array? > > Paul. > that's my analysis too. R?mi From coleen.phillimore at oracle.com Thu May 15 20:24:20 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 15 May 2014 16:24:20 -0400 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <5374F7B7.8030103@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> <5372B94C.4060905@oracle.com> <53739E1E.1080002@oracle.com> <5374F7B7.8030103@oracle.com> Message-ID: <53752274.1010105@oracle.com> Serguei, This looks good. We should eventually extend these tests to have jmethodIDs and stuff like that. Yes, you have to push the hotspot fix first, then wait a week or so to push the JDK one since pushes aren't coordinated just yet, and you don't want jdk to have a failing test until hotspot is integrated. Thanks for figuring this out and the discussion! Coleen On 5/15/14, 1:21 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen and Dan, > > I've created a j.l.i test covering this issue. > The jdk webrev is: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/jdk/8042796-JVMTI-del-tests.1 > > > The updated hotspot webrev is: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-del.1 > > > I guess, it is Ok to push the hotspot fix while the test is reviewed? > > Thanks, > Serguei > > > On 5/14/14 9:47 AM, Coleen Phillimore wrote: >> >> Hi Serguei, >> >> On 5/13/14, 8:31 PM, serguei.spitsyn at oracle.com wrote: >>> Coleen, >>> >>> >>> On 5/13/14 4:17 PM, Coleen Phillimore wrote: >>>> >>>> Serguei, >>>> >>>> I don't know if this is the right fix. We shouldn't have deleted >>>> methods in the cpCache because there's no way to keep these >>>> Method*s from being deallocated. I thought >>>> AdjustCpoolCacheAndVtable => >>>> ConstantPoolCache::adjust_method_entries should fix these entries. >>>> For deleted methods, shouldn't their cpCache entries be set to >>>> unresolved? Which class had the deleted methods? The one being >>>> redefined? >>> >>> Thank you for reviewing this and for the questions. >>> >>> Let's check if my understanding is right as it is relatively easy to >>> miss something. >>> >>> The original class has been redefined so that the new version does >>> not have several private static methods anymore. >>> These methods are called "deleted" but they are still present in the >>> original version of the class. >>> Even if we set their cpCache entries to unresolved these entries can >>> be set to resolved again. >>> So that I've decided this approach is not going to work in general >>> case. >> >> Okay, so you answered my question - since the deleted methods are >> private, they can only be found in the class being redefined's >> constant pool cache. Since the class being redefined's cpCache >> hasn't been deallocated, neither will the Method* that it's pointing >> to. Phew! Thank you for the explanation. >> >>> >>> The interesting question is: >>> Q: What is going to happen with these entries when the original >>> version of the class goes away? >>> >> >> Everything will get deallocated, which is fine. >> >>> I'll try to answer this question by looking at the code. >>> It won't be a surprise if there is nothing in place to cover this case. >>> >>>> >>>> It looks like the code never did this. I think this is missing >>>> functionality. >>> >>> Yes, that was my guess too. >>> >>>> Do we not have any tests that delete a method? >>> >>> Most likely, we don't. >>> >>>> Your change should add a test anyway - there are class redefinition >>>> tests in jdk/test/java/lang/instrument that can be adapted and we >>>> should probably keep them all together there. >>> >>> Right, it is better to have some test coverage here. >>> I'll check if any of the JLI or JDI tests can be extended to cover >>> this. >>> I'm not sure the JLI is good for this task though as deleting the >>> methods is not instrumentation. :) >>> >> >> So the change looks good pending a test case. There might be one >> already but why didn't it fail this assert? So, we need a test for >> this case somewhere. >> >> Thanks, >> Coleen >> >>> >>> Thanks! >>> Serguei >>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8042796 >>>>> >>>>> >>>>> Open webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >>>>> >>>>> >>>>> Summary: >>>>> >>>>> This is a Nightly Stabilization issue that was caused by a >>>>> combination of two problems: >>>>> - A regression introduced by the fix of: >>>>> https://bugs.openjdk.java.net/browse/JDK-7182152 >>>>> - An SQE testbase infra regression: >>>>> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >>>>> >>>>> A number of the vm.mlvm tests hits this guarantee taht was added >>>>> by 7182152 (must be relaxed). >>>>> The issue is with the deleted static private methods that are >>>>> still present in the CP cache. >>>>> The fix is to mark the deleted methods with the flag >>>>> JVM_ACC_IS_DELETED and >>>>> then use it to relax the guarantee condition. >>>>> >>>>> >>>>> Testing: >>>>> Running the failing tests: vm.mlvm.indy.func.jvmti >>>>> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs >>>>> on sparcv9 and amd64. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>> >>> >> > From serguei.spitsyn at oracle.com Thu May 15 21:00:27 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 15 May 2014 14:00:27 -0700 Subject: RFR (S) 8042796: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <53752274.1010105@oracle.com> References: <536D1C8A.4080503@oracle.com> <5372A7EE.2080109@oracle.com> <5372B94C.4060905@oracle.com> <53739E1E.1080002@oracle.com> <5374F7B7.8030103@oracle.com> <53752274.1010105@oracle.com> Message-ID: <53752AEB.3030307@oracle.com> Thank you, Coleen! Serguei On 5/15/14 1:24 PM, Coleen Phillimore wrote: > > Serguei, > > This looks good. > We should eventually extend these tests to have jmethodIDs and stuff > like that. > > Yes, you have to push the hotspot fix first, then wait a week or so to > push the JDK one since pushes aren't coordinated just yet, and you > don't want jdk to have a failing test until hotspot is integrated. > > Thanks for figuring this out and the discussion! > > Coleen > > On 5/15/14, 1:21 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen and Dan, >> >> I've created a j.l.i test covering this issue. >> The jdk webrev is: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/jdk/8042796-JVMTI-del-tests.1 >> >> >> The updated hotspot webrev is: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-del.1 >> >> >> I guess, it is Ok to push the hotspot fix while the test is reviewed? >> >> Thanks, >> Serguei >> >> >> On 5/14/14 9:47 AM, Coleen Phillimore wrote: >>> >>> Hi Serguei, >>> >>> On 5/13/14, 8:31 PM, serguei.spitsyn at oracle.com wrote: >>>> Coleen, >>>> >>>> >>>> On 5/13/14 4:17 PM, Coleen Phillimore wrote: >>>>> >>>>> Serguei, >>>>> >>>>> I don't know if this is the right fix. We shouldn't have deleted >>>>> methods in the cpCache because there's no way to keep these >>>>> Method*s from being deallocated. I thought >>>>> AdjustCpoolCacheAndVtable => >>>>> ConstantPoolCache::adjust_method_entries should fix these entries. >>>>> For deleted methods, shouldn't their cpCache entries be set to >>>>> unresolved? Which class had the deleted methods? The one being >>>>> redefined? >>>> >>>> Thank you for reviewing this and for the questions. >>>> >>>> Let's check if my understanding is right as it is relatively easy >>>> to miss something. >>>> >>>> The original class has been redefined so that the new version does >>>> not have several private static methods anymore. >>>> These methods are called "deleted" but they are still present in >>>> the original version of the class. >>>> Even if we set their cpCache entries to unresolved these entries >>>> can be set to resolved again. >>>> So that I've decided this approach is not going to work in general >>>> case. >>> >>> Okay, so you answered my question - since the deleted methods are >>> private, they can only be found in the class being redefined's >>> constant pool cache. Since the class being redefined's cpCache >>> hasn't been deallocated, neither will the Method* that it's pointing >>> to. Phew! Thank you for the explanation. >>> >>>> >>>> The interesting question is: >>>> Q: What is going to happen with these entries when the original >>>> version of the class goes away? >>>> >>> >>> Everything will get deallocated, which is fine. >>> >>>> I'll try to answer this question by looking at the code. >>>> It won't be a surprise if there is nothing in place to cover this >>>> case. >>>> >>>>> >>>>> It looks like the code never did this. I think this is missing >>>>> functionality. >>>> >>>> Yes, that was my guess too. >>>> >>>>> Do we not have any tests that delete a method? >>>> >>>> Most likely, we don't. >>>> >>>>> Your change should add a test anyway - there are class >>>>> redefinition tests in jdk/test/java/lang/instrument that can be >>>>> adapted and we should probably keep them all together there. >>>> >>>> Right, it is better to have some test coverage here. >>>> I'll check if any of the JLI or JDI tests can be extended to cover >>>> this. >>>> I'm not sure the JLI is good for this task though as deleting the >>>> methods is not instrumentation. :) >>>> >>> >>> So the change looks good pending a test case. There might be one >>> already but why didn't it fail this assert? So, we need a test for >>> this case somewhere. >>> >>> Thanks, >>> Coleen >>> >>>> >>>> Thanks! >>>> Serguei >>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 5/9/14, 2:20 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8042796 >>>>>> >>>>>> >>>>>> Open webrev: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8042796-JVMTI-OLD.1 >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> This is a Nightly Stabilization issue that was caused by a >>>>>> combination of two problems: >>>>>> - A regression introduced by the fix of: >>>>>> https://bugs.openjdk.java.net/browse/JDK-7182152 >>>>>> - An SQE testbase infra regression: >>>>>> https://bugs.openjdk.java.net/browse/INTJDK-7611018 >>>>>> >>>>>> A number of the vm.mlvm tests hits this guarantee taht was >>>>>> added by 7182152 (must be relaxed). >>>>>> The issue is with the deleted static private methods that are >>>>>> still present in the CP cache. >>>>>> The fix is to mark the deleted methods with the flag >>>>>> JVM_ACC_IS_DELETED and >>>>>> then use it to relax the guarantee condition. >>>>>> >>>>>> >>>>>> Testing: >>>>>> Running the failing tests: vm.mlvm.indy.func.jvmti >>>>>> In progress: nsk.jvmti, nsk.jdi, java.lang.instrument test runs >>>>>> on sparcv9 and amd64. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Thu May 15 21:54:13 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 15 May 2014 17:54:13 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <53745DAC.9080105@oracle.com> References: <53716503.7060503@oracle.com> <53745DAC.9080105@oracle.com> Message-ID: <53753785.2000203@oracle.com> Hi Stefan, Thank you for reviewing again. On 5/15/14, 2:24 AM, Stefan Karlsson wrote: > Hi Coleen, > > On 2014-05-13 02:19, Coleen Phillimore wrote: >> Summary: Only prune metaspace virtual spaces at safepoint so walking >> them is safe outside a safepoint. >> >> Walking class loader data graph for contains is really slow for >> applications like nashorn that has a lot of class loader data. >> >> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also ran >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 > > Thanks for fixing this. > > I looked at the version at: > http://cr.openjdk.java.net/~coleenp/8038212_3 > > Could you explain why you added the storestore here? Shouldn't it be > placed after the node has been initialized but before it is linked > into the list? > *** 1219,1228 **** > --- 1237,1248 ---- > if (virtual_space_list() == NULL) { > set_virtual_space_list(new_entry); > } else { > current_virtual_space()->set_next(new_entry); > } > + // ensure lock-free iteration sees fully initialized node > + OrderAccess::storestore(); > set_current_virtual_space(new_entry); > inc_reserved_words(new_entry->reserved_words()); > inc_committed_words(new_entry->committed_words()); > inc_virtual_space_count(); > #ifdef ASSERT I meant to revert this to where it was before the change to iterate over ClassLoaderDataGraph, but forgot where it was. I have restored it to where it was now, here: @@ -1210,6 +1228,8 @@ } else { assert(new_entry->reserved_words() == vs_word_size, "Reserved memory size differs from requested memory size"); + // ensure lock-free iteration sees fully initialized node + OrderAccess::storestore(); link_vs(new_entry); return true; } Thanks for catching that. Coleen > > Otherwise, this looks good. > > thanks, > StefanK >> >> Marcus L. tested it with nashorn tests. >> >> Thanks, >> Coleen > From coleen.phillimore at oracle.com Thu May 15 21:54:46 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 15 May 2014 17:54:46 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <537366AD.3030602@oracle.com> References: <53716503.7060503@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> <5372691F.2030107@oracle.com> <3786299.dyoyOuaSQ9@vboxbuntu> <53727E78.7030307@oracle.com> <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> <537366AD.3030602@oracle.com> Message-ID: <537537A6.2060203@oracle.com> Thank you Harold! Coleen On 5/14/14, 8:50 AM, harold seigel wrote: > Hi Coleen, > > Your changes look good to me, also. > > Harold > > On 5/14/2014 5:10 AM, Markus Gr?nlund wrote: >> Hi Coleen, >> >> I have taken a look as well as running the nashorn crypto tests with >> the patches - these changes looks good. >> >> Thanks for addressing this. >> >> /Markus >> >> >> -----Original Message----- >> From: Coleen Phillimore >> Sent: den 13 maj 2014 22:20 >> To: Mikael Gerdin >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: RFR 8038212: Method::is_valid_method() check has >> performance regression impact for stackwalking >> >> >> On 5/13/14, 3:38 PM, Mikael Gerdin wrote: >>> On Tuesday 13 May 2014 14.49.03 Coleen Phillimore wrote: >>>> On 5/13/14, 2:41 PM, Mikael Gerdin wrote: >>>>> On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: >>>>>> Hi again, >>>>>> >>>>>> I have moved the metaspace vm internal test to the end of >>>>>> execution, and also test if klasses are in metaspace, which works >>>>>> fine. Thank you for the suggestion! Here is a new webrev: >>>>> I'm not too happy about moving the test to the before_exit handler. >>>> Why not? >>> I guess that most of the other unit tests are (and in my opinion >>> should be) as independent as possible of JVM state such as loaded >>> classes. >>> I was fine with the test before since it's a reasonably small amount >>> of code to get a test and it adds some test coverage. >>> When the change involves adding additional hooks for unit tests in the >>> exit path I'm just slightly more bothered by it. >>> >>> If we want a proper unit test for Metaspace::contains which tests all >>> sorts of alllocations/classloaders and such then it needs to be a bit >>> more explicit with setting up a known test environment. >>> >>>>> Also, how can using the run_unit_test-macro (defined in jni.cpp) >>>>> even work in java.cpp? >>>> It's moved to globalDefinitions.hpp. >>> I see. >>> >>>>> About methods and classes being available I think we may have >>>>> preloaded some classes in Threads::create_vm, right? >>>> Yes, we have loaded classes before the initial ExecuteVMInternalTests >>>> but they are only in the boot class loader, which is only in one >>>> metaspace. So it's not much of a test. At least at the end, we will >>>> have classes in the application class loader that we are testing >>>> whether they are contained in metaspace. >>> Ok. >>> >>>> I am not motivated to add the test for just for the sake of having a >>>> test. This change fixes the nashorn performance problem. Maybe >>>> that's >>>> test enough. >>>> >>>> before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 >>>> after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 >>> Well, I guess the test isn't really for the correctness of the >>> ::contains function anyway? And we don't really have a good way to do >>> a measured test to make sure that ::contains isn't "too slow". >> We actually already test that classes are Metaspace::contains() in >> the system dictionary verification at the end of the debug build >> (Dictionary::verify => instanceKlass::verify => Metaspace::contains) >> >> I think this test adds nothing now and given your concerns, I'm going >> to revert it. >> >> Thanks, >> Coleen >> >>> /Mikael >>> >>>> Coleen >>>> >>>>> /Mikael >>>>> >>>>>> http://cr.openjdk.java.net/~coleenp/8038212_2/ >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: >>>>>>> Hi Mikael, >>>>>>> >>>>>>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>>>>>>>> Summary: Only prune metaspace virtual spaces at safepoint so >>>>>>>>> walking them is safe outside a safepoint. >>>>>>>>> >>>>>>>>> Walking class loader data graph for contains is really slow for >>>>>>>>> applications like nashorn that has a lot of class loader data. >>>>>>>>> >>>>>>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also >>>>>>>>> ran >>>>>>>>> >>>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >>>>>>>> Thanks for taking care of this fix. >>>>>>>> I think the idea of deferring CLDG::purge to the next safepoint >>>>>>>> cleanup is sound. >>>>>>> Oh great! I'm glad you agree. >>>>>>> >>>>>>>> I like the removal of CLDG::contains since it makes more sense >>>>>>>> to ask >>>>>>>> Metaspace straight up. >>>>>>>> >>>>>>>> I have a small question about the test you added: >>>>>>>> >>>>>>>> 3798 static void test_contains() { >>>>>>>> 3799 // Test that all the methods in the CLDG are contained >>>>>>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >>>>>>>> 3801 } >>>>>>>> >>>>>>>> Have we always allocated methods at this point in the VM startup? >>>>>>> I thought this ExecuteVMInternalTests (or whatever the option is >>>>>>> called) is at shutdown? I'll check. >>>>>>> >>>>>>>> Should we also test CLDG::classes_do to make sure it works for the >>>>>>>> compressed >>>>>>>> class space? >>>>>>> I had that same thought this morning. Yes, I'll add it. >>>>>>> >>>>>>> Coleen >>>>>>> >>>>>>>> /Mikael >>>>>>>> >>>>>>>>> Marcus L. tested it with nashorn tests. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen > From coleen.phillimore at oracle.com Thu May 15 21:55:03 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 15 May 2014 17:55:03 -0400 Subject: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> References: <53716503.7060503@oracle.com> <2511461.tAbzLMKlmA@vboxbuntu> <5372691F.2030107@oracle.com> <3786299.dyoyOuaSQ9@vboxbuntu> <53727E78.7030307@oracle.com> <02a98768-4b02-4bc3-b9f5-6c9491b3be1c@default> Message-ID: <537537B7.7010307@oracle.com> Thank you Markus, Coleen On 5/14/14, 5:10 AM, Markus Gr?nlund wrote: > Hi Coleen, > > I have taken a look as well as running the nashorn crypto tests with the patches - these changes looks good. > > Thanks for addressing this. > > /Markus > > > -----Original Message----- > From: Coleen Phillimore > Sent: den 13 maj 2014 22:20 > To: Mikael Gerdin > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking > > > On 5/13/14, 3:38 PM, Mikael Gerdin wrote: >> On Tuesday 13 May 2014 14.49.03 Coleen Phillimore wrote: >>> On 5/13/14, 2:41 PM, Mikael Gerdin wrote: >>>> On Tuesday 13 May 2014 14.16.30 Coleen Phillimore wrote: >>>>> Hi again, >>>>> >>>>> I have moved the metaspace vm internal test to the end of >>>>> execution, and also test if klasses are in metaspace, which works >>>>> fine. Thank you for the suggestion! Here is a new webrev: >>>> I'm not too happy about moving the test to the before_exit handler. >>> Why not? >> I guess that most of the other unit tests are (and in my opinion >> should be) as independent as possible of JVM state such as loaded classes. >> I was fine with the test before since it's a reasonably small amount >> of code to get a test and it adds some test coverage. >> When the change involves adding additional hooks for unit tests in the >> exit path I'm just slightly more bothered by it. >> >> If we want a proper unit test for Metaspace::contains which tests all >> sorts of alllocations/classloaders and such then it needs to be a bit >> more explicit with setting up a known test environment. >> >>>> Also, how can using the run_unit_test-macro (defined in jni.cpp) >>>> even work in java.cpp? >>> It's moved to globalDefinitions.hpp. >> I see. >> >>>> About methods and classes being available I think we may have >>>> preloaded some classes in Threads::create_vm, right? >>> Yes, we have loaded classes before the initial ExecuteVMInternalTests >>> but they are only in the boot class loader, which is only in one >>> metaspace. So it's not much of a test. At least at the end, we will >>> have classes in the application class loader that we are testing >>> whether they are contained in metaspace. >> Ok. >> >>> I am not motivated to add the test for just for the sake of having a >>> test. This change fixes the nashorn performance problem. Maybe that's >>> test enough. >>> >>> before my change: [crypto] 1273 ops/minute (502-1511), warmup=141 >>> after my change: [crypto] 8548 ops/minute (5861-9017), warmup=960 >> Well, I guess the test isn't really for the correctness of the >> ::contains function anyway? And we don't really have a good way to do >> a measured test to make sure that ::contains isn't "too slow". > We actually already test that classes are Metaspace::contains() in the system dictionary verification at the end of the debug build (Dictionary::verify => instanceKlass::verify => Metaspace::contains) > > I think this test adds nothing now and given your concerns, I'm going to revert it. > > Thanks, > Coleen > >> /Mikael >> >>> Coleen >>> >>>> /Mikael >>>> >>>>> http://cr.openjdk.java.net/~coleenp/8038212_2/ >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 5/13/14, 9:51 AM, Coleen Phillimore wrote: >>>>>> Hi Mikael, >>>>>> >>>>>> On 5/13/14, 6:33 AM, Mikael Gerdin wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> On Monday 12 May 2014 20.19.15 Coleen Phillimore wrote: >>>>>>>> Summary: Only prune metaspace virtual spaces at safepoint so >>>>>>>> walking them is safe outside a safepoint. >>>>>>>> >>>>>>>> Walking class loader data graph for contains is really slow for >>>>>>>> applications like nashorn that has a lot of class loader data. >>>>>>>> >>>>>>>> Tested with nsk.quick.testlist, jtreg tests, and jck tests. Also >>>>>>>> ran >>>>>>>> >>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8038212/ >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >>>>>>> Thanks for taking care of this fix. >>>>>>> I think the idea of deferring CLDG::purge to the next safepoint >>>>>>> cleanup is sound. >>>>>> Oh great! I'm glad you agree. >>>>>> >>>>>>> I like the removal of CLDG::contains since it makes more sense to ask >>>>>>> Metaspace straight up. >>>>>>> >>>>>>> I have a small question about the test you added: >>>>>>> >>>>>>> 3798 static void test_contains() { >>>>>>> 3799 // Test that all the methods in the CLDG are contained >>>>>>> 3800 ClassLoaderDataGraph::methods_do(assert_contains); >>>>>>> 3801 } >>>>>>> >>>>>>> Have we always allocated methods at this point in the VM startup? >>>>>> I thought this ExecuteVMInternalTests (or whatever the option is >>>>>> called) is at shutdown? I'll check. >>>>>> >>>>>>> Should we also test CLDG::classes_do to make sure it works for the >>>>>>> compressed >>>>>>> class space? >>>>>> I had that same thought this morning. Yes, I'll add it. >>>>>> >>>>>> Coleen >>>>>> >>>>>>> /Mikael >>>>>>> >>>>>>>> Marcus L. tested it with nashorn tests. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen From david.holmes at oracle.com Fri May 16 06:04:30 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 May 2014 16:04:30 +1000 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <53750694.3080702@univ-mlv.fr> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> <53750694.3080702@univ-mlv.fr> Message-ID: <5375AA6E.6030402@oracle.com> On 16/05/2014 4:25 AM, Remi Forax wrote: > > On 05/15/2014 04:26 PM, Paul Sandoz wrote: >> On May 15, 2014, at 1:29 PM, Vladimir Ivanov >> wrote: >> >>>>>> I may be wrong but for me cachedLambdaForm() is used in a fast path. >>>>>> If it's not the case, I agree that a lock is enough. >>>>> I hold the opinion that we only need to synchronize writers and not >>>>> readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). >>>>> >>>>> Current usage pattern is the following: >>>>> MethodType type = ...; >>>>> LambdaForm lform = type.form().cachedLambdaForm(idx); >>>>> if (lform == NULL) { >>>>> // construct new LambaForm >>>>> lform = type.form().setCachedLambdaForm(idx, lform); >>>>> } >>>>> return lform; >>>>> >>>>> Cache is written only once, so it has only 2 states (null and non-null >>>>> value) during it's lifecycle. >>>>> >>>>> The only stale value cachedLambdaForm() can see is null, but then the >>>>> caller tries to initialize the cache and after acquiring the lock in >>>>> setCachedLambdaForm() sees actual value. >>>>> >>>>> So, the worst thing can happen if readers aren't synchronized is >>>>> creation of unnecessary LF (which go dead right away) in rare cases. >>>> Unless the cache is volatile this would be unsafe publication and the >>>> reader may see a non-null reference to the LF but the LF may not be >>>> properly initialized. >>> Good point. The cache is an array element. How does safe publication >>> of array element look like? Is accessing the element through volatile >>> reference to the array enough or additional synchronization actions >>> are needed? >>> >> I thought that since the only fields explicitly set when constructing >> a LambdaForm are final fields it should be safe to publish the form >> reference as an element in the array? >> >> Paul. >> > > that's my analysis too. Sorry didn't realize a LF only had final fields. David > R?mi > From roland.westrelin at oracle.com Fri May 16 06:57:10 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 16 May 2014 08:57:10 +0200 Subject: [8u20] 8041481: JVM crashes with collect_args_for_profiling In-Reply-To: <5374F2CF.407@oracle.com> References: <99063C7A-C6FD-407C-9302-9D0698BB2441@oracle.com> <5374F2CF.407@oracle.com> Message-ID: <26F23341-2B97-46D5-852D-9724C742980D@oracle.com> Thanks, Vladimir. Roland. On May 15, 2014, at 7:01 PM, Vladimir Kozlov wrote: > Good. > > Thanks, > Vladimir > > On 5/15/14 2:39 AM, Roland Westrelin wrote: >> 8u20 backport request. The change was pushed to jdk9 almost 3 weeks ago and nightly testing hasn?t shown any problem. >> >> The change applies cleanly to 8u. >> http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/a35e424a76a4 >> https://bugs.openjdk.java.net/browse/JDK-8041481 >> >> Roland. >> From fredrik.arvidsson at oracle.com Fri May 16 08:47:08 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Fri, 16 May 2014 10:47:08 +0200 Subject: Need comments on JEP-158: Unified JVM Logging Message-ID: <5375D08C.9070206@oracle.com> Hi all Please help me review this updated version of the JEP-158 Unified JVM Logging that was pushed today to the JEP repository. Since the service posting the JEPs to http://openjdk.java.net/jeps/ site seems to be non working at the moment I am linking directly to the HG repository instead. The updated JEP can be found here in raw markdown format: http://hg.openjdk.java.net/jep/jeps/file/b04a394c4df4/jep-158.md Primarily I would like to have feedback on the overall functionality scope, the logging API and the command line argument format. All type of comments are of course welcome, but I would like to stay away from any implementation specifics at this point. I have a POC implementation which is nearly feature complete up and running and it looks good so far. /Fredrik From fredrik.arvidsson at oracle.com Fri May 16 11:20:02 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Fri, 16 May 2014 13:20:02 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> Message-ID: <5375F462.60000@oracle.com> Hi Richard, thanks for the feedback. In the JEP I tried to explain the concept of 'sub' loggers but I did not put in anything about the intent of having them. I think that your requirement easily could be solved using sub loggers, or sub components if you like that wording better. Loggers are ordered in an hierarchical tree where 'gc' might be the root for all logging in the gc 'area'. The 'details', 'cause' and 'safepoint' content types could be translated in to sub loggers to the 'gc' logger. By setting log level for the separate sub loggers to enable logging I cant see that you would not be able to get the filtering you want. By using log levels wisely you will be able to get even more control over what gets output in the logs. The above was a much simplified example, in reality you would probably have a more elaborate tree of gc loggers. There is nothing in the logger three hierarchy approach that implies that they should be big blocks of stuff. Logging topics, tags, or markers were something we considered at the beginning of the design phase but we came to the conclusion that we probably could solve the requirements by using sub loggers instead. Please tell me if you don't think this approach would work, and in that case why. Regards /Fredrik // Adding back hotspot-dev.openjdk.java.net // On 2014-05-16 12:45, Richard Warburton wrote: > Hi, > > Please help me review this updated version of the JEP-158 Unified > JVM Logging that was pushed today to the JEP repository. > > Since the service posting the JEPs to > http://openjdk.java.net/jeps/ site seems to be non working at the > moment I am linking directly to the HG repository instead. > > The updated JEP can be found here in raw markdown format: > http://hg.openjdk.java.net/jep/jeps/file/b04a394c4df4/jep-158.md > > Primarily I would like to have feedback on the overall > functionality scope, the logging API and the command line argument > format. All type of comments are of course welcome, but I would > like to stay away from any implementation specifics at this point. > > > At the moment the GC logs have a way of filtering the content > semantically, based upon the content itself. For example flags like > PrintGCDetails, PrintGCCause, PrintSafepointStatistics offer control > over what I can see based upon what I want to see. I may have > misinterpreted the wording of the JEP and I'd love to be corrected if > this is the case, but it looks like moving to levels is a backwards > step over this type of control. It means that you can only enable or > disable logging in big locksteps and that each level of logging needs > to have a set of things associated with it. > > I'm not saying that an abstract framework should depend upon the > details of the different components but if you had component definable > markers for the type of information in order to offer more fine > grained control I think it would be a most appreciated. That way > something like a GC logger could define its own markers for details, > cause, application stopped time, safepoints etc. An end user who is > trying to diagnose a specific problem could leave their gc logger at > warning, but also then has the option to enable a specific marker for > something like application stopped time which is probably a lower > level than warning. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From tobias.hartmann at oracle.com Fri May 16 11:56:48 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 16 May 2014 13:56:48 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5375AA6E.6030402@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> <53750694.3080702@univ-mlv.fr> <5375AA6E.6030402@oracle.com> Message-ID: <5375FD00.90603@oracle.com> Hi, thanks to everyone for the feedback. Is it sufficient then to use synchronized (lambdaForms) { ... } in setCachedLambdaForm(..) and a normal read in cachedLambdaForm(..)? Thanks, Tobias >>> >>>>>>> I may be wrong but for me cachedLambdaForm() is used in a fast >>>>>>> path. >>>>>>> If it's not the case, I agree that a lock is enough. >>>>>> I hold the opinion that we only need to synchronize writers and not >>>>>> readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). >>>>>> >>>>>> Current usage pattern is the following: >>>>>> MethodType type = ...; >>>>>> LambdaForm lform = type.form().cachedLambdaForm(idx); >>>>>> if (lform == NULL) { >>>>>> // construct new LambaForm >>>>>> lform = type.form().setCachedLambdaForm(idx, lform); >>>>>> } >>>>>> return lform; >>>>>> >>>>>> Cache is written only once, so it has only 2 states (null and >>>>>> non-null >>>>>> value) during it's lifecycle. >>>>>> >>>>>> The only stale value cachedLambdaForm() can see is null, but then >>>>>> the >>>>>> caller tries to initialize the cache and after acquiring the lock in >>>>>> setCachedLambdaForm() sees actual value. >>>>>> >>>>>> So, the worst thing can happen if readers aren't synchronized is >>>>>> creation of unnecessary LF (which go dead right away) in rare cases. >>>>> Unless the cache is volatile this would be unsafe publication and the >>>>> reader may see a non-null reference to the LF but the LF may not be >>>>> properly initialized. >>>> Good point. The cache is an array element. How does safe publication >>>> of array element look like? Is accessing the element through volatile >>>> reference to the array enough or additional synchronization actions >>>> are needed? >>>> >>> I thought that since the only fields explicitly set when constructing >>> a LambdaForm are final fields it should be safe to publish the form >>> reference as an element in the array? >>> >>> Paul. >>> >> >> that's my analysis too. > > Sorry didn't realize a LF only had final fields. > > David > >> R?mi >> From dmitry.samersoff at oracle.com Fri May 16 13:40:46 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 16 May 2014 17:40:46 +0400 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: <5375D08C.9070206@oracle.com> References: <5375D08C.9070206@oracle.com> Message-ID: <5376155E.5010303@oracle.com> Fredrik, 1. I'm not sure we have to provide user-configurable decoration. It add complication to logger because not all decorations available for all components and it creates a burden for customer support because all tools would have to support different log formats. 2. Backend might be a better word than adapter. STDOUT, STDERR and text file is essentially the same backend. 3. I'm not sure we have to care about log rotation by size, time etc - everything is done here during last 20 years, and there are lots of tools that do it. But, for file backend, we have to provide jcmd command to close current log and start new one at runtime. 4. Proposed VM API is too verbose and require lots of typing. Also to add new component you have to touch log code, split or rename components is also problematic. So it might be better to use Log::set_component(Log::gc); Log::warn( format, ... ); Log::error(); Log::trace(); Log::debug() Log::start(); Log::add( format, ... ) Log::commit(Log::level()); 5. I'm not sure that transaction is the best term here. -Dmitry On 2014-05-16 12:47, Fredrik Arvidsson wrote: > Hi all > Please help me review this updated version of the JEP-158 Unified JVM > Logging that was pushed today to the JEP repository. > > Since the service posting the JEPs to http://openjdk.java.net/jeps/ site > seems to be non working at the moment I am linking directly to the HG > repository instead. > > The updated JEP can be found here in raw markdown format: > http://hg.openjdk.java.net/jep/jeps/file/b04a394c4df4/jep-158.md > > Primarily I would like to have feedback on the overall functionality > scope, the logging API and the command line argument format. All type of > comments are of course welcome, but I would like to stay away from any > implementation specifics at this point. > > I have a POC implementation which is nearly feature complete up and > running and it looks good so far. > > /Fredrik > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From forax at univ-mlv.fr Fri May 16 15:05:39 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 16 May 2014 17:05:39 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5375FD00.90603@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> <53750694.3080702@univ-mlv.fr> <5375AA6E.6030402@oracle.com> <5375FD00.90603@oracle.com> Message-ID: <53762943.1030301@univ-mlv.fr> On 05/16/2014 01:56 PM, Tobias Hartmann wrote: > Hi, > > thanks to everyone for the feedback. > > Is it sufficient then to use synchronized (lambdaForms) { ... } in > setCachedLambdaForm(..) and a normal read in cachedLambdaForm(..)? > > Thanks, > Tobias no, it's not, The memory model doesn't guarantee that a normal read will see that another thread has changed the array item, the normal read may always see null. You need a synchronized block around the normal read. R?mi > >>>> >>>>>>>> I may be wrong but for me cachedLambdaForm() is used in a fast >>>>>>>> path. >>>>>>>> If it's not the case, I agree that a lock is enough. >>>>>>> I hold the opinion that we only need to synchronize writers and not >>>>>>> readers (MTF.setCachedLambdaForm() and not MTF.cachedLambdaForm()). >>>>>>> >>>>>>> Current usage pattern is the following: >>>>>>> MethodType type = ...; >>>>>>> LambdaForm lform = type.form().cachedLambdaForm(idx); >>>>>>> if (lform == NULL) { >>>>>>> // construct new LambaForm >>>>>>> lform = type.form().setCachedLambdaForm(idx, lform); >>>>>>> } >>>>>>> return lform; >>>>>>> >>>>>>> Cache is written only once, so it has only 2 states (null and >>>>>>> non-null >>>>>>> value) during it's lifecycle. >>>>>>> >>>>>>> The only stale value cachedLambdaForm() can see is null, but >>>>>>> then the >>>>>>> caller tries to initialize the cache and after acquiring the >>>>>>> lock in >>>>>>> setCachedLambdaForm() sees actual value. >>>>>>> >>>>>>> So, the worst thing can happen if readers aren't synchronized is >>>>>>> creation of unnecessary LF (which go dead right away) in rare >>>>>>> cases. >>>>>> Unless the cache is volatile this would be unsafe publication and >>>>>> the >>>>>> reader may see a non-null reference to the LF but the LF may not be >>>>>> properly initialized. >>>>> Good point. The cache is an array element. How does safe publication >>>>> of array element look like? Is accessing the element through volatile >>>>> reference to the array enough or additional synchronization actions >>>>> are needed? >>>>> >>>> I thought that since the only fields explicitly set when constructing >>>> a LambdaForm are final fields it should be safe to publish the form >>>> reference as an element in the array? >>>> >>>> Paul. >>>> >>> >>> that's my analysis too. >> >> Sorry didn't realize a LF only had final fields. >> >> David >> >>> R?mi >>> > From paul.sandoz at oracle.com Fri May 16 15:22:04 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 16 May 2014 17:22:04 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <53762943.1030301@univ-mlv.fr> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> <53750694.3080702@univ-mlv.fr> <5375AA6E.6030402@oracle.com> <5375FD00.90603@oracle.com> <53762943.1030301@univ-mlv.fr> Message-ID: On May 16, 2014, at 5:05 PM, Remi Forax wrote: > > On 05/16/2014 01:56 PM, Tobias Hartmann wrote: >> Hi, >> >> thanks to everyone for the feedback. >> >> Is it sufficient then to use synchronized (lambdaForms) { ... } in setCachedLambdaForm(..) and a normal read in cachedLambdaForm(..)? >> >> Thanks, >> Tobias > > no, it's not, > The memory model doesn't guarantee that a normal read will see that another thread has changed the array item, > the normal read may always see null. > > You need a synchronized block around the normal read. > I don't think that is required since the pattern is: form = cachedLambdaForm(which); if (form != null) return form ... form = /new form/ ... form = setCachedLambdaForm(which, form); return form; and the setCachedLambdaForm will, within it's the synchronized block, first perform a read to check if there is already a non-default value present. Paul. From cnewland at chrisnewland.com Sat May 17 07:47:12 2014 From: cnewland at chrisnewland.com (Chris Newland) Date: Sat, 17 May 2014 08:47:12 +0100 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> Message-ID: Hi Fredrik, The discussion I had with David Holmes and John Rose on hotspot-dev back in February might be relevant to this JEP: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-February/012718.html I'm the author of a JITWatch[1], a JIT compiler log analysis tool which relies on -XX:+TraceClassLoading -XX:+LogCompilation -XX:+PrintAssembly all being present in hotspot.log Occasionally the lack of thread safety in the VM logging presents me with jumbled output: [Entry Point] [Constants] # {method} 'ind [Loaded sun.nio.cs.ThreadLocalCoders from C:\Program Files\Java\jre7\lib\rt.jar] exOf' '(II)I' in 'java/lang/String' # this: ecx = 'java/lang/String' # parm0: edx = int # parm1: [sp+0x20] = int (sp of caller) 0x009e07e0: nop The JEP goal of no interleaving would fix this problem but if the locking adds too much cost/complexity then John's suggestion of tagging partial lines would be a great help to log tool writers. Otherwise my only comment is that the goals look like a massive improvement to current VM logging but for post-execution log tool writers it is also useful for end users to be able to easily locate and load a single log file. Kind regards, Chris [1] https://github.com/AdoptOpenJDK/jitwatch >>> ---------- Forwarded message ---------- >>> From: Fredrik Arvidsson >>> Date: 16 May 2014 09:47 >>> Subject: Need comments on JEP-158: Unified JVM Logging >>> To: "serviceability-dev at openjdk.java.net >>> serviceability-dev at openjdk.java.net" >>> , >>> hotspot-dev at openjdk.java.net >>> >>> >>> Hi all >>> Please help me review this updated version of the JEP-158 Unified JVM >>> Logging that was pushed today to the JEP repository. >>> >>> Since the service posting the JEPs to http://openjdk.java.net/jeps/ >>> site >>> seems to be non working at the moment I am linking directly to the HG >>> repository instead. >>> >>> The updated JEP can be found here in raw markdown format: >>> http://hg.openjdk.java.net/jep/jeps/file/b04a394c4df4/jep-158.md >>> >>> Primarily I would like to have feedback on the overall functionality >>> scope, the logging API and the command line argument format. All type >>> of >>> comments are of course welcome, but I would like to stay away from any >>> implementation specifics at this point. >>> >>> I have a POC implementation which is nearly feature complete up and >>> running and it looks good so far. >>> >>> /Fredrik >>> >>> From richard.warburton at gmail.com Sun May 18 09:47:32 2014 From: richard.warburton at gmail.com (Richard Warburton) Date: Sun, 18 May 2014 10:47:32 +0100 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: <5375F462.60000@oracle.com> References: <5375D08C.9070206@oracle.com> <5375F462.60000@oracle.com> Message-ID: Hi Fredrik, In the JEP I tried to explain the concept of 'sub' loggers but I did not > put in anything about the intent of having them. I think that your > requirement easily could be solved using sub loggers, or sub components if > you like that wording better. > > Loggers are ordered in an hierarchical tree where 'gc' might be the root > for all logging in the gc 'area'. The 'details', 'cause' and 'safepoint' > content types could be translated in to sub loggers to the 'gc' logger. By > setting log level for the separate sub loggers to enable logging I cant see > that you would not be able to get the filtering you want. By using log > levels wisely you will be able to get even more control over what gets > output in the logs. The above was a much simplified example, in reality you > would probably have a more elaborate tree of gc loggers. There is nothing > in the logger three hierarchy approach that implies that they should be big > blocks of stuff. > > Logging topics, tags, or markers were something we considered at the > beginning of the design phase but we came to the conclusion that we > probably could solve the requirements by using sub loggers instead. > > Please tell me if you don't think this approach would work, and in that > case why. > Thanks for clarifying this point. I think this explanation addresses how that particular use case is met. I suspect that it might be worth adding a bit more explanation around this topic to the JEP itself, because I don't think I'm the only person with this concern. Also a heads up that there's discussion happening related to this topic off-list at: https://groups.google.com/a/jclarity.com/forum/#!topic/friends/NA0EyOJk6bs regards, Richard Warburton http://insightfullogic.com @RichardWarburto From fredrik.arvidsson at oracle.com Sun May 18 10:04:34 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Sun, 18 May 2014 12:04:34 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> <5375F462.60000@oracle.com> Message-ID: <537885B2.50102@oracle.com> Hi Richard I will add more text in the JEP describing the intent of the hierarchical logger design. As you said, it is not that clear in the current one. Thanks. Regarding out off band discussions. I have noted that and I will try to keep myself updated to that. It is not optimal to have discussions in many places and I will not be able to screen all the internets for this information. But this is what we have to deal with I am afraid. Cheers /Fredrik On 2014-05-18 11:47, Richard Warburton wrote: > Hi Fredrik, > > In the JEP I tried to explain the concept of 'sub' loggers but I > did not put in anything about the intent of having them. I think > that your requirement easily could be solved using sub loggers, or > sub components if you like that wording better. > > Loggers are ordered in an hierarchical tree where 'gc' might be > the root for all logging in the gc 'area'. The 'details', 'cause' > and 'safepoint' content types could be translated in to sub > loggers to the 'gc' logger. By setting log level for the separate > sub loggers to enable logging I cant see that you would not be > able to get the filtering you want. By using log levels wisely you > will be able to get even more control over what gets output in the > logs. The above was a much simplified example, in reality you > would probably have a more elaborate tree of gc loggers. There is > nothing in the logger three hierarchy approach that implies that > they should be big blocks of stuff. > > Logging topics, tags, or markers were something we considered at > the beginning of the design phase but we came to the conclusion > that we probably could solve the requirements by using sub loggers > instead. > > Please tell me if you don't think this approach would work, and in > that case why. > > > Thanks for clarifying this point. I think this explanation addresses > how that particular use case is met. I suspect that it might be worth > adding a bit more explanation around this topic to the JEP itself, > because I don't think I'm the only person with this concern. > > Also a heads up that there's discussion happening related to this > topic off-list at: > https://groups.google.com/a/jclarity.com/forum/#!topic/friends/NA0EyOJk6bs > > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From kirk at kodewerk.com Sun May 18 10:08:14 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Sun, 18 May 2014 12:08:14 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> Message-ID: <27AF2F28-F69E-4981-A81D-55481E6CCD8A@kodewerk.com> Hi Chris, I too am bothered by the jumbled lines. However I don?t think that we need strict ordering. However I don?t believe it?s up to the logging framework to enforce order. If order is needed it should be up to the producer or the consumer of messages to deal with the issue of order. However I do believe these requirements should be struck from the document Logging is performed at different levels: error, warning, info, debug, trace Logging messages are in human-readable plain text Messages can be ?decorated? with level, component, thread id, timestamp Enable/disable individual log messages by message id (for example by using__FILE__ / __LINE__) IMHO, that is proposal specifies a taxonomy that is hierarchal and devoid of context in the domains in which it will be working is very limiting and over reaching. As I have stated in past discussion, feel that the mention of levels should be dropped. It it my view that at it?s core, logging is a messaging problem and there are many lessons that we should be taking from messaging that seem to be missing from this proposal. Even lessons or ideas learned from social media messaging (Twitter messages???) is missing from the proposal. The goals that I pointed to all specify all work against the flexibility of allowing non-hieracrchal taxonomies. In other words, allow users of the framework to define categories that the makers of the framework cannot not possibly conjure up and then allow those categories to be collected or acted upon independent of other categories. This scheme does not nor should not preclude one from using a hierarchal logging system. However it does not limit one to a hierarchical scheme when that scheme does not make sense. If we take GC logging into consideration. The current system of flags allow me to understand what it is I?m getting as I turn on each flag. I can turn on combinations of flags to get different bits of information as the need arises. Unfortunately the newer code is translating these flags to Fine, Finer, and Finest which means some developer is not deciding what categories of logging goes into each of these limited number of categories which may.. or may not be an appropriate mix. My experience with logging in the field is that everything sooner or later degenerates into the lowest common denominator? which is why I get called in as the logging becomes the primary bottleneck in the application. To let you know how much of a problem this it now represents about 30-40% of all rescue tuning that I?m involved in. Now I know you guys are all very clever and you don?t think that you?d be susceptible to making the same type of mistake? however I should caution that a number of the teams that locked themselves into the logging/bottleneck problem were also very clever. This is just something that happens organically? over time without anyone really taking notice.. much like the frog in the pot or stone soup story. I completely agree with this statement; "It is outside the scope of this JEP to add the actual logging calls from all JVM components. This JEP will only provide the infrastructure to do the logging.? I?d add to this that the implementation should not over reach in that defining categories is not within the scope of the framework Finally, the statement "It is also outside the scope of the JEP to enforce a logging format, apart from the format of the decorations and the use of human-readable plain text? is also objectionable as it precludes logging binary formats. This IMHO is a huge over-reach based on the dogma that logs should be in human readable form. The last time I checked ASCII was not human readable in that I need a program to view it in a meaningful format on a screen. Please, lets not let the problems found in the current set of logging frameworks that Java developers have had been forced to use leak into the JVM. Lets reconsider this JEP in it?s current form before moving to an implementation. Kind regards, Kirk Pepperdine On May 17, 2014, at 9:47 AM, Chris Newland wrote: > Hi Fredrik, > > The discussion I had with David Holmes and John Rose on hotspot-dev back > in February might be relevant to this JEP: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-February/012718.html > > I'm the author of a JITWatch[1], a JIT compiler log analysis tool which > relies on -XX:+TraceClassLoading -XX:+LogCompilation -XX:+PrintAssembly > all being present in hotspot.log > > Occasionally the lack of thread safety in the VM logging presents me with > jumbled output: > > [Entry Point] > [Constants] > # {method} 'ind > > [Loaded sun.nio.cs.ThreadLocalCoders from C:\Program > Files\Java\jre7\lib\rt.jar] > > exOf' '(II)I' in 'java/lang/String' > # this: ecx = 'java/lang/String' > # parm0: edx = int > # parm1: [sp+0x20] = int (sp of caller) > 0x009e07e0: nop > > The JEP goal of no interleaving would fix this problem but if the locking > adds too much cost/complexity then John's suggestion of tagging partial > lines would be a great help to log tool writers. > > Otherwise my only comment is that the goals look like a massive > improvement to current VM logging but for post-execution log tool writers > it is also useful for end users to be able to easily locate and load a > single log file. > > Kind regards, > > Chris > > [1] https://github.com/AdoptOpenJDK/jitwatch > >>>> ---------- Forwarded message ---------- >>>> From: Fredrik Arvidsson >>>> Date: 16 May 2014 09:47 >>>> Subject: Need comments on JEP-158: Unified JVM Logging >>>> To: "serviceability-dev at openjdk.java.net >>>> serviceability-dev at openjdk.java.net" >>>> , >>>> hotspot-dev at openjdk.java.net >>>> >>>> >>>> Hi all >>>> Please help me review this updated version of the JEP-158 Unified JVM >>>> Logging that was pushed today to the JEP repository. >>>> >>>> Since the service posting the JEPs to http://openjdk.java.net/jeps/ >>>> site >>>> seems to be non working at the moment I am linking directly to the HG >>>> repository instead. >>>> >>>> The updated JEP can be found here in raw markdown format: >>>> http://hg.openjdk.java.net/jep/jeps/file/b04a394c4df4/jep-158.md >>>> >>>> Primarily I would like to have feedback on the overall functionality >>>> scope, the logging API and the command line argument format. All type >>>> of >>>> comments are of course welcome, but I would like to stay away from any >>>> implementation specifics at this point. >>>> >>>> I have a POC implementation which is nearly feature complete up and >>>> running and it looks good so far. >>>> >>>> /Fredrik >>>> >>>> > From kirk at kodewerk.com Sun May 18 10:14:47 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Sun, 18 May 2014 12:14:47 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> <5375F462.60000@oracle.com> Message-ID: Hi Fredrik, I do need more information but I think a flat structure would provide less overhead rather than passing messages through a network of ?sub? loggers. Peter Lawrey wrote JChronicle which at it?s core is a messaging system but it?s also a wickedly fast logger. That said, there isn?t a formalized logging interface to it though it?s something being discussed/thought about as we speak. https://github.com/peter-lawrey/Java-Chronicle. I?m not suggesting you take this implementation but there might be things to learn from it as this spec is being strengthened. Kind regards, Kirk Pepperdine On May 18, 2014, at 11:47 AM, Richard Warburton wrote: > Hi Fredrik, > > In the JEP I tried to explain the concept of 'sub' loggers but I did not >> put in anything about the intent of having them. I think that your >> requirement easily could be solved using sub loggers, or sub components if >> you like that wording better. >> >> Loggers are ordered in an hierarchical tree where 'gc' might be the root >> for all logging in the gc 'area'. The 'details', 'cause' and 'safepoint' >> content types could be translated in to sub loggers to the 'gc' logger. By >> setting log level for the separate sub loggers to enable logging I cant see >> that you would not be able to get the filtering you want. By using log >> levels wisely you will be able to get even more control over what gets >> output in the logs. The above was a much simplified example, in reality you >> would probably have a more elaborate tree of gc loggers. There is nothing >> in the logger three hierarchy approach that implies that they should be big >> blocks of stuff. >> >> Logging topics, tags, or markers were something we considered at the >> beginning of the design phase but we came to the conclusion that we >> probably could solve the requirements by using sub loggers instead. >> >> Please tell me if you don't think this approach would work, and in that >> case why. >> > > Thanks for clarifying this point. I think this explanation addresses how > that particular use case is met. I suspect that it might be worth adding a > bit more explanation around this topic to the JEP itself, because I don't > think I'm the only person with this concern. > > Also a heads up that there's discussion happening related to this topic > off-list at: > https://groups.google.com/a/jclarity.com/forum/#!topic/friends/NA0EyOJk6bs > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto From fredrik.arvidsson at oracle.com Sun May 18 11:11:49 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Sun, 18 May 2014 13:11:49 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> <5375F462.60000@oracle.com> Message-ID: <53789575.4050608@oracle.com> Hi all On 2014-05-18 12:14, Kirk Pepperdine wrote: > Hi Fredrik, > > I do need more information but I think a flat structure would provide less overhead rather than passing messages through a network of ?sub? loggers. Kirk, I can understand that you are concerned that the hierarchical structure would create unnecessary overhead, but it does not have to. With a good implementation design messages does not have to be passed through any other logger than the one it is logged to. I wont go in to any details here, but in my current poc implementation I think the code is quite efficient, even if I don?t have spent that much time on optimization yet. I think that having a good design that answers well to the requirements is better than doing premature sub optimization by crippling the design. Functionality first, optimization later, that's how I like to do it. I will do my best to spend some time looking at Peters code, I am sure there are good things in there to consider. Cheers /Fredrik > Peter Lawrey wrote JChronicle which at it?s core is a messaging system but it?s also a wickedly fast logger. That said, there isn?t a formalized logging interface to it though it?s something being discussed/thought about as we speak. > https://github.com/peter-lawrey/Java-Chronicle. > > I?m not suggesting you take this implementation but there might be things to learn from it as this spec is being strengthened. > > Kind regards, > Kirk Pepperdine > > On May 18, 2014, at 11:47 AM, Richard Warburton wrote: > >> Hi Fredrik, >> >> In the JEP I tried to explain the concept of 'sub' loggers but I did not >>> put in anything about the intent of having them. I think that your >>> requirement easily could be solved using sub loggers, or sub components if >>> you like that wording better. >>> >>> Loggers are ordered in an hierarchical tree where 'gc' might be the root >>> for all logging in the gc 'area'. The 'details', 'cause' and 'safepoint' >>> content types could be translated in to sub loggers to the 'gc' logger. By >>> setting log level for the separate sub loggers to enable logging I cant see >>> that you would not be able to get the filtering you want. By using log >>> levels wisely you will be able to get even more control over what gets >>> output in the logs. The above was a much simplified example, in reality you >>> would probably have a more elaborate tree of gc loggers. There is nothing >>> in the logger three hierarchy approach that implies that they should be big >>> blocks of stuff. >>> >>> Logging topics, tags, or markers were something we considered at the >>> beginning of the design phase but we came to the conclusion that we >>> probably could solve the requirements by using sub loggers instead. >>> >>> Please tell me if you don't think this approach would work, and in that >>> case why. >>> >> Thanks for clarifying this point. I think this explanation addresses how >> that particular use case is met. I suspect that it might be worth adding a >> bit more explanation around this topic to the JEP itself, because I don't >> think I'm the only person with this concern. >> >> Also a heads up that there's discussion happening related to this topic >> off-list at: >> https://groups.google.com/a/jclarity.com/forum/#!topic/friends/NA0EyOJk6bs >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto From martijnverburg at gmail.com Sun May 18 11:26:40 2014 From: martijnverburg at gmail.com (Martijn Verburg) Date: Sun, 18 May 2014 12:26:40 +0100 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: <537885B2.50102@oracle.com> References: <5375D08C.9070206@oracle.com> <5375F462.60000@oracle.com> <537885B2.50102@oracle.com> Message-ID: Hi Fredrik, I've asked all of the discussions on our list to be fed back here or directly to you. Thanks for opening the discussion, we really appreciate it! Cheers, Martijn On Sunday, 18 May 2014, Fredrik Arvidsson wrote: > Hi Richard > > I will add more text in the JEP describing the intent of the hierarchical > logger design. As you said, it is not that clear in the current one. Thanks. > > Regarding out off band discussions. I have noted that and I will try to > keep myself updated to that. It is not optimal to have discussions in many > places and I will not be able to screen all the internets for this > information. But this is what we have to deal with I am afraid. > > Cheers > /Fredrik > > On 2014-05-18 11:47, Richard Warburton wrote: > >> Hi Fredrik, >> >> In the JEP I tried to explain the concept of 'sub' loggers but I >> did not put in anything about the intent of having them. I think >> that your requirement easily could be solved using sub loggers, or >> sub components if you like that wording better. >> >> Loggers are ordered in an hierarchical tree where 'gc' might be >> the root for all logging in the gc 'area'. The 'details', 'cause' >> and 'safepoint' content types could be translated in to sub >> loggers to the 'gc' logger. By setting log level for the separate >> sub loggers to enable logging I cant see that you would not be >> able to get the filtering you want. By using log levels wisely you >> will be able to get even more control over what gets output in the >> logs. The above was a much simplified example, in reality you >> would probably have a more elaborate tree of gc loggers. There is >> nothing in the logger three hierarchy approach that implies that >> they should be big blocks of stuff. >> >> Logging topics, tags, or markers were something we considered at >> the beginning of the design phase but we came to the conclusion >> that we probably could solve the requirements by using sub loggers >> instead. >> >> Please tell me if you don't think this approach would work, and in >> that case why. >> >> >> Thanks for clarifying this point. I think this explanation addresses how >> that particular use case is met. I suspect that it might be worth adding a >> bit more explanation around this topic to the JEP itself, because I don't >> think I'm the only person with this concern. >> >> Also a heads up that there's discussion happening related to this topic >> off-list at: https://groups.google.com/a/jclarity.com/forum/#!topic/ >> friends/NA0EyOJk6bs > jclarity.com/forum/#%21topic/friends/NA0EyOJk6bs> >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto >> > > -- Cheers, Martijn From fredrik.arvidsson at oracle.com Sun May 18 15:07:17 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Sun, 18 May 2014 17:07:17 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: <5376155E.5010303@oracle.com> References: <5375D08C.9070206@oracle.com> <5376155E.5010303@oracle.com> Message-ID: <5378CCA5.3040303@oracle.com> Hi Dmitry, thanks for the feedback. I will comment in-line below. On 2014-05-16 15:40, Dmitry Samersoff wrote: > Fredrik, > > 1. I'm not sure we have to provide user-configurable decoration. > > It add complication to logger because not all decorations > available for all components and it creates a burden > for customer support because all tools would have > to support different log formats. Agreed. Any extra information information could easily be added to the actual log message if needed. This is of course only feasible if the output is in text format. Examples of user-provided decorations that people asked for is gc number, and compiler number. > 2. Backend might be a better word than adapter. > > STDOUT, STDERR and text file is essentially the same backend. You're right, these three adapters are all file stream adapters. Its only the stream written to that differs. from a configuration point of view they are three different targets though. Adapter is used in some other logging frameworks, I choose that name to have people recognise it. > > 3. I'm not sure we have to care about log rotation by size, time etc - > everything is done here during last 20 years, and there are lots of > tools that do it. I think it is very nice to have this simple functionality in the logging framework, it removes one external tool dependency when setting up logging. But as you say, it is not strictly necessary. > But, for file backend, we have to provide jcmd command to close > current log and start new one at runtime. Good idea. I will add this to the JEP somewhere. > > 4. Proposed VM API is too verbose and require lots of typing. > Also to add new component you have to touch log code, split > or rename components is also problematic. > > So it might be better to use > > Log::set_component(Log::gc); > Log::warn( format, ... ); > Log::error(); > Log::trace(); > Log::debug() > > Log::start(); > Log::add( format, ... ) > Log::commit(Log::level()); The VM API was just an example in the JEP. There are basically no loggers defined in the raw implementation. It is up to the developers to add loggers/categories when there is a need for it. if you want to you can name them a, b and c :) I cant really see that verbosity is an issue here. I really hope most of you are typing fast enough to handle a few more characters. I think the readability is more important than the number of character in the name. there are enough abbreviated names in the JVM code as it is now. The number of lines that are logging are few in comparison with actual code lines anyway I think. Split and rename is only an issue if you aren?t using a modern editor/ide to re-factor stuff (hope I didn?t step on any toes here ;) ). In my poc implementation you only have to add one line of 'code' to have a new logger. I used Eclipse when I tried it and it picks up all info to provide good contextual help on the spot. I would expect Visual Studio and other tools to work equally well. Very easy to use and simple to manage. > 5. I'm not sure that transaction is the best term here. > > -Dmitry > Maybe not, Ill try to find a better word for the buffered logging. Cheers /Fredrik From fredrik.arvidsson at oracle.com Sun May 18 15:15:32 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Sun, 18 May 2014 17:15:32 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> <5375F462.60000@oracle.com> <537885B2.50102@oracle.com> Message-ID: <5378CE94.6060502@oracle.com> Hi Martijn Thank you for doing that. I'm afraid it is going to be hard enough anyway to keep myself updated on all feedback given :) For me it is a matter of course that these kind of discussions should be held in the open. We are all stake holders in this. As long as the discussion does not turn in to 'bike-shedding' or someone is deliberately trying to pervert the intention of the JEP I will try to keep it in the open as much as possible. Cheers /Fredrik On 2014-05-18 13:26, Martijn Verburg wrote: > Hi Fredrik, > > I've asked all of the discussions on our list to be fed back here or > directly to you. Thanks for opening the discussion, we really > appreciate it! > > Cheers, > Martijn > > On Sunday, 18 May 2014, Fredrik Arvidsson > > > wrote: > > Hi Richard > > I will add more text in the JEP describing the intent of the > hierarchical logger design. As you said, it is not that clear in > the current one. Thanks. > > Regarding out off band discussions. I have noted that and I will > try to keep myself updated to that. It is not optimal to have > discussions in many places and I will not be able to screen all > the internets for this information. But this is what we have to > deal with I am afraid. > > Cheers > /Fredrik > > On 2014-05-18 11:47, Richard Warburton wrote: > > Hi Fredrik, > > In the JEP I tried to explain the concept of 'sub' loggers > but I > did not put in anything about the intent of having them. I > think > that your requirement easily could be solved using sub > loggers, or > sub components if you like that wording better. > > Loggers are ordered in an hierarchical tree where 'gc' > might be > the root for all logging in the gc 'area'. The 'details', > 'cause' > and 'safepoint' content types could be translated in to sub > loggers to the 'gc' logger. By setting log level for the > separate > sub loggers to enable logging I cant see that you would not be > able to get the filtering you want. By using log levels > wisely you > will be able to get even more control over what gets > output in the > logs. The above was a much simplified example, in reality you > would probably have a more elaborate tree of gc loggers. > There is > nothing in the logger three hierarchy approach that > implies that > they should be big blocks of stuff. > > Logging topics, tags, or markers were something we > considered at > the beginning of the design phase but we came to the > conclusion > that we probably could solve the requirements by using sub > loggers > instead. > > Please tell me if you don't think this approach would > work, and in > that case why. > > > Thanks for clarifying this point. I think this explanation > addresses how that particular use case is met. I suspect that > it might be worth adding a bit more explanation around this > topic to the JEP itself, because I don't think I'm the only > person with this concern. > > Also a heads up that there's discussion happening related to > this topic off-list at: > https://groups.google.com/a/jclarity.com/forum/#!topic/friends/NA0EyOJk6bs > > > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > > > > > -- > Cheers, > Martijn From fredrik.arvidsson at oracle.com Sun May 18 15:29:50 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Sun, 18 May 2014 17:29:50 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> Message-ID: <5378D1EE.3020309@oracle.com> Hi Chris You will find my comments in-line below. On 2014-05-17 09:47, Chris Newland wrote: > Hi Fredrik, > > The discussion I had with David Holmes and John Rose on hotspot-dev back > in February might be relevant to this JEP: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-February/012718.html I will use some time to read this discussion. Thanks. > I'm the author of a JITWatch[1], a JIT compiler log analysis tool which > relies on -XX:+TraceClassLoading -XX:+LogCompilation -XX:+PrintAssembly > all being present in hotspot.log > > Occasionally the lack of thread safety in the VM logging presents me with > jumbled output: > > [Entry Point] > [Constants] > # {method} 'ind > > [Loaded sun.nio.cs.ThreadLocalCoders from C:\Program > Files\Java\jre7\lib\rt.jar] > > exOf' '(II)I' in 'java/lang/String' > # this: ecx = 'java/lang/String' > # parm0: edx = int > # parm1: [sp+0x20] = int (sp of caller) > 0x009e07e0: nop > > The JEP goal of no interleaving would fix this problem but if the locking > adds too much cost/complexity then John's suggestion of tagging partial > lines would be a great help to log tool writers. There could be several reasons for the current logs to be interleaved I think. One reason can be that the lowest level log writing primitives aren't thread safe themselves. A second reason can be that the current log framework doesn?t provide any form of log transactions making it easy for developer to implement the logging in a way that guarantees non partial lines in the log. JEP-158 tries to address both of these reasons. Thread safety issues can be addressed using the leanest locking/critical section technique we can have. This might have different solutions depending on platform and log adapter in question. The 'broken up log code' is addressed with our log transaction which enables the developer to split the actual line of logging in to several calls to the log API. The log line is then committed to the underlying framework to be written all at once. > > Otherwise my only comment is that the goals look like a massive > improvement to current VM logging but for post-execution log tool writers > it is also useful for end users to be able to easily locate and load a > single log file. This really captures the intent of JEP-158 I think. We want to have a common framework for logging so that all output can be formatted the same well defined way, and we want to be able to have a single entry of configuration letting users configuring all logging in a consolidated and clear way. > Kind regards, > > Chris > > [1] https://github.com/AdoptOpenJDK/jitwatch > Cheers /Fredrik From goetz.lindenmaier at sap.com Mon May 19 10:58:05 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 19 May 2014 10:58:05 +0000 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> Hi Christian, thanks for the review! Although prefetch is not used that often, I think it's good that all headers in the os_cpu directories are handled similarly now, i.e., they are only included in the respective shared file. (For some it's the file in the /cpu/ dir, like copy_linux_x86.inline.hpp.) Could you sponsor the change? Do I need a second reviewer? Best regards, Goetz. -----Original Message----- From: Christian Thalinger [mailto:christian.thalinger at oracle.com] Sent: Donnerstag, 15. Mai 2014 17:14 To: Lindenmaier, Goetz Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp Looks good to me. I thought that prefetch is used more often but apparently not. On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: > Hi, > > This change introduces an umbrella header for prefetch__.inline.hpp > files: runtime/prefetch.inline.hpp as proposed in > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html > This follows the scheme applied to atomic.inline.hpp. > > Further this change adds includes of prefetch.inline.hpp in all .cpp > and .inline.hpp files where a method of Prefetch declared 'inline' is > called. > > Finally it moves macros calling inline methods of Prefetch from > space.hpp to the corresponding .inline.hpp file and adds the necessary > includes in files using the moved macros. > > Please review and test this change. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ > > Thanks and best regards, > Goetz. From stefan.karlsson at oracle.com Mon May 19 11:20:57 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 19 May 2014 13:20:57 +0200 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> Message-ID: <5379E919.7010803@oracle.com> Hi Goetz, On 2014-05-19 12:58, Lindenmaier, Goetz wrote: > Hi Christian, > > thanks for the review! > > Although prefetch is not used that often, I think it's good > that all headers in the os_cpu directories are handled > similarly now, i.e., they are only included in the respective > shared file. (For some it's the file in the /cpu/ dir, like > copy_linux_x86.inline.hpp.) > > Could you sponsor the change? Do I need a second reviewer? Thanks for cleaning this up. Some minor comments: http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html You shouldn't have to include prefetch.inline.hpp from this file. http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html heapRegion.cpp needs to include space.inline.hpp since it uses the SCAN_AND_FORWARD macro. http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html I don't think its necessary to add the prefetch.inline.hpp to precompiled.hpp. Otherwise, this looks good to me. thanks, StefanK > > Best regards, > Goetz. > > > > -----Original Message----- > From: Christian Thalinger [mailto:christian.thalinger at oracle.com] > Sent: Donnerstag, 15. Mai 2014 17:14 > To: Lindenmaier, Goetz > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp > > Looks good to me. I thought that prefetch is used more often but apparently not. > > On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: > >> Hi, >> >> This change introduces an umbrella header for prefetch__.inline.hpp >> files: runtime/prefetch.inline.hpp as proposed in >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >> This follows the scheme applied to atomic.inline.hpp. >> >> Further this change adds includes of prefetch.inline.hpp in all .cpp >> and .inline.hpp files where a method of Prefetch declared 'inline' is >> called. >> >> Finally it moves macros calling inline methods of Prefetch from >> space.hpp to the corresponding .inline.hpp file and adds the necessary >> includes in files using the moved macros. >> >> Please review and test this change. I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >> >> Thanks and best regards, >> Goetz. From jvanek at redhat.com Mon May 19 13:21:19 2014 From: jvanek at redhat.com (Jiri Vanek) Date: Mon, 19 May 2014 15:21:19 +0200 Subject: [rfc] hs_err log in temp readable for all Message-ID: <537A054F.6060507@redhat.com> Hi! I run JVM in read only directory in shared server and form time to time it fails. However, I noted that hs_logs which then go to TMP direcotry, are readable to all. It may contains private information, and I think that this file should be readable only to its owner. Attached webrevs fixes it: for all hs_err logs - http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/minimal/1/webrev/ for hs_log in tmp only - http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/better/1/webrev/ If I will be even more paranoid, then i think that hs_err{pid} name is predictable, and can point to pre-created pipe. Maybe this 0600 file should be placed in 0700 directory with salt in name. Or add the salt to file itself (when created in tmp) I will be happy to adapt the patch for your needs. Best regards, J. From zhengyu.gu at oracle.com Mon May 19 13:58:52 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Mon, 19 May 2014 09:58:52 -0400 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' Message-ID: <537A0E1C.4070605@oracle.com> This is a simple fix for incorrect lock state. The timing on setting thread's pending monitor can result stack trace dump reporting incorrect lock state. The solution is to check the monitor's owner, if the owner is other than the current thread, then the monitor, is or is in process of being, set the pending monitor of current thread. Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ Thanks, -Zhengyu From goetz.lindenmaier at sap.com Mon May 19 14:43:50 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 19 May 2014 14:43:50 +0000 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <5379E919.7010803@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> Hi Stefan, thanks for the comments! I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. I also saw another usage of Prefetch in klass.hpp. I removed the macro altogether as it's unused. I kept the include in precompiled.hpp though, as for other files it's listed there, too. If you insist, and there are no other objections, I'll remove it. The new webrev is at http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 Best regards, Goetz. -----Original Message----- From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] Sent: Montag, 19. Mai 2014 13:21 To: Lindenmaier, Goetz; Christian Thalinger Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp Hi Goetz, On 2014-05-19 12:58, Lindenmaier, Goetz wrote: > Hi Christian, > > thanks for the review! > > Although prefetch is not used that often, I think it's good > that all headers in the os_cpu directories are handled > similarly now, i.e., they are only included in the respective > shared file. (For some it's the file in the /cpu/ dir, like > copy_linux_x86.inline.hpp.) > > Could you sponsor the change? Do I need a second reviewer? Thanks for cleaning this up. Some minor comments: http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html You shouldn't have to include prefetch.inline.hpp from this file. http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html heapRegion.cpp needs to include space.inline.hpp since it uses the SCAN_AND_FORWARD macro. http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html I don't think its necessary to add the prefetch.inline.hpp to precompiled.hpp. Otherwise, this looks good to me. thanks, StefanK > > Best regards, > Goetz. > > > > -----Original Message----- > From: Christian Thalinger [mailto:christian.thalinger at oracle.com] > Sent: Donnerstag, 15. Mai 2014 17:14 > To: Lindenmaier, Goetz > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp > > Looks good to me. I thought that prefetch is used more often but apparently not. > > On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: > >> Hi, >> >> This change introduces an umbrella header for prefetch__.inline.hpp >> files: runtime/prefetch.inline.hpp as proposed in >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >> This follows the scheme applied to atomic.inline.hpp. >> >> Further this change adds includes of prefetch.inline.hpp in all .cpp >> and .inline.hpp files where a method of Prefetch declared 'inline' is >> called. >> >> Finally it moves macros calling inline methods of Prefetch from >> space.hpp to the corresponding .inline.hpp file and adds the necessary >> includes in files using the moved macros. >> >> Please review and test this change. I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >> >> Thanks and best regards, >> Goetz. From stefan.karlsson at oracle.com Mon May 19 15:18:11 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 19 May 2014 17:18:11 +0200 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> Message-ID: <537A20B3.1040104@oracle.com> On 2014-05-19 16:43, Lindenmaier, Goetz wrote: > Hi Stefan, > > thanks for the comments! > I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. > I also saw another usage of Prefetch in klass.hpp. I removed > the macro altogether as it's unused. > > I kept the include in precompiled.hpp though, as for other > files it's listed there, too. If you insist, and > there are no other objections, I'll remove it. OK. > > The new webrev is at > http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 Looks good. thanks, StefanK > > Best regards, > Goetz. > > -----Original Message----- > From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] > Sent: Montag, 19. Mai 2014 13:21 > To: Lindenmaier, Goetz; Christian Thalinger > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp > > Hi Goetz, > > On 2014-05-19 12:58, Lindenmaier, Goetz wrote: >> Hi Christian, >> >> thanks for the review! >> >> Although prefetch is not used that often, I think it's good >> that all headers in the os_cpu directories are handled >> similarly now, i.e., they are only included in the respective >> shared file. (For some it's the file in the /cpu/ dir, like >> copy_linux_x86.inline.hpp.) >> >> Could you sponsor the change? Do I need a second reviewer? > Thanks for cleaning this up. > > Some minor comments: > > http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html > > You shouldn't have to include prefetch.inline.hpp from this file. > > > http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html > > heapRegion.cpp needs to include space.inline.hpp since it uses the > SCAN_AND_FORWARD macro. > > > http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html > > I don't think its necessary to add the prefetch.inline.hpp to > precompiled.hpp. > > > Otherwise, this looks good to me. > > thanks, > StefanK > >> Best regards, >> Goetz. >> >> >> >> -----Original Message----- >> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >> Sent: Donnerstag, 15. Mai 2014 17:14 >> To: Lindenmaier, Goetz >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >> >> Looks good to me. I thought that prefetch is used more often but apparently not. >> >> On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: >> >>> Hi, >>> >>> This change introduces an umbrella header for prefetch__.inline.hpp >>> files: runtime/prefetch.inline.hpp as proposed in >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >>> This follows the scheme applied to atomic.inline.hpp. >>> >>> Further this change adds includes of prefetch.inline.hpp in all .cpp >>> and .inline.hpp files where a method of Prefetch declared 'inline' is >>> called. >>> >>> Finally it moves macros calling inline methods of Prefetch from >>> space.hpp to the corresponding .inline.hpp file and adds the necessary >>> includes in files using the moved macros. >>> >>> Please review and test this change. I please need a sponsor. >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >>> >>> Thanks and best regards, >>> Goetz. From jon.masamitsu at oracle.com Mon May 19 16:24:58 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 19 May 2014 09:24:58 -0700 Subject: RFR: 8039489: Refactor test framework for dynamic VM options In-Reply-To: <53737D83.2050205@oracle.com> References: <535A524D.1000701@oracle.com> <53737D83.2050205@oracle.com> Message-ID: <537A305A.3060708@oracle.com> On 05/14/2014 07:28 AM, Dmitry Fazunenko wrote: > Hi, > > Would you please look at the new version of a common library for > testing dynamic VM options. > > Description: > Some time ago a library for testing VM flags which could be changed > dynamically was integrated into the hs repository: > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOptionChecker.java > test/testlibrary/com/oracle/java/testlibrary/TestDynamicVMOption.java > > The library was designed in not the most optimal way. > This change is an attempt to redesign it: > > test/testlibrary/com/oracle/java/testlibrary/DynamicVMOption.java > > is a new version which provides a convenient access to the HS API on > changing VM flags. > The tests based on the library are also refactored to meet new API. > > You comments will be much appreciated. > > http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8039489 > 49 String badValues[] = { 50 null, 51 "", 52 "not a number", 53 "8.5", 54 Integer.toString(Integer.MIN_VALUE), 55 Integer.toString(Integer.MAX_VALUE), 56 Integer.toString(minValue - 1), 57 "-1024", "-10", "-1", "101", "190", "1997" 58 }; I can see how having one negative integer among the bad values and one positive integer above 100 among the bad values is worth it but what does having 2 more negative values and 2 more positive value over 100 add to the test? How about "-8.5"? http://cr.openjdk.java.net/~ehelin/dmitry/8039489/webrev.00/test/gc/parallelScavenge/TestDynShrinkHeap.java.frames.html 83 int min = DynamicVMOption.getInt(MIN_FREE_RATIO_FLAG_NAME); 84 DynamicVMOption.setInt(MAX_FREE_RATIO_FLAG_NAME, min + 1); Would it be better to set MaxHeapFreeRatio to "min" instead of "min + 1" since some future execution of the test could set the min value to 100? Rest looks good. Jon > I want to thank Erik Helin for his incredible assistance with working > on this change. > > Thanks, > Dima From christian.thalinger at oracle.com Mon May 19 16:48:45 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 19 May 2014 09:48:45 -0700 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <537A20B3.1040104@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> <537A20B3.1040104@oracle.com> Message-ID: <7E3E56CF-46B6-4869-97E0-D3D014844DF6@oracle.com> Alright. I will push this to hs-comp. On May 19, 2014, at 8:18 AM, Stefan Karlsson wrote: > On 2014-05-19 16:43, Lindenmaier, Goetz wrote: >> Hi Stefan, >> >> thanks for the comments! >> I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. >> I also saw another usage of Prefetch in klass.hpp. I removed >> the macro altogether as it's unused. >> >> I kept the include in precompiled.hpp though, as for other >> files it's listed there, too. If you insist, and >> there are no other objections, I'll remove it. > > OK. > >> >> The new webrev is at >> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 > > Looks good. > > thanks, > StefanK >> >> Best regards, >> Goetz. >> >> -----Original Message----- >> From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] >> Sent: Montag, 19. Mai 2014 13:21 >> To: Lindenmaier, Goetz; Christian Thalinger >> Cc: hotspot-dev at openjdk.java.net >> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >> >> Hi Goetz, >> >> On 2014-05-19 12:58, Lindenmaier, Goetz wrote: >>> Hi Christian, >>> >>> thanks for the review! >>> >>> Although prefetch is not used that often, I think it's good >>> that all headers in the os_cpu directories are handled >>> similarly now, i.e., they are only included in the respective >>> shared file. (For some it's the file in the /cpu/ dir, like >>> copy_linux_x86.inline.hpp.) >>> >>> Could you sponsor the change? Do I need a second reviewer? >> Thanks for cleaning this up. >> >> Some minor comments: >> >> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html >> >> You shouldn't have to include prefetch.inline.hpp from this file. >> >> >> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html >> >> heapRegion.cpp needs to include space.inline.hpp since it uses the >> SCAN_AND_FORWARD macro. >> >> >> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html >> >> I don't think its necessary to add the prefetch.inline.hpp to >> precompiled.hpp. >> >> >> Otherwise, this looks good to me. >> >> thanks, >> StefanK >> >>> Best regards, >>> Goetz. >>> >>> >>> >>> -----Original Message----- >>> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >>> Sent: Donnerstag, 15. Mai 2014 17:14 >>> To: Lindenmaier, Goetz >>> Cc: hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>> >>> Looks good to me. I thought that prefetch is used more often but apparently not. >>> >>> On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: >>> >>>> Hi, >>>> >>>> This change introduces an umbrella header for prefetch__.inline.hpp >>>> files: runtime/prefetch.inline.hpp as proposed in >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >>>> This follows the scheme applied to atomic.inline.hpp. >>>> >>>> Further this change adds includes of prefetch.inline.hpp in all .cpp >>>> and .inline.hpp files where a method of Prefetch declared 'inline' is >>>> called. >>>> >>>> Finally it moves macros calling inline methods of Prefetch from >>>> space.hpp to the corresponding .inline.hpp file and adds the necessary >>>> includes in files using the moved macros. >>>> >>>> Please review and test this change. I please need a sponsor. >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >>>> >>>> Thanks and best regards, >>>> Goetz. > From christian.thalinger at oracle.com Mon May 19 16:50:25 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 19 May 2014 09:50:25 -0700 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <7E3E56CF-46B6-4869-97E0-D3D014844DF6@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> <537A20B3.1040104@oracle.com> <7E3E56CF-46B6-4869-97E0-D3D014844DF6@oracle.com> Message-ID: <21CE3D5B-9E7D-4188-BF14-DC097F4C780C@oracle.com> Actually, since you are a Commiter can you prepare a changeset so I can just take and push it? Saves me some time :-) On May 19, 2014, at 9:48 AM, Christian Thalinger wrote: > Alright. I will push this to hs-comp. > > On May 19, 2014, at 8:18 AM, Stefan Karlsson wrote: > >> On 2014-05-19 16:43, Lindenmaier, Goetz wrote: >>> Hi Stefan, >>> >>> thanks for the comments! >>> I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. >>> I also saw another usage of Prefetch in klass.hpp. I removed >>> the macro altogether as it's unused. >>> >>> I kept the include in precompiled.hpp though, as for other >>> files it's listed there, too. If you insist, and >>> there are no other objections, I'll remove it. >> >> OK. >> >>> >>> The new webrev is at >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 >> >> Looks good. >> >> thanks, >> StefanK >>> >>> Best regards, >>> Goetz. >>> >>> -----Original Message----- >>> From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] >>> Sent: Montag, 19. Mai 2014 13:21 >>> To: Lindenmaier, Goetz; Christian Thalinger >>> Cc: hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>> >>> Hi Goetz, >>> >>> On 2014-05-19 12:58, Lindenmaier, Goetz wrote: >>>> Hi Christian, >>>> >>>> thanks for the review! >>>> >>>> Although prefetch is not used that often, I think it's good >>>> that all headers in the os_cpu directories are handled >>>> similarly now, i.e., they are only included in the respective >>>> shared file. (For some it's the file in the /cpu/ dir, like >>>> copy_linux_x86.inline.hpp.) >>>> >>>> Could you sponsor the change? Do I need a second reviewer? >>> Thanks for cleaning this up. >>> >>> Some minor comments: >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html >>> >>> You shouldn't have to include prefetch.inline.hpp from this file. >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html >>> >>> heapRegion.cpp needs to include space.inline.hpp since it uses the >>> SCAN_AND_FORWARD macro. >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html >>> >>> I don't think its necessary to add the prefetch.inline.hpp to >>> precompiled.hpp. >>> >>> >>> Otherwise, this looks good to me. >>> >>> thanks, >>> StefanK >>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >>>> Sent: Donnerstag, 15. Mai 2014 17:14 >>>> To: Lindenmaier, Goetz >>>> Cc: hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>> >>>> Looks good to me. I thought that prefetch is used more often but apparently not. >>>> >>>> On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: >>>> >>>>> Hi, >>>>> >>>>> This change introduces an umbrella header for prefetch__.inline.hpp >>>>> files: runtime/prefetch.inline.hpp as proposed in >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >>>>> This follows the scheme applied to atomic.inline.hpp. >>>>> >>>>> Further this change adds includes of prefetch.inline.hpp in all .cpp >>>>> and .inline.hpp files where a method of Prefetch declared 'inline' is >>>>> called. >>>>> >>>>> Finally it moves macros calling inline methods of Prefetch from >>>>> space.hpp to the corresponding .inline.hpp file and adds the necessary >>>>> includes in files using the moved macros. >>>>> >>>>> Please review and test this change. I please need a sponsor. >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >>>>> >>>>> Thanks and best regards, >>>>> Goetz. >> > From john.r.rose at oracle.com Mon May 19 18:31:55 2014 From: john.r.rose at oracle.com (John Rose) Date: Mon, 19 May 2014 11:31:55 -0700 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5375FD00.90603@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> <53750694.3080702@univ-mlv.fr> <5375AA6E.6030402@oracle.com> <5375FD00.90603@oracle.com> Message-ID: <76B72B6C-B078-42F0-80DE-77DD56D92F92@oracle.com> On May 16, 2014, at 4:56 AM, Tobias Hartmann wrote: > Is it sufficient then to use synchronized (lambdaForms) { ... } in setCachedLambdaForm(..) and a normal read in cachedLambdaForm(..)? Yes, that is how I see it. The fast path is a racy non-volatile read of a safely-published structure. (If safe publication via arrays were broken, java.lang.String would be broken. But the JMM is carefully designed to support safe publication of array elements, and through array elements.) ? John From john.r.rose at oracle.com Mon May 19 19:24:25 2014 From: john.r.rose at oracle.com (John Rose) Date: Mon, 19 May 2014 12:24:25 -0700 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5374832F.5000800@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> Message-ID: On May 15, 2014, at 2:04 AM, Vladimir Ivanov wrote: > Cache is written only once, so it has only 2 states (null and non-null value) during it's lifecycle. > > The only stale value cachedLambdaForm() can see is null, but then the caller tries to initialize the cache and after acquiring the lock in setCachedLambdaForm() sees actual value. > > So, the worst thing can happen if readers aren't synchronized is creation of unnecessary LF (which go dead right away) in rare cases. Lazy evaluation is a pattern that we use (or try to use) in a lot of places. It's important to understand it and get it right. With JDK-8005873, the root cause is that lazy evaluation is producing multiple right answers, and an assert is throwing a panic. The correct fix is to use a CAS or lock (or equivalent) to make sure that, even if several threads race to produce a right answer, only one right answer gets published and used. I coded this completely free of locks, in 'setCachedLambdaForm', because I thought that multiple right answers would be harmless. That's arguably still correct, but if it makes assertions fire, we should clean up the edge case, rather than detune the assert. Uniquification can be done with a lock or a CAS. CAS is preferable, but hard to code. AtomicReferenceArray provides easy access to CAS, but you pay a cost for simple 'get', since that is volatile. That's not desirable; in fact the @Stable property is more appropriate. On balance I would prefer a simple lock, such as this: + synchronized public LambdaForm setCachedLambdaForm(int which, LambdaForm form) { - // Should we perform some sort of CAS, to avoid racy duplication? + // Simulate a CAS, to avoid racy duplication of results. + LambdaForm prev = lambdaForms[which]; + if (prev != null) return prev; return lambdaForms[which] = form; } Once lazy results are guaranteed unique the rest of the algorithm is fine, I think. See below. ? John P.S. More discussion of lazy evaluation and races, assuming published values are unique. If the state variable behind a lazy evaluation is a pointer to safely-published data, then a racy read will pick up either the initial state (null) or the desired data. A slow path for null detection needs to reliably pick up the true value of the variable, to avoid starvation and multiple initialization. Doing the slow-path read and write inside of a lock satisfies the requirements of the JMM for correct race-free code. Outside of the lock, it is important that anything observed by the racy read eventually lead to a correct answer, even if the racy read gives an out-of-date answer. If the lock's critical section is trivial (no object construction or side effects) then a CAS is usually faster, but it is hard to code correctly. The new JMM will fix this we hope and make it possible to perform safe and sane CAS operations. Meanwhile, the atomic reference array would work, but it is (IMO) overly expensive since it performs The bottom line is that we don't need any volatile references on the fast path. One hazard is that the JMM does not guarantee safe publication in either of the following two cases: Caveat #1. Data is accessed along a path that does not use a final field. Caveat #2. Data is accessed along a path that has been modified since construction. Caveat #1 is usually (by itself) not a problem, if the access path goes through an object with at least one final field. (That's because existing implementations use a broad fence at the end of a constructor if an object contains any non-static finals.) And the new JMM may further downgrade or remove Caveat #1. Caveat #2 includes the case of a non-final field in a "mostly final" object. The field LambdaForm.vmentry is such a field. Any non-final fields accessed through a vmentry patched into a published LambdaForm are racy, unless some sort of fence has been issued before the vmentry object is patched into the lambda form. Caveat #2 It also includes the case where the path goes through an array element which was changed since the root object was constructed. This happens in various tabular structures where an array that starts out full of nulls ends up containing lazily evaluated entries. The entries should each be safely published. Caveat #2 does *not* apply to stable array substructures like LamdaForm.names, if (as is the case with 'names') the arrays are assigned before the enclosing object is fully constructed, and the array references themselves are final. From forax at univ-mlv.fr Mon May 19 21:42:54 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 19 May 2014 23:42:54 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> Message-ID: <146166fdbc0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> Hi John, i don't understand how the Jit can remove the memory barrier due to the synchronized when prev is not null ? R?mi Envoy? avec AquaMail pour Android http://www.aqua-mail.com Le 19 mai 2014 21:52:20 John Rose a ?crit : > On May 15, 2014, at 2:04 AM, Vladimir Ivanov > wrote: > > > Cache is written only once, so it has only 2 states (null and non-null > value) during it's lifecycle. > > The only stale value cachedLambdaForm() can see is null, but then the > caller tries to initialize the cache and after acquiring the lock in > setCachedLambdaForm() sees actual value. > > So, the worst thing can happen if readers aren't synchronized is creation > of unnecessary LF (which go dead right away) in rare cases. > > > Lazy evaluation is a pattern that we use (or try to use) in a lot of > places. It's important to understand it and get it right. > > With JDK-8005873, the root cause is that lazy evaluation is producing > multiple right answers, and an assert is throwing a panic. The correct fix > is to use a CAS or lock (or equivalent) to make sure that, even if several > threads race to produce a right answer, only one right answer gets > published and used. > > I coded this completely free of locks, in 'setCachedLambdaForm', because I > thought that multiple right answers would be harmless. That's arguably > still correct, but if it makes assertions fire, we should clean up the edge > case, rather than detune the assert. > > Uniquification can be done with a lock or a CAS. CAS is preferable, but > hard to code. AtomicReferenceArray provides easy access to CAS, but you > pay a cost for simple 'get', since that is volatile. That's not desirable; > in fact the @Stable property is more appropriate. On balance I would > prefer a simple lock, such as this: > > + synchronized > public LambdaForm setCachedLambdaForm(int which, LambdaForm form) { > - // Should we perform some sort of CAS, to avoid racy duplication? > + // Simulate a CAS, to avoid racy duplication of results. > + LambdaForm prev = lambdaForms[which]; > + if (prev != null) return prev; > return lambdaForms[which] = form; > } > > Once lazy results are guaranteed unique the rest of the algorithm is fine, > I think. See below. > > ? John > > P.S. More discussion of lazy evaluation and races, assuming published > values are unique. > > If the state variable behind a lazy evaluation is a pointer to > safely-published data, then a racy read will pick up either the initial > state (null) or the desired data. > > A slow path for null detection needs to reliably pick up the true value of > the variable, to avoid starvation and multiple initialization. Doing the > slow-path read and write inside of a lock satisfies the requirements of the > JMM for correct race-free code. > > Outside of the lock, it is important that anything observed by the racy > read eventually lead to a correct answer, even if the racy read gives an > out-of-date answer. > > If the lock's critical section is trivial (no object construction or side > effects) then a CAS is usually faster, but it is hard to code correctly. > The new JMM will fix this we hope and make it possible to perform safe and > sane CAS operations. Meanwhile, the atomic reference array would work, but > it is (IMO) overly expensive since it performs > > The bottom line is that we don't need any volatile references on the fast path. > > One hazard is that the JMM does not guarantee safe publication in either of > the following two cases: > > Caveat #1. Data is accessed along a path that does not use a final field. > Caveat #2. Data is accessed along a path that has been modified since > construction. > > Caveat #1 is usually (by itself) not a problem, if the access path goes > through an object with at least one final field. (That's because existing > implementations use a broad fence at the end of a constructor if an object > contains any non-static finals.) And the new JMM may further downgrade or > remove Caveat #1. > > Caveat #2 includes the case of a non-final field in a "mostly final" > object. The field LambdaForm.vmentry is such a field. Any non-final > fields accessed through a vmentry patched into a published LambdaForm are > racy, unless some sort of fence has been issued before the vmentry object > is patched into the lambda form. > > Caveat #2 It also includes the case where the path goes through an array > element which was changed since the root object was constructed. This > happens in various tabular structures where an array that starts out full > of nulls ends up containing lazily evaluated entries. The entries should > each be safely published. > > Caveat #2 does *not* apply to stable array substructures like > LamdaForm.names, if (as is the case with 'names') the arrays are assigned > before the enclosing object is fully constructed, and the array references > themselves are final. > > From vitalyd at gmail.com Mon May 19 21:49:17 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 19 May 2014 17:49:17 -0400 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <146166fdbc0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <146166fdbc0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> Message-ID: Synchronized is only on the setter, which is the slow path. Sent from my phone On May 19, 2014 5:43 PM, "Remi Forax" wrote: > Hi John, > i don't understand how the Jit can remove the memory barrier due to the > synchronized when prev is not null ? > > R?mi > > Envoy? avec AquaMail pour Android > http://www.aqua-mail.com > > > Le 19 mai 2014 21:52:20 John Rose a ?crit : > > On May 15, 2014, at 2:04 AM, Vladimir Ivanov < >> vladimir.x.ivanov at oracle.com> wrote: >> >> > Cache is written only once, so it has only 2 states (null and non-null >> value) during it's lifecycle. >> > The only stale value cachedLambdaForm() can see is null, but then the >> caller tries to initialize the cache and after acquiring the lock in >> setCachedLambdaForm() sees actual value. >> > So, the worst thing can happen if readers aren't synchronized is >> creation of unnecessary LF (which go dead right away) in rare cases. >> >> >> Lazy evaluation is a pattern that we use (or try to use) in a lot of >> places. It's important to understand it and get it right. >> >> With JDK-8005873, the root cause is that lazy evaluation is producing >> multiple right answers, and an assert is throwing a panic. The correct fix >> is to use a CAS or lock (or equivalent) to make sure that, even if several >> threads race to produce a right answer, only one right answer gets >> published and used. >> >> I coded this completely free of locks, in 'setCachedLambdaForm', because >> I thought that multiple right answers would be harmless. That's arguably >> still correct, but if it makes assertions fire, we should clean up the edge >> case, rather than detune the assert. >> >> Uniquification can be done with a lock or a CAS. CAS is preferable, but >> hard to code. AtomicReferenceArray provides easy access to CAS, but you >> pay a cost for simple 'get', since that is volatile. That's not desirable; >> in fact the @Stable property is more appropriate. On balance I would >> prefer a simple lock, such as this: >> >> + synchronized >> public LambdaForm setCachedLambdaForm(int which, LambdaForm form) { >> - // Should we perform some sort of CAS, to avoid racy duplication? >> + // Simulate a CAS, to avoid racy duplication of results. >> + LambdaForm prev = lambdaForms[which]; >> + if (prev != null) return prev; >> return lambdaForms[which] = form; >> } >> >> Once lazy results are guaranteed unique the rest of the algorithm is >> fine, I think. See below. >> >> ? John >> >> P.S. More discussion of lazy evaluation and races, assuming published >> values are unique. >> >> If the state variable behind a lazy evaluation is a pointer to >> safely-published data, then a racy read will pick up either the initial >> state (null) or the desired data. >> >> A slow path for null detection needs to reliably pick up the true value >> of the variable, to avoid starvation and multiple initialization. Doing >> the slow-path read and write inside of a lock satisfies the requirements of >> the JMM for correct race-free code. >> >> Outside of the lock, it is important that anything observed by the racy >> read eventually lead to a correct answer, even if the racy read gives an >> out-of-date answer. >> >> If the lock's critical section is trivial (no object construction or side >> effects) then a CAS is usually faster, but it is hard to code correctly. >> The new JMM will fix this we hope and make it possible to perform safe and >> sane CAS operations. Meanwhile, the atomic reference array would work, but >> it is (IMO) overly expensive since it performs >> >> The bottom line is that we don't need any volatile references on the fast >> path. >> >> One hazard is that the JMM does not guarantee safe publication in either >> of the following two cases: >> >> Caveat #1. Data is accessed along a path that does not use a final field. >> Caveat #2. Data is accessed along a path that has been modified since >> construction. >> >> Caveat #1 is usually (by itself) not a problem, if the access path goes >> through an object with at least one final field. (That's because existing >> implementations use a broad fence at the end of a constructor if an object >> contains any non-static finals.) And the new JMM may further downgrade or >> remove Caveat #1. >> >> Caveat #2 includes the case of a non-final field in a "mostly final" >> object. The field LambdaForm.vmentry is such a field. Any non-final >> fields accessed through a vmentry patched into a published LambdaForm are >> racy, unless some sort of fence has been issued before the vmentry object >> is patched into the lambda form. >> >> Caveat #2 It also includes the case where the path goes through an array >> element which was changed since the root object was constructed. This >> happens in various tabular structures where an array that starts out full >> of nulls ends up containing lazily evaluated entries. The entries should >> each be safely published. >> >> Caveat #2 does *not* apply to stable array substructures like >> LamdaForm.names, if (as is the case with 'names') the arrays are assigned >> before the enclosing object is fully constructed, and the array references >> themselves are final. >> >> >> From rednaxelafx at gmail.com Mon May 19 22:00:07 2014 From: rednaxelafx at gmail.com (Krystal Mok) Date: Mon, 19 May 2014 15:00:07 -0700 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537A0E1C.4070605@oracle.com> References: <537A0E1C.4070605@oracle.com> Message-ID: Hi Zhengyu, I just glanced over your changes, and the description looks like something I've seen in the past: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2012-July/004106.html Could you please take a look at the old discussion thread and see if they're the same thing? Thanks, Kris On Mon, May 19, 2014 at 6:58 AM, Zhengyu Gu wrote: > This is a simple fix for incorrect lock state. > > The timing on setting thread's pending monitor can result stack trace dump > reporting incorrect lock state. The solution is to check the monitor's > owner, if the owner is other than the current thread, then the monitor, is > or is in process of being, set the pending monitor of current thread. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 > Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ > > > Thanks, > > -Zhengyu > From daniel.daugherty at oracle.com Mon May 19 23:47:14 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 19 May 2014 17:47:14 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537A0E1C.4070605@oracle.com> References: <537A0E1C.4070605@oracle.com> Message-ID: <537A9802.70707@oracle.com> On 5/19/14 7:58 AM, Zhengyu Gu wrote: > This is a simple fix for incorrect lock state. > > The timing on setting thread's pending monitor can result stack trace > dump reporting incorrect lock state. The solution is to check the > monitor's owner, if the owner is other than the current thread, then > the monitor, is or is in process of being, set the pending monitor of > current thread. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 > Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ src/share/vm/runtime/vframe.cpp First off, I concur that the existing code has a problem. However, I think the proposed solution has at least one different problem. Here's the original code: 183 // Print out all monitors that we have locked or are trying to lock 184 GrowableArray* mons = monitors(); 185 if (!mons->is_empty()) { 186 bool found_first_monitor = false; 187 for (int index = (mons->length()-1); index >= 0; index--) { 188 MonitorInfo* monitor = mons->at(index); 189 if (monitor->eliminated() && is_compiled_frame()) { // Eliminated in compiled code 190 if (monitor->owner_is_scalar_replaced()) { 191 Klass* k = java_lang_Class::as_Klass(monitor->owner_klass()); 192 st->print("\t- eliminated (a %s)", k->external_name()); 193 } else { 194 oop obj = monitor->owner(); 195 if (obj != NULL) { 196 print_locked_object_class_name(st, obj, "eliminated"); 197 } 198 } 199 continue; 200 } 201 if (monitor->owner() != NULL) { 202 203 // First, assume we have the monitor locked. If we haven't found an 204 // owned monitor before and this is the first frame, then we need to 205 // see if we have completed the lock or we are blocked trying to 206 // acquire it - we can only be blocked if the monitor is inflated 207 208 const char *lock_state = "locked"; // assume we have the monitor locked 209 if (!found_first_monitor && frame_count == 0) { 210 markOop mark = monitor->owner()->mark(); 211 if (mark->has_monitor() && 212 mark->monitor() == thread()->current_pending_monitor()) { 213 lock_state = "waiting to lock"; 214 } 215 } 216 217 found_first_monitor = true; 218 print_locked_object_class_name(st, monitor->owner(), lock_state); 219 } 220 } 221 } The algorithm is pretty basic (no pun intended): - for each monitor (BasicLock) in the Java frame - if the monitor has been eliminated and this is a compiled frame - do compiler specific handling - continue - if the monitor has a owner - set lock_state = "locked" - if this is the first monitor and the top most frame - set mark = associated Object's mark - if the mark has an ObjectMonitor and the Java thread is pending on that monitor - set lock_state = "waiting to lock" - set found_first_monitor flag - print info about current monitor including the lock_state A BasicLock is a Java monitor from the Java frame's point of view. It might be implemented as a stack lock, a biased lock or an inflated lock (ObjectMonitor). The problem that I see with the algorithm is that it is traversing a collection of BasicLocks and then asking lock implementation specific questions in a an effort to determine if the BasicLock is owned by the Java thread whose frames we are traversing. We already know that this code: mark->monitor() == thread()->current_pending_monitor() is specific to inflated monitors (ObjectMonitor) and it has a flaw because the target thread can be stopped at a safepoint before the _current_pending_monitor field is set. The proposed code: mark->monitor()->owner() != thread() compares the owner of the inflated monitor to the Java thread whose frames we are traversing. The problem is that the ObjectMonitor::_owner field is not always a pointer to the JavaThread. Right after a stack lock is inflated, the ObjectMonitor::_owner field is set to the mark->locker() stack address; see: src/share/vm/runtime/synchronizer.cpp: line 1283: m->set_owner(mark->locker()); So the proposed code will report "waiting to lock" for a Java monitor where the monitor is just transitioning from a stack lock to an inflated lock. Of course, the race is even tighter in this case. The target Java thread (T1) has to have just acquired the uninflated monitor and still be in the same frame in which that monitor was acquired at the same time that another thread (T2) lost the race to acquire the monitor. So T2 has to have just inflated the monitor, T1 has still be in the same frame in which it acquired the monitor and someone has to ask for a thread dump. Pretty cool. Another thought has occurred to me. T1 owns the monitor, T2 is contending for the monitor and is starting to inflate it, we request a thread dump. Is it possible to see the BasicLock in T2's Java frame, see that the Object does not yet have an inflated monitor (!mark->has_monitor()) and report that T2 has the monitor "locked"? This last scenario would again result in more than one thread being reported as having the same monitor locked. To sum up: I think both the current code and the proposed code have issues and we need to look at this a different way. We probably something like: MonitorInfo::is_locked_by_thread(JavaThread jt) that encapsulates and isolates the logic needed to determine if JavaThread 'jt' owns the BasicLock. It will need to know how to check: - stack lock ownership (see JavaThread::is_lock_owned()) - biased lock ownership (don't know how to do this one) - inflated lock ownership (_owner field == jt) The stack lock ownership check needs to be done before the inflated lock ownership check. That's because the _owner field in the inflated lock can be the mark->locker() stack address when it is initially inflated. I couldn't figure out the biased locking ownership check quickly so I don't know where it fits in the checking ownership pecking order. Sorry this turned out to be not so simple... Welcome to the joy of locking... Dan P.S. Of course, now that we've discovered at least one misuse of current_pending_monitor(), we should probably audit the other uses of it also... > > Thanks, > > -Zhengyu From mike.duigou at oracle.com Tue May 20 00:50:34 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Mon, 19 May 2014 17:50:34 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: References: <531F05D3.6020008@oracle.com> Message-ID: <6A7B5F26-FE6C-439E-8C40-4BEE2DD739C7@oracle.com> I am running final (I _really_ hope) tests on this issue but had to make a few changes to the hotspot makefiles: The patch has been enhanced since the last revision to ensure that the linker flags are incorporated into the actual linker flags for fastdebug mode. Previously this was only being done on Solaris due to unexpected behavioural differences between the platform's makefiles. The bsd, linux and solaris makefiles now handle the FASTDEBUG_CFLAGS the same way. I've also checked and don't believe that changing the handling of FASTDEBUG_CFLAGS resulted in any unexpected flag changes. http://cr.openjdk.java.net/~mduigou/JDK-8032045/7/webrev/ Mike On May 8 2014, at 12:13 , Mike Duigou wrote: > Ping. Still needing reviewers on this issue. > > I plan to push this change through the hotspot-rt repo. > > Mike > > On May 3 2014, at 20:24 , Mike Duigou wrote: > >> Hello; >> >> Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. >> >> This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. >> >> https://bugs.openjdk.java.net/browse/JDK-8032045 >> http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ >> >> Mike >> >> On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: >> >>> On 2014-03-11 00:49, Mike Duigou wrote: >>>> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>>> >>>> This version is, hopefully, almost ready to be pushed. >>> >>> I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. >>> >>> The top-level build changes looks fine. Thank you Mike for cleaning things up! >>> >>> /Magnus >>> >>>> >>>> Mike >>>> >>>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>>> >>>>> Hello all; >>>>> >>>>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>>>> >>>>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>>>> >>>>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>>>> >>>>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>>>> >>>>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>>>> >>>>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>>> >>>>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>>> >>>>> Thanks, >>>>> >>>>> Mike >>> >> > From vladimir.kozlov at oracle.com Tue May 20 01:45:33 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 19 May 2014 18:45:33 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <6A7B5F26-FE6C-439E-8C40-4BEE2DD739C7@oracle.com> References: <531F05D3.6020008@oracle.com> <6A7B5F26-FE6C-439E-8C40-4BEE2DD739C7@oracle.com> Message-ID: <537AB3BD.9060708@oracle.com> Looks fine to me. We don't use gcc on Solaris. How you tested solaris/makefiles/gcc.make changes? Thanks, Vladimir On 5/19/14 5:50 PM, Mike Duigou wrote: > I am running final (I _really_ hope) tests on this issue but had to make a few changes to the hotspot makefiles: > > The patch has been enhanced since the last revision to ensure that the linker flags are incorporated into the actual linker flags for fastdebug mode. Previously this was only being done on Solaris due to unexpected behavioural differences between the platform's makefiles. The bsd, linux and solaris makefiles now handle the FASTDEBUG_CFLAGS the same way. I've also checked and don't believe that changing the handling of FASTDEBUG_CFLAGS resulted in any unexpected flag changes. > > http://cr.openjdk.java.net/~mduigou/JDK-8032045/7/webrev/ > > Mike > > On May 8 2014, at 12:13 , Mike Duigou wrote: > >> Ping. Still needing reviewers on this issue. >> >> I plan to push this change through the hotspot-rt repo. >> >> Mike >> >> On May 3 2014, at 20:24 , Mike Duigou wrote: >> >>> Hello; >>> >>> Finally getting back to this issue I have done some cleanup and adjusted the hotspot gcc.make files to use VARIANT rather than DEBUG_LEVEL. >>> >>> This version also add support for the "-fsanitize=undefined" undefined behaviour warnings feature when it is available (Clang and GCC 4.9). The code to emit the option has been added for Clang but I haven't yet added test for the option's availability under Clang so currently it will be enabled only for GCC. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/6/webrev/ >>> >>> Mike >>> >>> On Mar 11 2014, at 05:47 , Magnus Ihse Bursie wrote: >>> >>>> On 2014-03-11 00:49, Mike Duigou wrote: >>>>> I have updated the patch to respond to Magnus's feedback and to accommodate intervening changes to the configure and hotspot make files. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/3/webrev/ >>>>> >>>>> This version is, hopefully, almost ready to be pushed. >>>> >>>> I have only glanced at the hotspot build changes and can't really say anything about them. The hotspot team still owns these; I'm cc:ing them now. >>>> >>>> The top-level build changes looks fine. Thank you Mike for cleaning things up! >>>> >>>> /Magnus >>>> >>>>> >>>>> Mike >>>>> >>>>> On Feb 20 2014, at 15:43 , Mike Duigou wrote: >>>>> >>>>>> Hello all; >>>>>> >>>>>> This issue is a followon to JDK-8030350 (https://bugs.openjdk.java.net/browse/JDK-8030350) which enhanced the compiler warnings used for compiling native code. The proposed changes principally impact the Linux platform. >>>>>> >>>>>> While 8030350 was focused on compiler warnings which did not impact code generation, this changeset will, for some configurations, change the native code generated and likely change performance. These proposed option changes prevent specific types of relocation table, stack and heap memory corruption in native code. Preventing these types of memory corruption may be useful for finding certain kinds of bugs though and do provide some minimal additional protections against malicious attacks. They aren't, by any means, a substitute for following appropriate secure coding guidelines. >>>>>> >>>>>> The rationale behind the implementation is as follows. For release builds during the initial phase of JDK 9 I would like to enable only compile time checks. This ends up being similar to the warnings in JDK-8030350. These options have no runtime impact on footprint or performance and very minimal additional compile time cost while providing value. **Release builds are not expected to see any performance or footprint change as a result of this changeset** >>>>>> >>>>>> For fast debug builds we can enable linker protections (relro) and static compile time bounds checks (FORTIFY_SOURCE=1). FORTIFY_SOURCE=1 might be moved to the production builds as well because it has no runtime cost or executable size cost. >>>>>> >>>>>> For slow debug builds we can enable full linker protection (at a potential cost in startup time), runtime bounds checks and stack protection (FORTIFY_SOURCE=2 -fprotect-stack-all). We will likely enable -fprotect-stack-strong when available in GCC 4.9 >>>>>> >>>>>> The basis for enabling the additional protections in debug builds is that it will help us find bugs in our native code and we aren't as concerned in debug builds with footprint and performance. Since many developers already do their personal builds in fastdebug or slowdebug mode for testing this will provide good opportunity to shake out any problems with the options while not impacting release builds. Should we find that any of the options provide significant value for their cost we can move them to fastdebug or release. If any of the options prove too costly they can be demoted or removed entirely. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8032045 >>>>>> http://cr.openjdk.java.net/~mduigou/JDK-8032045/2/webrev/ >>>>>> >>>>>> Additional to enabling the various compiler options I attempted to rationalize some of the skew between the various hotspot/make/{platform}/makefiles/gcc.make files while avoiding changing existing behaviour. I have also introduced the new -Og "optimize for debugging" option and there are now an explicit C{XX}_O_FLAG_DEBUG definitions to complement the C{XX}_O_FLAG_{DEBUG|NORM|HI|HIGHEST|NONE} optimization options. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Mike >>>> >>> >> > From david.holmes at oracle.com Tue May 20 02:09:23 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 May 2014 12:09:23 +1000 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537A9802.70707@oracle.com> References: <537A0E1C.4070605@oracle.com> <537A9802.70707@oracle.com> Message-ID: <537AB953.7080906@oracle.com> Dan, My understanding of the existing code and hence the modified code is that we have already established that we are dealing with an inflated monitor and hence the owner really must be the owning thread - not a stack address for a BasicLock. Isn't that what mark->has_monitor() establishes? Note that we can not have reached a safepoint inflation is in progress. Thanks, David On 20/05/2014 9:47 AM, Daniel D. Daugherty wrote: > On 5/19/14 7:58 AM, Zhengyu Gu wrote: >> This is a simple fix for incorrect lock state. >> >> The timing on setting thread's pending monitor can result stack trace >> dump reporting incorrect lock state. The solution is to check the >> monitor's owner, if the owner is other than the current thread, then >> the monitor, is or is in process of being, set the pending monitor of >> current thread. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ > > src/share/vm/runtime/vframe.cpp > First off, I concur that the existing code has a problem. > However, I think the proposed solution has at least one > different problem. > > Here's the original code: > > 183 // Print out all monitors that we have locked or are trying to lock > 184 GrowableArray* mons = monitors(); > 185 if (!mons->is_empty()) { > 186 bool found_first_monitor = false; > 187 for (int index = (mons->length()-1); index >= 0; index--) { > 188 MonitorInfo* monitor = mons->at(index); > 189 if (monitor->eliminated() && is_compiled_frame()) { // > Eliminated in compiled code > 190 if (monitor->owner_is_scalar_replaced()) { > 191 Klass* k = > java_lang_Class::as_Klass(monitor->owner_klass()); > 192 st->print("\t- eliminated (a > %s)", k->external_name()); > 193 } else { > 194 oop obj = monitor->owner(); > 195 if (obj != NULL) { > 196 print_locked_object_class_name(st, obj, "eliminated"); > 197 } > 198 } > 199 continue; > 200 } > 201 if (monitor->owner() != NULL) { > 202 > 203 // First, assume we have the monitor locked. If we haven't > found an > 204 // owned monitor before and this is the first frame, then > we need to > 205 // see if we have completed the lock or we are blocked > trying to > 206 // acquire it - we can only be blocked if the monitor is > inflated > 207 > 208 const char *lock_state = "locked"; // assume we have the > monitor locked > 209 if (!found_first_monitor && frame_count == 0) { > 210 markOop mark = monitor->owner()->mark(); > 211 if (mark->has_monitor() && > 212 mark->monitor() == > thread()->current_pending_monitor()) { > 213 lock_state = "waiting to lock"; > 214 } > 215 } > 216 > 217 found_first_monitor = true; > 218 print_locked_object_class_name(st, monitor->owner(), > lock_state); > 219 } > 220 } > 221 } > > The algorithm is pretty basic (no pun intended): > > - for each monitor (BasicLock) in the Java frame > - if the monitor has been eliminated and this is > a compiled frame > - do compiler specific handling > - continue > - if the monitor has a owner > - set lock_state = "locked" > - if this is the first monitor and the top most frame > - set mark = associated Object's mark > - if the mark has an ObjectMonitor and > the Java thread is pending on that monitor > - set lock_state = "waiting to lock" > - set found_first_monitor flag > - print info about current monitor including > the lock_state > > A BasicLock is a Java monitor from the Java frame's point of > view. It might be implemented as a stack lock, a biased lock > or an inflated lock (ObjectMonitor). > > The problem that I see with the algorithm is that it is > traversing a collection of BasicLocks and then asking > lock implementation specific questions in a an effort to > determine if the BasicLock is owned by the Java thread > whose frames we are traversing. > > We already know that this code: > > mark->monitor() == thread()->current_pending_monitor() > > is specific to inflated monitors (ObjectMonitor) and it > has a flaw because the target thread can be stopped at > a safepoint before the _current_pending_monitor field > is set. > > The proposed code: > > mark->monitor()->owner() != thread() > > compares the owner of the inflated monitor to the Java > thread whose frames we are traversing. The problem is > that the ObjectMonitor::_owner field is not always a > pointer to the JavaThread. Right after a stack lock > is inflated, the ObjectMonitor::_owner field is set > to the mark->locker() stack address; see: > > src/share/vm/runtime/synchronizer.cpp: > line 1283: m->set_owner(mark->locker()); > > So the proposed code will report "waiting to lock" for a > Java monitor where the monitor is just transitioning > from a stack lock to an inflated lock. Of course, the > race is even tighter in this case. The target Java thread > (T1) has to have just acquired the uninflated monitor > and still be in the same frame in which that monitor was > acquired at the same time that another thread (T2) lost > the race to acquire the monitor. So T2 has to have just > inflated the monitor, T1 has still be in the same frame > in which it acquired the monitor and someone has to ask > for a thread dump. Pretty cool. > > Another thought has occurred to me. T1 owns the monitor, > T2 is contending for the monitor and is starting to inflate > it, we request a thread dump. Is it possible to see the > BasicLock in T2's Java frame, see that the Object does not > yet have an inflated monitor (!mark->has_monitor()) and > report that T2 has the monitor "locked"? > > This last scenario would again result in more than one > thread being reported as having the same monitor locked. > > To sum up: > > I think both the current code and the proposed code have > issues and we need to look at this a different way. We > probably something like: > > MonitorInfo::is_locked_by_thread(JavaThread jt) > > that encapsulates and isolates the logic needed to > determine if JavaThread 'jt' owns the BasicLock. It > will need to know how to check: > > - stack lock ownership (see JavaThread::is_lock_owned()) > - biased lock ownership (don't know how to do this one) > - inflated lock ownership (_owner field == jt) > > The stack lock ownership check needs to be done before > the inflated lock ownership check. That's because the > _owner field in the inflated lock can be the > mark->locker() stack address when it is initially > inflated. > > I couldn't figure out the biased locking ownership > check quickly so I don't know where it fits in the > checking ownership pecking order. > > > Sorry this turned out to be not so simple... Welcome to > the joy of locking... > > Dan > > P.S. > Of course, now that we've discovered at least one > misuse of current_pending_monitor(), we should probably > audit the other uses of it also... > > > >> >> Thanks, >> >> -Zhengyu > From david.holmes at oracle.com Tue May 20 02:11:58 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 May 2014 12:11:58 +1000 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: References: <537A0E1C.4070605@oracle.com> Message-ID: <537AB9EE.4050601@oracle.com> Kris, This does indeed appear to be the same issue. I don't recall why that discussion in July 2012 just came to a halt. :( Sorry about that. David On 20/05/2014 8:00 AM, Krystal Mok wrote: > Hi Zhengyu, > > I just glanced over your changes, and the description looks like something > I've seen in the past: > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2012-July/004106.html > > Could you please take a look at the old discussion thread and see if > they're the same thing? > > Thanks, > Kris > > > On Mon, May 19, 2014 at 6:58 AM, Zhengyu Gu wrote: > >> This is a simple fix for incorrect lock state. >> >> The timing on setting thread's pending monitor can result stack trace dump >> reporting incorrect lock state. The solution is to check the monitor's >> owner, if the owner is other than the current thread, then the monitor, is >> or is in process of being, set the pending monitor of current thread. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >> >> >> Thanks, >> >> -Zhengyu >> From mike.duigou at oracle.com Tue May 20 02:14:59 2014 From: mike.duigou at oracle.com (Mike Duigou) Date: Mon, 19 May 2014 19:14:59 -0700 Subject: RFR : JDK-8032045 : (m/l) Enable compiler and linker safety switches for debug builds In-Reply-To: <537AB3BD.9060708@oracle.com> References: <531F05D3.6020008@oracle.com> <6A7B5F26-FE6C-439E-8C40-4BEE2DD739C7@oracle.com> <537AB3BD.9060708@oracle.com> Message-ID: On May 19 2014, at 18:45 , Vladimir Kozlov wrote: > Looks fine to me. > > We don't use gcc on Solaris. How you tested solaris/makefiles/gcc.make changes? I have not been able to test the Solaris changes. It is my understanding that the gcc on solaris makefiles have not been tested in some time and have quite likely bit-rotted. I tried to transplant changes between linux and bsd into Solaris in as conservative a fashion as possible but I did want to avoid additional skew between the solaris, bsd and linux gcc.make files. Mike From daniel.daugherty at oracle.com Tue May 20 05:04:55 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 19 May 2014 23:04:55 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537AB953.7080906@oracle.com> References: <537A0E1C.4070605@oracle.com> <537A9802.70707@oracle.com> <537AB953.7080906@oracle.com> Message-ID: <537AE277.6050808@oracle.com> David, I'll have to politely disagree... Here's the code in ObjectMonitor::exit() that handles the case when a thread finds itself exiting a monitor that was stack locked when it entered, but is now inflated: src/share/vm/runtime/objectMonitor.cpp: 912 void ATTR ObjectMonitor::exit(bool not_suspended, TRAPS) { 913 Thread * Self = THREAD ; 914 if (THREAD != _owner) { 915 if (THREAD->is_lock_owned((address) _owner)) { 916 // Transmute _owner from a BasicLock pointer to a Thread address . 917 // We don't need to hold _mutex for this transition. 918 // Non-null to Non-null is safe as long as all readers can 919 // tolerate either flavor. 920 assert (_recursions == 0, "invariant") ; 921 _owner = THREAD ; 922 _recursions = 0 ; 923 OwnerIsThread = 1 ; 924 } else { 925 // NOTE: we need to handle unbalanced monitor enter/exit 926 // in native code by throwing an exception. 927 // TODO: Throw an IllegalMonitorStateException ? 928 TEVENT (Exit - Throw IMSX) ; 929 assert(false, "Non-balanced monitor enter/exit!"); 930 if (false) { 931 THROW(vmSymbols::java_lang_IllegalMonitorStateException()); 932 } 933 return; 934 } 935 } so that _owner field when we did the stack dump still contained the BaseLock pointer instead of the JavaThread pointer... Dan On 5/19/14 8:09 PM, David Holmes wrote: > Dan, > > My understanding of the existing code and hence the modified code is > that we have already established that we are dealing with an inflated > monitor and hence the owner really must be the owning thread - not a > stack address for a BasicLock. Isn't that what mark->has_monitor() > establishes? > > Note that we can not have reached a safepoint inflation is in progress. > > Thanks, > David > > On 20/05/2014 9:47 AM, Daniel D. Daugherty wrote: >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack trace >>> dump reporting incorrect lock state. The solution is to check the >>> monitor's owner, if the owner is other than the current thread, then >>> the monitor, is or is in process of being, set the pending monitor of >>> current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >> >> src/share/vm/runtime/vframe.cpp >> First off, I concur that the existing code has a problem. >> However, I think the proposed solution has at least one >> different problem. >> >> Here's the original code: >> >> 183 // Print out all monitors that we have locked or are trying >> to lock >> 184 GrowableArray* mons = monitors(); >> 185 if (!mons->is_empty()) { >> 186 bool found_first_monitor = false; >> 187 for (int index = (mons->length()-1); index >= 0; index--) { >> 188 MonitorInfo* monitor = mons->at(index); >> 189 if (monitor->eliminated() && is_compiled_frame()) { // >> Eliminated in compiled code >> 190 if (monitor->owner_is_scalar_replaced()) { >> 191 Klass* k = >> java_lang_Class::as_Klass(monitor->owner_klass()); >> 192 st->print("\t- eliminated (a >> %s)", k->external_name()); >> 193 } else { >> 194 oop obj = monitor->owner(); >> 195 if (obj != NULL) { >> 196 print_locked_object_class_name(st, obj, "eliminated"); >> 197 } >> 198 } >> 199 continue; >> 200 } >> 201 if (monitor->owner() != NULL) { >> 202 >> 203 // First, assume we have the monitor locked. If we haven't >> found an >> 204 // owned monitor before and this is the first frame, then >> we need to >> 205 // see if we have completed the lock or we are blocked >> trying to >> 206 // acquire it - we can only be blocked if the monitor is >> inflated >> 207 >> 208 const char *lock_state = "locked"; // assume we have the >> monitor locked >> 209 if (!found_first_monitor && frame_count == 0) { >> 210 markOop mark = monitor->owner()->mark(); >> 211 if (mark->has_monitor() && >> 212 mark->monitor() == >> thread()->current_pending_monitor()) { >> 213 lock_state = "waiting to lock"; >> 214 } >> 215 } >> 216 >> 217 found_first_monitor = true; >> 218 print_locked_object_class_name(st, monitor->owner(), >> lock_state); >> 219 } >> 220 } >> 221 } >> >> The algorithm is pretty basic (no pun intended): >> >> - for each monitor (BasicLock) in the Java frame >> - if the monitor has been eliminated and this is >> a compiled frame >> - do compiler specific handling >> - continue >> - if the monitor has a owner >> - set lock_state = "locked" >> - if this is the first monitor and the top most frame >> - set mark = associated Object's mark >> - if the mark has an ObjectMonitor and >> the Java thread is pending on that monitor >> - set lock_state = "waiting to lock" >> - set found_first_monitor flag >> - print info about current monitor including >> the lock_state >> >> A BasicLock is a Java monitor from the Java frame's point of >> view. It might be implemented as a stack lock, a biased lock >> or an inflated lock (ObjectMonitor). >> >> The problem that I see with the algorithm is that it is >> traversing a collection of BasicLocks and then asking >> lock implementation specific questions in a an effort to >> determine if the BasicLock is owned by the Java thread >> whose frames we are traversing. >> >> We already know that this code: >> >> mark->monitor() == thread()->current_pending_monitor() >> >> is specific to inflated monitors (ObjectMonitor) and it >> has a flaw because the target thread can be stopped at >> a safepoint before the _current_pending_monitor field >> is set. >> >> The proposed code: >> >> mark->monitor()->owner() != thread() >> >> compares the owner of the inflated monitor to the Java >> thread whose frames we are traversing. The problem is >> that the ObjectMonitor::_owner field is not always a >> pointer to the JavaThread. Right after a stack lock >> is inflated, the ObjectMonitor::_owner field is set >> to the mark->locker() stack address; see: >> >> src/share/vm/runtime/synchronizer.cpp: >> line 1283: m->set_owner(mark->locker()); >> >> So the proposed code will report "waiting to lock" for a >> Java monitor where the monitor is just transitioning >> from a stack lock to an inflated lock. Of course, the >> race is even tighter in this case. The target Java thread >> (T1) has to have just acquired the uninflated monitor >> and still be in the same frame in which that monitor was >> acquired at the same time that another thread (T2) lost >> the race to acquire the monitor. So T2 has to have just >> inflated the monitor, T1 has still be in the same frame >> in which it acquired the monitor and someone has to ask >> for a thread dump. Pretty cool. >> >> Another thought has occurred to me. T1 owns the monitor, >> T2 is contending for the monitor and is starting to inflate >> it, we request a thread dump. Is it possible to see the >> BasicLock in T2's Java frame, see that the Object does not >> yet have an inflated monitor (!mark->has_monitor()) and >> report that T2 has the monitor "locked"? >> >> This last scenario would again result in more than one >> thread being reported as having the same monitor locked. >> >> To sum up: >> >> I think both the current code and the proposed code have >> issues and we need to look at this a different way. We >> probably something like: >> >> MonitorInfo::is_locked_by_thread(JavaThread jt) >> >> that encapsulates and isolates the logic needed to >> determine if JavaThread 'jt' owns the BasicLock. It >> will need to know how to check: >> >> - stack lock ownership (see JavaThread::is_lock_owned()) >> - biased lock ownership (don't know how to do this one) >> - inflated lock ownership (_owner field == jt) >> >> The stack lock ownership check needs to be done before >> the inflated lock ownership check. That's because the >> _owner field in the inflated lock can be the >> mark->locker() stack address when it is initially >> inflated. >> >> I couldn't figure out the biased locking ownership >> check quickly so I don't know where it fits in the >> checking ownership pecking order. >> >> >> Sorry this turned out to be not so simple... Welcome to >> the joy of locking... >> >> Dan >> >> P.S. >> Of course, now that we've discovered at least one >> misuse of current_pending_monitor(), we should probably >> audit the other uses of it also... >> >> >> >>> >>> Thanks, >>> >>> -Zhengyu >> From david.holmes at oracle.com Tue May 20 05:28:12 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 May 2014 15:28:12 +1000 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537AE277.6050808@oracle.com> References: <537A0E1C.4070605@oracle.com> <537A9802.70707@oracle.com> <537AB953.7080906@oracle.com> <537AE277.6050808@oracle.com> Message-ID: <537AE7EC.1080906@oracle.com> Hi Dan, On 20/05/2014 3:04 PM, Daniel D. Daugherty wrote: > David, > > I'll have to politely disagree... That's why this code is so much fun ;-) You are of course correct. I misread this part of the inflate() code: m->set_owner(mark->locker()); to be setting the thread id as the owner, but in fact it is the BasicLock address as you note. As I flagged with Zhengyu in email, and as per the July 2012 discussion Kris Mok referred back to, I was wondering why this code did not do the "obvious" thing - and indeed why no one suggested checking the owner directly when I made the check_pending_monitor change back in 2006. Now I know. :) However I don't think we need to be concerned about BiasedLocking because that goes away as soon as we inflate - and we have inflated. So _owner is either a Thread ID or a BasicLock address. But even so isn't the check: mark->monitor()->owner() != thread() perfectly valid because a BasicLock address will never match a thread pointer? Thanks, David > Here's the code in ObjectMonitor::exit() that handles the case > when a thread finds itself exiting a monitor that was stack > locked when it entered, but is now inflated: > > src/share/vm/runtime/objectMonitor.cpp: > > 912 void ATTR ObjectMonitor::exit(bool not_suspended, TRAPS) { > 913 Thread * Self = THREAD ; > 914 if (THREAD != _owner) { > 915 if (THREAD->is_lock_owned((address) _owner)) { > 916 // Transmute _owner from a BasicLock pointer to a Thread > address > . > 917 // We don't need to hold _mutex for this transition. > 918 // Non-null to Non-null is safe as long as all readers can > 919 // tolerate either flavor. > 920 assert (_recursions == 0, "invariant") ; > 921 _owner = THREAD ; > 922 _recursions = 0 ; > 923 OwnerIsThread = 1 ; > 924 } else { > 925 // NOTE: we need to handle unbalanced monitor enter/exit > 926 // in native code by throwing an exception. > 927 // TODO: Throw an IllegalMonitorStateException ? > 928 TEVENT (Exit - Throw IMSX) ; > 929 assert(false, "Non-balanced monitor enter/exit!"); > 930 if (false) { > 931 THROW(vmSymbols::java_lang_IllegalMonitorStateException()); > 932 } > 933 return; > 934 } > 935 } > > so that _owner field when we did the stack dump still contained > the BaseLock pointer instead of the JavaThread pointer... > > Dan > > > On 5/19/14 8:09 PM, David Holmes wrote: >> Dan, >> >> My understanding of the existing code and hence the modified code is >> that we have already established that we are dealing with an inflated >> monitor and hence the owner really must be the owning thread - not a >> stack address for a BasicLock. Isn't that what mark->has_monitor() >> establishes? >> >> Note that we can not have reached a safepoint inflation is in progress. >> >> Thanks, >> David >> >> On 20/05/2014 9:47 AM, Daniel D. Daugherty wrote: >>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>> This is a simple fix for incorrect lock state. >>>> >>>> The timing on setting thread's pending monitor can result stack trace >>>> dump reporting incorrect lock state. The solution is to check the >>>> monitor's owner, if the owner is other than the current thread, then >>>> the monitor, is or is in process of being, set the pending monitor of >>>> current thread. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> src/share/vm/runtime/vframe.cpp >>> First off, I concur that the existing code has a problem. >>> However, I think the proposed solution has at least one >>> different problem. >>> >>> Here's the original code: >>> >>> 183 // Print out all monitors that we have locked or are trying >>> to lock >>> 184 GrowableArray* mons = monitors(); >>> 185 if (!mons->is_empty()) { >>> 186 bool found_first_monitor = false; >>> 187 for (int index = (mons->length()-1); index >= 0; index--) { >>> 188 MonitorInfo* monitor = mons->at(index); >>> 189 if (monitor->eliminated() && is_compiled_frame()) { // >>> Eliminated in compiled code >>> 190 if (monitor->owner_is_scalar_replaced()) { >>> 191 Klass* k = >>> java_lang_Class::as_Klass(monitor->owner_klass()); >>> 192 st->print("\t- eliminated (a >>> %s)", k->external_name()); >>> 193 } else { >>> 194 oop obj = monitor->owner(); >>> 195 if (obj != NULL) { >>> 196 print_locked_object_class_name(st, obj, "eliminated"); >>> 197 } >>> 198 } >>> 199 continue; >>> 200 } >>> 201 if (monitor->owner() != NULL) { >>> 202 >>> 203 // First, assume we have the monitor locked. If we haven't >>> found an >>> 204 // owned monitor before and this is the first frame, then >>> we need to >>> 205 // see if we have completed the lock or we are blocked >>> trying to >>> 206 // acquire it - we can only be blocked if the monitor is >>> inflated >>> 207 >>> 208 const char *lock_state = "locked"; // assume we have the >>> monitor locked >>> 209 if (!found_first_monitor && frame_count == 0) { >>> 210 markOop mark = monitor->owner()->mark(); >>> 211 if (mark->has_monitor() && >>> 212 mark->monitor() == >>> thread()->current_pending_monitor()) { >>> 213 lock_state = "waiting to lock"; >>> 214 } >>> 215 } >>> 216 >>> 217 found_first_monitor = true; >>> 218 print_locked_object_class_name(st, monitor->owner(), >>> lock_state); >>> 219 } >>> 220 } >>> 221 } >>> >>> The algorithm is pretty basic (no pun intended): >>> >>> - for each monitor (BasicLock) in the Java frame >>> - if the monitor has been eliminated and this is >>> a compiled frame >>> - do compiler specific handling >>> - continue >>> - if the monitor has a owner >>> - set lock_state = "locked" >>> - if this is the first monitor and the top most frame >>> - set mark = associated Object's mark >>> - if the mark has an ObjectMonitor and >>> the Java thread is pending on that monitor >>> - set lock_state = "waiting to lock" >>> - set found_first_monitor flag >>> - print info about current monitor including >>> the lock_state >>> >>> A BasicLock is a Java monitor from the Java frame's point of >>> view. It might be implemented as a stack lock, a biased lock >>> or an inflated lock (ObjectMonitor). >>> >>> The problem that I see with the algorithm is that it is >>> traversing a collection of BasicLocks and then asking >>> lock implementation specific questions in a an effort to >>> determine if the BasicLock is owned by the Java thread >>> whose frames we are traversing. >>> >>> We already know that this code: >>> >>> mark->monitor() == thread()->current_pending_monitor() >>> >>> is specific to inflated monitors (ObjectMonitor) and it >>> has a flaw because the target thread can be stopped at >>> a safepoint before the _current_pending_monitor field >>> is set. >>> >>> The proposed code: >>> >>> mark->monitor()->owner() != thread() >>> >>> compares the owner of the inflated monitor to the Java >>> thread whose frames we are traversing. The problem is >>> that the ObjectMonitor::_owner field is not always a >>> pointer to the JavaThread. Right after a stack lock >>> is inflated, the ObjectMonitor::_owner field is set >>> to the mark->locker() stack address; see: >>> >>> src/share/vm/runtime/synchronizer.cpp: >>> line 1283: m->set_owner(mark->locker()); >>> >>> So the proposed code will report "waiting to lock" for a >>> Java monitor where the monitor is just transitioning >>> from a stack lock to an inflated lock. Of course, the >>> race is even tighter in this case. The target Java thread >>> (T1) has to have just acquired the uninflated monitor >>> and still be in the same frame in which that monitor was >>> acquired at the same time that another thread (T2) lost >>> the race to acquire the monitor. So T2 has to have just >>> inflated the monitor, T1 has still be in the same frame >>> in which it acquired the monitor and someone has to ask >>> for a thread dump. Pretty cool. >>> >>> Another thought has occurred to me. T1 owns the monitor, >>> T2 is contending for the monitor and is starting to inflate >>> it, we request a thread dump. Is it possible to see the >>> BasicLock in T2's Java frame, see that the Object does not >>> yet have an inflated monitor (!mark->has_monitor()) and >>> report that T2 has the monitor "locked"? >>> >>> This last scenario would again result in more than one >>> thread being reported as having the same monitor locked. >>> >>> To sum up: >>> >>> I think both the current code and the proposed code have >>> issues and we need to look at this a different way. We >>> probably something like: >>> >>> MonitorInfo::is_locked_by_thread(JavaThread jt) >>> >>> that encapsulates and isolates the logic needed to >>> determine if JavaThread 'jt' owns the BasicLock. It >>> will need to know how to check: >>> >>> - stack lock ownership (see JavaThread::is_lock_owned()) >>> - biased lock ownership (don't know how to do this one) >>> - inflated lock ownership (_owner field == jt) >>> >>> The stack lock ownership check needs to be done before >>> the inflated lock ownership check. That's because the >>> _owner field in the inflated lock can be the >>> mark->locker() stack address when it is initially >>> inflated. >>> >>> I couldn't figure out the biased locking ownership >>> check quickly so I don't know where it fits in the >>> checking ownership pecking order. >>> >>> >>> Sorry this turned out to be not so simple... Welcome to >>> the joy of locking... >>> >>> Dan >>> >>> P.S. >>> Of course, now that we've discovered at least one >>> misuse of current_pending_monitor(), we should probably >>> audit the other uses of it also... >>> >>> >>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>> > From goetz.lindenmaier at sap.com Tue May 20 07:32:40 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 20 May 2014 07:32:40 +0000 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <21CE3D5B-9E7D-4188-BF14-DC097F4C780C@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> <537A20B3.1040104@oracle.com> <7E3E56CF-46B6-4869-97E0-D3D014844DF6@oracle.com> <21CE3D5B-9E7D-4188-BF14-DC097F4C780C@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC3C36@DEWDFEMB12A.global.corp.sap> Hi Christian, I added the Reviewed-by line to the changeset I uploaded, so it jchecks fine. Is this what you needed? Can I prepare that some more? I tested the changeset on linuxx86_64, windowsx86_64, solaris sparc 64 & 32 bit, darwinintel64, linuxppc64 and aix. Opt & fastdbg build on all (without precompiled headers) and dbg build on most of the platforms. Thanks for your help, Goetz. -----Original Message----- From: Christian Thalinger [mailto:christian.thalinger at oracle.com] Sent: Montag, 19. Mai 2014 18:50 To: Stefan Karlsson Cc: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp Actually, since you are a Commiter can you prepare a changeset so I can just take and push it? Saves me some time :-) On May 19, 2014, at 9:48 AM, Christian Thalinger wrote: > Alright. I will push this to hs-comp. > > On May 19, 2014, at 8:18 AM, Stefan Karlsson wrote: > >> On 2014-05-19 16:43, Lindenmaier, Goetz wrote: >>> Hi Stefan, >>> >>> thanks for the comments! >>> I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. >>> I also saw another usage of Prefetch in klass.hpp. I removed >>> the macro altogether as it's unused. >>> >>> I kept the include in precompiled.hpp though, as for other >>> files it's listed there, too. If you insist, and >>> there are no other objections, I'll remove it. >> >> OK. >> >>> >>> The new webrev is at >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 >> >> Looks good. >> >> thanks, >> StefanK >>> >>> Best regards, >>> Goetz. >>> >>> -----Original Message----- >>> From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] >>> Sent: Montag, 19. Mai 2014 13:21 >>> To: Lindenmaier, Goetz; Christian Thalinger >>> Cc: hotspot-dev at openjdk.java.net >>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>> >>> Hi Goetz, >>> >>> On 2014-05-19 12:58, Lindenmaier, Goetz wrote: >>>> Hi Christian, >>>> >>>> thanks for the review! >>>> >>>> Although prefetch is not used that often, I think it's good >>>> that all headers in the os_cpu directories are handled >>>> similarly now, i.e., they are only included in the respective >>>> shared file. (For some it's the file in the /cpu/ dir, like >>>> copy_linux_x86.inline.hpp.) >>>> >>>> Could you sponsor the change? Do I need a second reviewer? >>> Thanks for cleaning this up. >>> >>> Some minor comments: >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html >>> >>> You shouldn't have to include prefetch.inline.hpp from this file. >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html >>> >>> heapRegion.cpp needs to include space.inline.hpp since it uses the >>> SCAN_AND_FORWARD macro. >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html >>> >>> I don't think its necessary to add the prefetch.inline.hpp to >>> precompiled.hpp. >>> >>> >>> Otherwise, this looks good to me. >>> >>> thanks, >>> StefanK >>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >>>> Sent: Donnerstag, 15. Mai 2014 17:14 >>>> To: Lindenmaier, Goetz >>>> Cc: hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>> >>>> Looks good to me. I thought that prefetch is used more often but apparently not. >>>> >>>> On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: >>>> >>>>> Hi, >>>>> >>>>> This change introduces an umbrella header for prefetch__.inline.hpp >>>>> files: runtime/prefetch.inline.hpp as proposed in >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >>>>> This follows the scheme applied to atomic.inline.hpp. >>>>> >>>>> Further this change adds includes of prefetch.inline.hpp in all .cpp >>>>> and .inline.hpp files where a method of Prefetch declared 'inline' is >>>>> called. >>>>> >>>>> Finally it moves macros calling inline methods of Prefetch from >>>>> space.hpp to the corresponding .inline.hpp file and adds the necessary >>>>> includes in files using the moved macros. >>>>> >>>>> Please review and test this change. I please need a sponsor. >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >>>>> >>>>> Thanks and best regards, >>>>> Goetz. >> > From thomas.stuefe at gmail.com Tue May 20 07:59:25 2014 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 20 May 2014 09:59:25 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: <5378D1EE.3020309@oracle.com> References: <5375D08C.9070206@oracle.com> <5378D1EE.3020309@oracle.com> Message-ID: Hi all, I like you proposal and hope this tracing system turns out well. I'm with the SAP JVM; we have had our own homebrewn tracing system for years, which resembles your proposal a lot. If your solution turns out well, we may decide to abandon our propietary solution in favor of your proposal. Here are some aspects we found important over the last years with our tracing system: 1) tracing during initialization phase. When tracing in code which gets executed very early it would be nice to have a solution which works right from the start. So, argument parsing for the tracing system should happen as early as possible, or not rely only on command line arguments. Often I found myself tracing initialization code only to find that my output was not visible, so there was always the question "did I not hit my code or was the tracing system not yet initialized?" - in the end, due to time, one often falls back to fprintf. 2) Tracing system should be implemented as "lowest VM layer", without dependencies to VM infrastructure This one is clear and also very important. It also affects all adapters. For instance, tracing system should not use os::malloc, because I might want to trace os::malloc. So, it should be implemented as an independend module, with no dependencies to other infrastructure. This is also related to (1), as it makes it possible to trace right from VM start without having to wait for VM initialization 3) inactive trace points should not incurr costs In our tracing system, we have a flat vector of "loggers" which are controlled by a global bit array; this means that at runtime, any decision to actually trace costs one memory access to a memory location which hopefully is already cached. Simple and stupid, but it works, and costs to not trace are at a minimum. In your proposal I would have to call "Log::log()". The logger class has to be resolved, and its state queried, which, depending on how this is implemented, may cause more than one memory accesses. 4) Adapters This whole thing seems a bit too complicated. - Do we really need the ability to attach different adapters to different Loggers? Would one adapter for the whole VM not suffice? - Do we really need the ability to chain adapters, especially chain different file loggers? This seems to me like something the OS does far better than the VM. This flexibility would cause a lot of testing effort, and you never can be sure that you tested all possible combinations. Also timing: different adapters may show different timing behaviour, and if different loggers may cause different adapters or combination of adapters, timing of different trace spots in respect to each other may differ greatly. I am also concerned about the reliability of adapters. In my experience, the tracing backends are a weak spot in a tracing system, especially if one allows other teams to extend them by writing new adapters. 5) Just cosmetics: I would prefer the logging calls to be simpler. Instead of writing Log::log(Log::vm_init(), LogLevel::info(), "Init took %lu ms to complete.", time); I would prefer something like logI(vm_init, "Init took %lu ms to complete.", time); or for an error: logE(vm_init, "All is lost."); Some minor proposals: 6) Shared memory adapter I would propose another adaptor (beside stdout and file), which writes to a preallocated large shared memory segment; overwriting old entries if full. That shared memory segment could be read from the outside with a command line tool. (Arguably the same could be done with memory file systems, in an OS dependend way). 7) LogMark Log& Log::().begin_transaction(); bool Log::().commit_transaction(); could be wrapped into something like a LogMark, similar to a ResourceMark, to end the transaction when the stack unwinds. Kind Regards Thomas St?fe On 18 May 2014 17:29, Fredrik Arvidsson wrote: > Hi Chris > > You will find my comments in-line below. > > > On 2014-05-17 09:47, Chris Newland wrote: > >> Hi Fredrik, >> >> The discussion I had with David Holmes and John Rose on hotspot-dev back >> in February might be relevant to this JEP: >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014- >> February/012718.html >> > I will use some time to read this discussion. Thanks. > > I'm the author of a JITWatch[1], a JIT compiler log analysis tool which >> relies on -XX:+TraceClassLoading -XX:+LogCompilation -XX:+PrintAssembly >> all being present in hotspot.log >> >> Occasionally the lack of thread safety in the VM logging presents me with >> jumbled output: >> >> [Entry Point] >> [Constants] >> # {method} 'ind >> >> [Loaded sun.nio.cs.ThreadLocalCoders from C:\Program >> Files\Java\jre7\lib\rt.jar] >> >> exOf' '(II)I' in 'java/lang/String' >> # this: ecx = 'java/lang/String' >> # parm0: edx = int >> # parm1: [sp+0x20] = int (sp of caller) >> 0x009e07e0: nop >> >> The JEP goal of no interleaving would fix this problem but if the locking >> adds too much cost/complexity then John's suggestion of tagging partial >> lines would be a great help to log tool writers. >> > There could be several reasons for the current logs to be interleaved I > think. One reason can be that the lowest level log writing primitives > aren't thread safe themselves. > A second reason can be that the current log framework doesn?t provide any > form of log transactions making it easy for developer to implement the > logging in a way that guarantees non partial lines in the log. > JEP-158 tries to address both of these reasons. Thread safety issues can > be addressed using the leanest locking/critical section technique we can > have. This might have different solutions depending on platform and log > adapter in question. The 'broken up log code' is addressed with our log > transaction which enables the developer to split the actual line of logging > in to several calls to the log API. The log line is then committed to the > underlying framework to be written all at once. > > >> Otherwise my only comment is that the goals look like a massive >> improvement to current VM logging but for post-execution log tool writers >> it is also useful for end users to be able to easily locate and load a >> single log file. >> > This really captures the intent of JEP-158 I think. We want to have a > common framework for logging so that all output can be formatted the same > well defined way, and we want to be able to have a single entry of > configuration letting users configuring all logging in a consolidated and > clear way. > > Kind regards, >> >> Chris >> >> [1] https://github.com/AdoptOpenJDK/jitwatch >> >> Cheers > > /Fredrik > From david.holmes at oracle.com Tue May 20 09:24:22 2014 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 May 2014 19:24:22 +1000 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537AE7EC.1080906@oracle.com> References: <537A0E1C.4070605@oracle.com> <537A9802.70707@oracle.com> <537AB953.7080906@oracle.com> <537AE277.6050808@oracle.com> <537AE7EC.1080906@oracle.com> Message-ID: <537B1F46.605@oracle.com> On 20/05/2014 3:28 PM, David Holmes wrote: > Hi Dan, > > On 20/05/2014 3:04 PM, Daniel D. Daugherty wrote: >> David, >> >> I'll have to politely disagree... > > That's why this code is so much fun ;-) > > You are of course correct. I misread this part of the inflate() code: > > m->set_owner(mark->locker()); > > to be setting the thread id as the owner, but in fact it is the > BasicLock address as you note. > > As I flagged with Zhengyu in email, and as per the July 2012 discussion > Kris Mok referred back to, I was wondering why this code did not do the > "obvious" thing - and indeed why no one suggested checking the owner > directly when I made the check_pending_monitor change back in 2006. Now > I know. :) > > However I don't think we need to be concerned about BiasedLocking > because that goes away as soon as we inflate - and we have inflated. So > _owner is either a Thread ID or a BasicLock address. > > But even so isn't the check: > > mark->monitor()->owner() != thread() > > perfectly valid because a BasicLock address will never match a thread > pointer? Ah but the thread being examined may in fact be the owner that has this monitor stack-locked - which Dan also pointed out in his email. So we do need to do additional checking for the BasicLock case. David > Thanks, > David > >> Here's the code in ObjectMonitor::exit() that handles the case >> when a thread finds itself exiting a monitor that was stack >> locked when it entered, but is now inflated: >> >> src/share/vm/runtime/objectMonitor.cpp: >> >> 912 void ATTR ObjectMonitor::exit(bool not_suspended, TRAPS) { >> 913 Thread * Self = THREAD ; >> 914 if (THREAD != _owner) { >> 915 if (THREAD->is_lock_owned((address) _owner)) { >> 916 // Transmute _owner from a BasicLock pointer to a Thread >> address >> . >> 917 // We don't need to hold _mutex for this transition. >> 918 // Non-null to Non-null is safe as long as all readers >> can >> 919 // tolerate either flavor. >> 920 assert (_recursions == 0, "invariant") ; >> 921 _owner = THREAD ; >> 922 _recursions = 0 ; >> 923 OwnerIsThread = 1 ; >> 924 } else { >> 925 // NOTE: we need to handle unbalanced monitor enter/exit >> 926 // in native code by throwing an exception. >> 927 // TODO: Throw an IllegalMonitorStateException ? >> 928 TEVENT (Exit - Throw IMSX) ; >> 929 assert(false, "Non-balanced monitor enter/exit!"); >> 930 if (false) { >> 931 THROW(vmSymbols::java_lang_IllegalMonitorStateException()); >> 932 } >> 933 return; >> 934 } >> 935 } >> >> so that _owner field when we did the stack dump still contained >> the BaseLock pointer instead of the JavaThread pointer... >> >> Dan >> >> >> On 5/19/14 8:09 PM, David Holmes wrote: >>> Dan, >>> >>> My understanding of the existing code and hence the modified code is >>> that we have already established that we are dealing with an inflated >>> monitor and hence the owner really must be the owning thread - not a >>> stack address for a BasicLock. Isn't that what mark->has_monitor() >>> establishes? >>> >>> Note that we can not have reached a safepoint inflation is in progress. >>> >>> Thanks, >>> David >>> >>> On 20/05/2014 9:47 AM, Daniel D. Daugherty wrote: >>>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>>> This is a simple fix for incorrect lock state. >>>>> >>>>> The timing on setting thread's pending monitor can result stack trace >>>>> dump reporting incorrect lock state. The solution is to check the >>>>> monitor's owner, if the owner is other than the current thread, then >>>>> the monitor, is or is in process of being, set the pending monitor of >>>>> current thread. >>>>> >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>> >>>> src/share/vm/runtime/vframe.cpp >>>> First off, I concur that the existing code has a problem. >>>> However, I think the proposed solution has at least one >>>> different problem. >>>> >>>> Here's the original code: >>>> >>>> 183 // Print out all monitors that we have locked or are trying >>>> to lock >>>> 184 GrowableArray* mons = monitors(); >>>> 185 if (!mons->is_empty()) { >>>> 186 bool found_first_monitor = false; >>>> 187 for (int index = (mons->length()-1); index >= 0; index--) { >>>> 188 MonitorInfo* monitor = mons->at(index); >>>> 189 if (monitor->eliminated() && is_compiled_frame()) { // >>>> Eliminated in compiled code >>>> 190 if (monitor->owner_is_scalar_replaced()) { >>>> 191 Klass* k = >>>> java_lang_Class::as_Klass(monitor->owner_klass()); >>>> 192 st->print("\t- eliminated (a >>>> %s)", k->external_name()); >>>> 193 } else { >>>> 194 oop obj = monitor->owner(); >>>> 195 if (obj != NULL) { >>>> 196 print_locked_object_class_name(st, obj, >>>> "eliminated"); >>>> 197 } >>>> 198 } >>>> 199 continue; >>>> 200 } >>>> 201 if (monitor->owner() != NULL) { >>>> 202 >>>> 203 // First, assume we have the monitor locked. If we >>>> haven't >>>> found an >>>> 204 // owned monitor before and this is the first frame, then >>>> we need to >>>> 205 // see if we have completed the lock or we are blocked >>>> trying to >>>> 206 // acquire it - we can only be blocked if the monitor is >>>> inflated >>>> 207 >>>> 208 const char *lock_state = "locked"; // assume we have the >>>> monitor locked >>>> 209 if (!found_first_monitor && frame_count == 0) { >>>> 210 markOop mark = monitor->owner()->mark(); >>>> 211 if (mark->has_monitor() && >>>> 212 mark->monitor() == >>>> thread()->current_pending_monitor()) { >>>> 213 lock_state = "waiting to lock"; >>>> 214 } >>>> 215 } >>>> 216 >>>> 217 found_first_monitor = true; >>>> 218 print_locked_object_class_name(st, monitor->owner(), >>>> lock_state); >>>> 219 } >>>> 220 } >>>> 221 } >>>> >>>> The algorithm is pretty basic (no pun intended): >>>> >>>> - for each monitor (BasicLock) in the Java frame >>>> - if the monitor has been eliminated and this is >>>> a compiled frame >>>> - do compiler specific handling >>>> - continue >>>> - if the monitor has a owner >>>> - set lock_state = "locked" >>>> - if this is the first monitor and the top most frame >>>> - set mark = associated Object's mark >>>> - if the mark has an ObjectMonitor and >>>> the Java thread is pending on that monitor >>>> - set lock_state = "waiting to lock" >>>> - set found_first_monitor flag >>>> - print info about current monitor including >>>> the lock_state >>>> >>>> A BasicLock is a Java monitor from the Java frame's point of >>>> view. It might be implemented as a stack lock, a biased lock >>>> or an inflated lock (ObjectMonitor). >>>> >>>> The problem that I see with the algorithm is that it is >>>> traversing a collection of BasicLocks and then asking >>>> lock implementation specific questions in a an effort to >>>> determine if the BasicLock is owned by the Java thread >>>> whose frames we are traversing. >>>> >>>> We already know that this code: >>>> >>>> mark->monitor() == thread()->current_pending_monitor() >>>> >>>> is specific to inflated monitors (ObjectMonitor) and it >>>> has a flaw because the target thread can be stopped at >>>> a safepoint before the _current_pending_monitor field >>>> is set. >>>> >>>> The proposed code: >>>> >>>> mark->monitor()->owner() != thread() >>>> >>>> compares the owner of the inflated monitor to the Java >>>> thread whose frames we are traversing. The problem is >>>> that the ObjectMonitor::_owner field is not always a >>>> pointer to the JavaThread. Right after a stack lock >>>> is inflated, the ObjectMonitor::_owner field is set >>>> to the mark->locker() stack address; see: >>>> >>>> src/share/vm/runtime/synchronizer.cpp: >>>> line 1283: m->set_owner(mark->locker()); >>>> >>>> So the proposed code will report "waiting to lock" for a >>>> Java monitor where the monitor is just transitioning >>>> from a stack lock to an inflated lock. Of course, the >>>> race is even tighter in this case. The target Java thread >>>> (T1) has to have just acquired the uninflated monitor >>>> and still be in the same frame in which that monitor was >>>> acquired at the same time that another thread (T2) lost >>>> the race to acquire the monitor. So T2 has to have just >>>> inflated the monitor, T1 has still be in the same frame >>>> in which it acquired the monitor and someone has to ask >>>> for a thread dump. Pretty cool. >>>> >>>> Another thought has occurred to me. T1 owns the monitor, >>>> T2 is contending for the monitor and is starting to inflate >>>> it, we request a thread dump. Is it possible to see the >>>> BasicLock in T2's Java frame, see that the Object does not >>>> yet have an inflated monitor (!mark->has_monitor()) and >>>> report that T2 has the monitor "locked"? >>>> >>>> This last scenario would again result in more than one >>>> thread being reported as having the same monitor locked. >>>> >>>> To sum up: >>>> >>>> I think both the current code and the proposed code have >>>> issues and we need to look at this a different way. We >>>> probably something like: >>>> >>>> MonitorInfo::is_locked_by_thread(JavaThread jt) >>>> >>>> that encapsulates and isolates the logic needed to >>>> determine if JavaThread 'jt' owns the BasicLock. It >>>> will need to know how to check: >>>> >>>> - stack lock ownership (see JavaThread::is_lock_owned()) >>>> - biased lock ownership (don't know how to do this one) >>>> - inflated lock ownership (_owner field == jt) >>>> >>>> The stack lock ownership check needs to be done before >>>> the inflated lock ownership check. That's because the >>>> _owner field in the inflated lock can be the >>>> mark->locker() stack address when it is initially >>>> inflated. >>>> >>>> I couldn't figure out the biased locking ownership >>>> check quickly so I don't know where it fits in the >>>> checking ownership pecking order. >>>> >>>> >>>> Sorry this turned out to be not so simple... Welcome to >>>> the joy of locking... >>>> >>>> Dan >>>> >>>> P.S. >>>> Of course, now that we've discovered at least one >>>> misuse of current_pending_monitor(), we should probably >>>> audit the other uses of it also... >>>> >>>> >>>> >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>> >> From daniel.daugherty at oracle.com Tue May 20 12:47:04 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 20 May 2014 06:47:04 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537AB9EE.4050601@oracle.com> References: <537A0E1C.4070605@oracle.com> <537AB9EE.4050601@oracle.com> Message-ID: <537B4EC8.70706@oracle.com> Looks like there were two e-mails threads along these lines: Subject: Are there any liveness issues about thread scheduling in the JVM? Subject: Multiple Java threads seem to have locked the same object monitor from the thread dumps The first thread started on 2012.07.10 and ended on 2012.07.16. The second thread started on 2012.07.12 and ended the same day. Both threads ended when the topic evolved to reproducers. Dan On 5/19/14 8:11 PM, David Holmes wrote: > Kris, > > This does indeed appear to be the same issue. I don't recall why that > discussion in July 2012 just came to a halt. :( > > Sorry about that. > > David > > On 20/05/2014 8:00 AM, Krystal Mok wrote: >> Hi Zhengyu, >> >> I just glanced over your changes, and the description looks like >> something >> I've seen in the past: >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2012-July/004106.html >> >> >> Could you please take a look at the old discussion thread and see if >> they're the same thing? >> >> Thanks, >> Kris >> >> >> On Mon, May 19, 2014 at 6:58 AM, Zhengyu Gu >> wrote: >> >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack >>> trace dump >>> reporting incorrect lock state. The solution is to check the monitor's >>> owner, if the owner is other than the current thread, then the >>> monitor, is >>> or is in process of being, set the pending monitor of current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> From zhengyu.gu at oracle.com Tue May 20 13:08:57 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Tue, 20 May 2014 09:08:57 -0400 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: References: <537A0E1C.4070605@oracle.com> Message-ID: <537B53E9.3030903@oracle.com> Hi Kris, Yes, they are the same things. Please see my early comment in the bug report. Thanks, -Zhengyu On 5/19/2014 6:00 PM, Krystal Mok wrote: > Hi Zhengyu, > > I just glanced over your changes, and the description looks like > something I've seen in the past: > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2012-July/004106.html > > Could you please take a look at the old discussion thread and see if > they're the same thing? > > Thanks, > Kris > > > On Mon, May 19, 2014 at 6:58 AM, Zhengyu Gu > wrote: > > This is a simple fix for incorrect lock state. > > The timing on setting thread's pending monitor can result stack > trace dump reporting incorrect lock state. The solution is to > check the monitor's owner, if the owner is other than the current > thread, then the monitor, is or is in process of being, set the > pending monitor of current thread. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 > Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ > > > > Thanks, > > -Zhengyu > > From zhengyu.gu at oracle.com Tue May 20 13:11:31 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Tue, 20 May 2014 09:11:31 -0400 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537B1F46.605@oracle.com> References: <537A0E1C.4070605@oracle.com> <537A9802.70707@oracle.com> <537AB953.7080906@oracle.com> <537AE277.6050808@oracle.com> <537AE7EC.1080906@oracle.com> <537B1F46.605@oracle.com> Message-ID: <537B5483.3000105@oracle.com> Thanks Dan and David. At this point, I would like withdraw this code review request. -Zhengyu On 5/20/2014 5:24 AM, David Holmes wrote: > > > On 20/05/2014 3:28 PM, David Holmes wrote: >> Hi Dan, >> >> On 20/05/2014 3:04 PM, Daniel D. Daugherty wrote: >>> David, >>> >>> I'll have to politely disagree... >> >> That's why this code is so much fun ;-) >> >> You are of course correct. I misread this part of the inflate() code: >> >> m->set_owner(mark->locker()); >> >> to be setting the thread id as the owner, but in fact it is the >> BasicLock address as you note. >> >> As I flagged with Zhengyu in email, and as per the July 2012 discussion >> Kris Mok referred back to, I was wondering why this code did not do the >> "obvious" thing - and indeed why no one suggested checking the owner >> directly when I made the check_pending_monitor change back in 2006. Now >> I know. :) >> >> However I don't think we need to be concerned about BiasedLocking >> because that goes away as soon as we inflate - and we have inflated. So >> _owner is either a Thread ID or a BasicLock address. >> >> But even so isn't the check: >> >> mark->monitor()->owner() != thread() >> >> perfectly valid because a BasicLock address will never match a thread >> pointer? > > Ah but the thread being examined may in fact be the owner that has > this monitor stack-locked - which Dan also pointed out in his email. > So we do need to do additional checking for the BasicLock case. > > David > >> Thanks, >> David >> >>> Here's the code in ObjectMonitor::exit() that handles the case >>> when a thread finds itself exiting a monitor that was stack >>> locked when it entered, but is now inflated: >>> >>> src/share/vm/runtime/objectMonitor.cpp: >>> >>> 912 void ATTR ObjectMonitor::exit(bool not_suspended, TRAPS) { >>> 913 Thread * Self = THREAD ; >>> 914 if (THREAD != _owner) { >>> 915 if (THREAD->is_lock_owned((address) _owner)) { >>> 916 // Transmute _owner from a BasicLock pointer to a >>> Thread >>> address >>> . >>> 917 // We don't need to hold _mutex for this transition. >>> 918 // Non-null to Non-null is safe as long as all readers >>> can >>> 919 // tolerate either flavor. >>> 920 assert (_recursions == 0, "invariant") ; >>> 921 _owner = THREAD ; >>> 922 _recursions = 0 ; >>> 923 OwnerIsThread = 1 ; >>> 924 } else { >>> 925 // NOTE: we need to handle unbalanced monitor >>> enter/exit >>> 926 // in native code by throwing an exception. >>> 927 // TODO: Throw an IllegalMonitorStateException ? >>> 928 TEVENT (Exit - Throw IMSX) ; >>> 929 assert(false, "Non-balanced monitor enter/exit!"); >>> 930 if (false) { >>> 931 THROW(vmSymbols::java_lang_IllegalMonitorStateException()); >>> 932 } >>> 933 return; >>> 934 } >>> 935 } >>> >>> so that _owner field when we did the stack dump still contained >>> the BaseLock pointer instead of the JavaThread pointer... >>> >>> Dan >>> >>> >>> On 5/19/14 8:09 PM, David Holmes wrote: >>>> Dan, >>>> >>>> My understanding of the existing code and hence the modified code is >>>> that we have already established that we are dealing with an inflated >>>> monitor and hence the owner really must be the owning thread - not a >>>> stack address for a BasicLock. Isn't that what mark->has_monitor() >>>> establishes? >>>> >>>> Note that we can not have reached a safepoint inflation is in >>>> progress. >>>> >>>> Thanks, >>>> David >>>> >>>> On 20/05/2014 9:47 AM, Daniel D. Daugherty wrote: >>>>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>>>> This is a simple fix for incorrect lock state. >>>>>> >>>>>> The timing on setting thread's pending monitor can result stack >>>>>> trace >>>>>> dump reporting incorrect lock state. The solution is to check the >>>>>> monitor's owner, if the owner is other than the current thread, then >>>>>> the monitor, is or is in process of being, set the pending >>>>>> monitor of >>>>>> current thread. >>>>>> >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>>> >>>>> src/share/vm/runtime/vframe.cpp >>>>> First off, I concur that the existing code has a problem. >>>>> However, I think the proposed solution has at least one >>>>> different problem. >>>>> >>>>> Here's the original code: >>>>> >>>>> 183 // Print out all monitors that we have locked or are trying >>>>> to lock >>>>> 184 GrowableArray* mons = monitors(); >>>>> 185 if (!mons->is_empty()) { >>>>> 186 bool found_first_monitor = false; >>>>> 187 for (int index = (mons->length()-1); index >= 0; index--) { >>>>> 188 MonitorInfo* monitor = mons->at(index); >>>>> 189 if (monitor->eliminated() && is_compiled_frame()) { // >>>>> Eliminated in compiled code >>>>> 190 if (monitor->owner_is_scalar_replaced()) { >>>>> 191 Klass* k = >>>>> java_lang_Class::as_Klass(monitor->owner_klass()); >>>>> 192 st->print("\t- eliminated >>>> replaced> (a >>>>> %s)", k->external_name()); >>>>> 193 } else { >>>>> 194 oop obj = monitor->owner(); >>>>> 195 if (obj != NULL) { >>>>> 196 print_locked_object_class_name(st, obj, >>>>> "eliminated"); >>>>> 197 } >>>>> 198 } >>>>> 199 continue; >>>>> 200 } >>>>> 201 if (monitor->owner() != NULL) { >>>>> 202 >>>>> 203 // First, assume we have the monitor locked. If we >>>>> haven't >>>>> found an >>>>> 204 // owned monitor before and this is the first frame, >>>>> then >>>>> we need to >>>>> 205 // see if we have completed the lock or we are blocked >>>>> trying to >>>>> 206 // acquire it - we can only be blocked if the >>>>> monitor is >>>>> inflated >>>>> 207 >>>>> 208 const char *lock_state = "locked"; // assume we have >>>>> the >>>>> monitor locked >>>>> 209 if (!found_first_monitor && frame_count == 0) { >>>>> 210 markOop mark = monitor->owner()->mark(); >>>>> 211 if (mark->has_monitor() && >>>>> 212 mark->monitor() == >>>>> thread()->current_pending_monitor()) { >>>>> 213 lock_state = "waiting to lock"; >>>>> 214 } >>>>> 215 } >>>>> 216 >>>>> 217 found_first_monitor = true; >>>>> 218 print_locked_object_class_name(st, monitor->owner(), >>>>> lock_state); >>>>> 219 } >>>>> 220 } >>>>> 221 } >>>>> >>>>> The algorithm is pretty basic (no pun intended): >>>>> >>>>> - for each monitor (BasicLock) in the Java frame >>>>> - if the monitor has been eliminated and this is >>>>> a compiled frame >>>>> - do compiler specific handling >>>>> - continue >>>>> - if the monitor has a owner >>>>> - set lock_state = "locked" >>>>> - if this is the first monitor and the top most frame >>>>> - set mark = associated Object's mark >>>>> - if the mark has an ObjectMonitor and >>>>> the Java thread is pending on that monitor >>>>> - set lock_state = "waiting to lock" >>>>> - set found_first_monitor flag >>>>> - print info about current monitor including >>>>> the lock_state >>>>> >>>>> A BasicLock is a Java monitor from the Java frame's point of >>>>> view. It might be implemented as a stack lock, a biased lock >>>>> or an inflated lock (ObjectMonitor). >>>>> >>>>> The problem that I see with the algorithm is that it is >>>>> traversing a collection of BasicLocks and then asking >>>>> lock implementation specific questions in a an effort to >>>>> determine if the BasicLock is owned by the Java thread >>>>> whose frames we are traversing. >>>>> >>>>> We already know that this code: >>>>> >>>>> mark->monitor() == thread()->current_pending_monitor() >>>>> >>>>> is specific to inflated monitors (ObjectMonitor) and it >>>>> has a flaw because the target thread can be stopped at >>>>> a safepoint before the _current_pending_monitor field >>>>> is set. >>>>> >>>>> The proposed code: >>>>> >>>>> mark->monitor()->owner() != thread() >>>>> >>>>> compares the owner of the inflated monitor to the Java >>>>> thread whose frames we are traversing. The problem is >>>>> that the ObjectMonitor::_owner field is not always a >>>>> pointer to the JavaThread. Right after a stack lock >>>>> is inflated, the ObjectMonitor::_owner field is set >>>>> to the mark->locker() stack address; see: >>>>> >>>>> src/share/vm/runtime/synchronizer.cpp: >>>>> line 1283: m->set_owner(mark->locker()); >>>>> >>>>> So the proposed code will report "waiting to lock" for a >>>>> Java monitor where the monitor is just transitioning >>>>> from a stack lock to an inflated lock. Of course, the >>>>> race is even tighter in this case. The target Java thread >>>>> (T1) has to have just acquired the uninflated monitor >>>>> and still be in the same frame in which that monitor was >>>>> acquired at the same time that another thread (T2) lost >>>>> the race to acquire the monitor. So T2 has to have just >>>>> inflated the monitor, T1 has still be in the same frame >>>>> in which it acquired the monitor and someone has to ask >>>>> for a thread dump. Pretty cool. >>>>> >>>>> Another thought has occurred to me. T1 owns the monitor, >>>>> T2 is contending for the monitor and is starting to inflate >>>>> it, we request a thread dump. Is it possible to see the >>>>> BasicLock in T2's Java frame, see that the Object does not >>>>> yet have an inflated monitor (!mark->has_monitor()) and >>>>> report that T2 has the monitor "locked"? >>>>> >>>>> This last scenario would again result in more than one >>>>> thread being reported as having the same monitor locked. >>>>> >>>>> To sum up: >>>>> >>>>> I think both the current code and the proposed code have >>>>> issues and we need to look at this a different way. We >>>>> probably something like: >>>>> >>>>> MonitorInfo::is_locked_by_thread(JavaThread jt) >>>>> >>>>> that encapsulates and isolates the logic needed to >>>>> determine if JavaThread 'jt' owns the BasicLock. It >>>>> will need to know how to check: >>>>> >>>>> - stack lock ownership (see JavaThread::is_lock_owned()) >>>>> - biased lock ownership (don't know how to do this one) >>>>> - inflated lock ownership (_owner field == jt) >>>>> >>>>> The stack lock ownership check needs to be done before >>>>> the inflated lock ownership check. That's because the >>>>> _owner field in the inflated lock can be the >>>>> mark->locker() stack address when it is initially >>>>> inflated. >>>>> >>>>> I couldn't figure out the biased locking ownership >>>>> check quickly so I don't know where it fits in the >>>>> checking ownership pecking order. >>>>> >>>>> >>>>> Sorry this turned out to be not so simple... Welcome to >>>>> the joy of locking... >>>>> >>>>> Dan >>>>> >>>>> P.S. >>>>> Of course, now that we've discovered at least one >>>>> misuse of current_pending_monitor(), we should probably >>>>> audit the other uses of it also... >>>>> >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -Zhengyu >>>>> >>> From christian.thalinger at oracle.com Tue May 20 15:16:32 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 20 May 2014 08:16:32 -0700 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEC3C36@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> <537A20B3.1040104@oracle.com> <7E3E56CF-46B6-4869-97E0-D3D014844DF6@oracle.com> <21CE3D5B-9E7D-4188-BF14-DC097F4C780C@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC3C36@DEWDFEMB12A.global.corp.sap> Message-ID: <949226D8-9228-4C8C-979C-3A51031981F9@oracle.com> On May 20, 2014, at 12:32 AM, Lindenmaier, Goetz wrote: > Hi Christian, > > I added the Reviewed-by line to the changeset I uploaded, so it jchecks fine. > Is this what you needed? Can I prepare that some more? Yes, that?s what I wanted. Thanks. > > I tested the changeset on linuxx86_64, windowsx86_64, solaris sparc 64 & 32 bit, > darwinintel64, linuxppc64 and aix. Opt & fastdbg build on all (without precompiled headers) > and dbg build on most of the platforms. > > Thanks for your help, > Goetz. > > > > -----Original Message----- > From: Christian Thalinger [mailto:christian.thalinger at oracle.com] > Sent: Montag, 19. Mai 2014 18:50 > To: Stefan Karlsson > Cc: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp > > Actually, since you are a Commiter can you prepare a changeset so I can just take and push it? Saves me some time :-) > > On May 19, 2014, at 9:48 AM, Christian Thalinger wrote: > >> Alright. I will push this to hs-comp. >> >> On May 19, 2014, at 8:18 AM, Stefan Karlsson wrote: >> >>> On 2014-05-19 16:43, Lindenmaier, Goetz wrote: >>>> Hi Stefan, >>>> >>>> thanks for the comments! >>>> I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. >>>> I also saw another usage of Prefetch in klass.hpp. I removed >>>> the macro altogether as it's unused. >>>> >>>> I kept the include in precompiled.hpp though, as for other >>>> files it's listed there, too. If you insist, and >>>> there are no other objections, I'll remove it. >>> >>> OK. >>> >>>> >>>> The new webrev is at >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 >>> >>> Looks good. >>> >>> thanks, >>> StefanK >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> -----Original Message----- >>>> From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] >>>> Sent: Montag, 19. Mai 2014 13:21 >>>> To: Lindenmaier, Goetz; Christian Thalinger >>>> Cc: hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>> >>>> Hi Goetz, >>>> >>>> On 2014-05-19 12:58, Lindenmaier, Goetz wrote: >>>>> Hi Christian, >>>>> >>>>> thanks for the review! >>>>> >>>>> Although prefetch is not used that often, I think it's good >>>>> that all headers in the os_cpu directories are handled >>>>> similarly now, i.e., they are only included in the respective >>>>> shared file. (For some it's the file in the /cpu/ dir, like >>>>> copy_linux_x86.inline.hpp.) >>>>> >>>>> Could you sponsor the change? Do I need a second reviewer? >>>> Thanks for cleaning this up. >>>> >>>> Some minor comments: >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html >>>> >>>> You shouldn't have to include prefetch.inline.hpp from this file. >>>> >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html >>>> >>>> heapRegion.cpp needs to include space.inline.hpp since it uses the >>>> SCAN_AND_FORWARD macro. >>>> >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html >>>> >>>> I don't think its necessary to add the prefetch.inline.hpp to >>>> precompiled.hpp. >>>> >>>> >>>> Otherwise, this looks good to me. >>>> >>>> thanks, >>>> StefanK >>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >>>>> Sent: Donnerstag, 15. Mai 2014 17:14 >>>>> To: Lindenmaier, Goetz >>>>> Cc: hotspot-dev at openjdk.java.net >>>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>>> >>>>> Looks good to me. I thought that prefetch is used more often but apparently not. >>>>> >>>>> On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> This change introduces an umbrella header for prefetch__.inline.hpp >>>>>> files: runtime/prefetch.inline.hpp as proposed in >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >>>>>> This follows the scheme applied to atomic.inline.hpp. >>>>>> >>>>>> Further this change adds includes of prefetch.inline.hpp in all .cpp >>>>>> and .inline.hpp files where a method of Prefetch declared 'inline' is >>>>>> called. >>>>>> >>>>>> Finally it moves macros calling inline methods of Prefetch from >>>>>> space.hpp to the corresponding .inline.hpp file and adds the necessary >>>>>> includes in files using the moved macros. >>>>>> >>>>>> Please review and test this change. I please need a sponsor. >>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >>>>>> >>>>>> Thanks and best regards, >>>>>> Goetz. >>> >> > From staffan.larsen at oracle.com Tue May 20 17:46:07 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 20 May 2014 19:46:07 +0200 Subject: RFR: Backout of fix for JDK-8041934 Message-ID: <87B6E481-88E6-47A4-9B84-5D1DE7208EB7@oracle.com> The fix for JDK-8041934 was bad and I would like to back it out while preparing a new fix. bug: https://bugs.openjdk.java.net/browse/JDK-8043314 webrev: http://cr.openjdk.java.net/~sla/8043314/webrev.00/ The change set was created with ?hg backout?. I will create a clone of 8041934 to track the real fix. Thanks, /Staffan From coleen.phillimore at oracle.com Tue May 20 17:50:02 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 20 May 2014 13:50:02 -0400 Subject: RFR: Backout of fix for JDK-8041934 In-Reply-To: <87B6E481-88E6-47A4-9B84-5D1DE7208EB7@oracle.com> References: <87B6E481-88E6-47A4-9B84-5D1DE7208EB7@oracle.com> Message-ID: <537B95CA.4070107@oracle.com> This looks good. Does hg backout actually work? Coleen On 5/20/14, 1:46 PM, Staffan Larsen wrote: > The fix for JDK-8041934 was bad and I would like to back it out while preparing a new fix. > > bug: https://bugs.openjdk.java.net/browse/JDK-8043314 > webrev: http://cr.openjdk.java.net/~sla/8043314/webrev.00/ > > The change set was created with ?hg backout?. I will create a clone of 8041934 to track the real fix. > > > Thanks, > /Staffan From staffan.larsen at oracle.com Tue May 20 17:51:57 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 20 May 2014 19:51:57 +0200 Subject: RFR: Backout of fix for JDK-8041934 In-Reply-To: <537B95CA.4070107@oracle.com> References: <87B6E481-88E6-47A4-9B84-5D1DE7208EB7@oracle.com> <537B95CA.4070107@oracle.com> Message-ID: <9F0CBFD3-9C4A-43ED-8AFC-6FED6038280B@oracle.com> Thanks. ?hg backout? just creates an anti-delta for a revision you give it. Nothing magic. /Staffan On 20 maj 2014, at 19:50, Coleen Phillimore wrote: > > This looks good. Does hg backout actually work? > > Coleen > > On 5/20/14, 1:46 PM, Staffan Larsen wrote: >> The fix for JDK-8041934 was bad and I would like to back it out while preparing a new fix. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8043314 >> webrev: http://cr.openjdk.java.net/~sla/8043314/webrev.00/ >> >> The change set was created with ?hg backout?. I will create a clone of 8041934 to track the real fix. >> >> >> Thanks, >> /Staffan > From serguei.spitsyn at oracle.com Tue May 20 18:31:45 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 May 2014 11:31:45 -0700 Subject: RFR: Backout of fix for JDK-8041934 In-Reply-To: <87B6E481-88E6-47A4-9B84-5D1DE7208EB7@oracle.com> References: <87B6E481-88E6-47A4-9B84-5D1DE7208EB7@oracle.com> Message-ID: <537B9F91.8060809@oracle.com> Staffan, I compared the anti-delta with the original fix. The anti-delta is correct. Thanks, Serguei On 5/20/14 10:46 AM, Staffan Larsen wrote: > The fix for JDK-8041934 was bad and I would like to back it out while preparing a new fix. > > bug: https://bugs.openjdk.java.net/browse/JDK-8043314 > webrev: http://cr.openjdk.java.net/~sla/8043314/webrev.00/ > > The change set was created with ?hg backout?. I will create a clone of 8041934 to track the real fix. > > > Thanks, > /Staffan From coleen.phillimore at oracle.com Tue May 20 21:47:25 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 20 May 2014 17:47:25 -0400 Subject: RFR 8042727: nsk/jdb/unwatch/unwatch001 crash in InstanceKlass::methods_do(void (*)(Method*)) Message-ID: <537BCD6D.2090402@oracle.com> Summary: Only walk methods in instanceklass if the class is loaded The bug was caused by walking methods in the CLDG while some classes were still being loaded, so you could get null for methods(). I also changed print_method_statistics to not unconditionally walk the CLDG methods at exit if there are no CompilerOracle print options. Tested with regression/other/RuntimeTest which crashed within an hour with this bug, and never with the fix. Also ran jtreg tests, and manually tested .hotspot_compiler prints a method requested. I didn't add a test because there should be some other compiler print tests already and this would be a small meaningless test. open webrev at http://cr.openjdk.java.net/~coleenp/8042727/ bug link https://bugs.openjdk.java.net/browse/JDK-8042727 Thanks, Coleen From david.holmes at oracle.com Wed May 21 05:58:41 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 May 2014 15:58:41 +1000 Subject: RFR 8042727: nsk/jdb/unwatch/unwatch001 crash in InstanceKlass::methods_do(void (*)(Method*)) In-Reply-To: <537BCD6D.2090402@oracle.com> References: <537BCD6D.2090402@oracle.com> Message-ID: <537C4091.8080607@oracle.com> Hi Coleen, This seems quite reasonable to me. David On 21/05/2014 7:47 AM, Coleen Phillimore wrote: > Summary: Only walk methods in instanceklass if the class is loaded > > The bug was caused by walking methods in the CLDG while some classes > were still being loaded, so you could get null for methods(). I also > changed print_method_statistics to not unconditionally walk the CLDG > methods at exit if there are no CompilerOracle print options. > > Tested with regression/other/RuntimeTest which crashed within an hour > with this bug, and never with the fix. > Also ran jtreg tests, and manually tested .hotspot_compiler prints a > method requested. I didn't add a test because there should be some > other compiler print tests already and this would be a small meaningless > test. > > open webrev at http://cr.openjdk.java.net/~coleenp/8042727/ > bug link https://bugs.openjdk.java.net/browse/JDK-8042727 > > Thanks, > Coleen From goetz.lindenmaier at sap.com Wed May 21 07:36:43 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 21 May 2014 07:36:43 +0000 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <949226D8-9228-4C8C-979C-3A51031981F9@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> <537A20B3.1040104@oracle.com> <7E3E56CF-46B6-4869-97E0-D3D014844DF6@oracle.com> <21CE3D5B-9E7D-4188-BF14-DC097F4C780C@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC3C36@DEWDFEMB12A.global.corp.sap> <949226D8-9228-4C8C-979C-3A51031981F9@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CEC3EBB@DEWDFEMB12A.global.corp.sap> Hi Christian, thanks for pushing the change! Best regards, Goetz. -----Original Message----- From: Christian Thalinger [mailto:christian.thalinger at oracle.com] Sent: Dienstag, 20. Mai 2014 17:17 To: Lindenmaier, Goetz Cc: Stefan Karlsson; hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp On May 20, 2014, at 12:32 AM, Lindenmaier, Goetz wrote: > Hi Christian, > > I added the Reviewed-by line to the changeset I uploaded, so it jchecks fine. > Is this what you needed? Can I prepare that some more? Yes, that's what I wanted. Thanks. > > I tested the changeset on linuxx86_64, windowsx86_64, solaris sparc 64 & 32 bit, > darwinintel64, linuxppc64 and aix. Opt & fastdbg build on all (without precompiled headers) > and dbg build on most of the platforms. > > Thanks for your help, > Goetz. > > > > -----Original Message----- > From: Christian Thalinger [mailto:christian.thalinger at oracle.com] > Sent: Montag, 19. Mai 2014 18:50 > To: Stefan Karlsson > Cc: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp > > Actually, since you are a Commiter can you prepare a changeset so I can just take and push it? Saves me some time :-) > > On May 19, 2014, at 9:48 AM, Christian Thalinger wrote: > >> Alright. I will push this to hs-comp. >> >> On May 19, 2014, at 8:18 AM, Stefan Karlsson wrote: >> >>> On 2014-05-19 16:43, Lindenmaier, Goetz wrote: >>>> Hi Stefan, >>>> >>>> thanks for the comments! >>>> I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. >>>> I also saw another usage of Prefetch in klass.hpp. I removed >>>> the macro altogether as it's unused. >>>> >>>> I kept the include in precompiled.hpp though, as for other >>>> files it's listed there, too. If you insist, and >>>> there are no other objections, I'll remove it. >>> >>> OK. >>> >>>> >>>> The new webrev is at >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 >>> >>> Looks good. >>> >>> thanks, >>> StefanK >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> -----Original Message----- >>>> From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] >>>> Sent: Montag, 19. Mai 2014 13:21 >>>> To: Lindenmaier, Goetz; Christian Thalinger >>>> Cc: hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>> >>>> Hi Goetz, >>>> >>>> On 2014-05-19 12:58, Lindenmaier, Goetz wrote: >>>>> Hi Christian, >>>>> >>>>> thanks for the review! >>>>> >>>>> Although prefetch is not used that often, I think it's good >>>>> that all headers in the os_cpu directories are handled >>>>> similarly now, i.e., they are only included in the respective >>>>> shared file. (For some it's the file in the /cpu/ dir, like >>>>> copy_linux_x86.inline.hpp.) >>>>> >>>>> Could you sponsor the change? Do I need a second reviewer? >>>> Thanks for cleaning this up. >>>> >>>> Some minor comments: >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html >>>> >>>> You shouldn't have to include prefetch.inline.hpp from this file. >>>> >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html >>>> >>>> heapRegion.cpp needs to include space.inline.hpp since it uses the >>>> SCAN_AND_FORWARD macro. >>>> >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html >>>> >>>> I don't think its necessary to add the prefetch.inline.hpp to >>>> precompiled.hpp. >>>> >>>> >>>> Otherwise, this looks good to me. >>>> >>>> thanks, >>>> StefanK >>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >>>>> Sent: Donnerstag, 15. Mai 2014 17:14 >>>>> To: Lindenmaier, Goetz >>>>> Cc: hotspot-dev at openjdk.java.net >>>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>>> >>>>> Looks good to me. I thought that prefetch is used more often but apparently not. >>>>> >>>>> On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> This change introduces an umbrella header for prefetch__.inline.hpp >>>>>> files: runtime/prefetch.inline.hpp as proposed in >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >>>>>> This follows the scheme applied to atomic.inline.hpp. >>>>>> >>>>>> Further this change adds includes of prefetch.inline.hpp in all .cpp >>>>>> and .inline.hpp files where a method of Prefetch declared 'inline' is >>>>>> called. >>>>>> >>>>>> Finally it moves macros calling inline methods of Prefetch from >>>>>> space.hpp to the corresponding .inline.hpp file and adds the necessary >>>>>> includes in files using the moved macros. >>>>>> >>>>>> Please review and test this change. I please need a sponsor. >>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >>>>>> >>>>>> Thanks and best regards, >>>>>> Goetz. >>> >> > From bengt.rutisson at oracle.com Wed May 21 09:31:16 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 21 May 2014 11:31:16 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps Message-ID: <537C7264.70200@oracle.com> Hi everyone, Can I have some reviews for this patch: http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8043607 This is mostly a GC change but I am sending it to the broader list since I added a method to ostream.hpp/cpp. We had this code duplicated in many places in the GC code: gclog_or_tty->date_stamp(PrintGCDateStamps); gclog_or_tty->stamp(PrintGCTimeStamps); And with my patch I had to add one more section to this code duplication. Instead I wrapped all three calls in a method called gclog_stamp(). It is a bit hard to know where to place that method. ostream seems like the wrong place for GC specific code, but there is already lots of GC specific code there and even gclog_or_tty is declared there. So I added the method there for now: void outputStream::gclog_stamp(uint gc_id) { date_stamp(PrintGCDateStamps); stamp(PrintGCTimeStamps); if (PrintGCID) { print("#%u: ", gc_id); } } Other than that this is a pure GC change. Background: With event based tracing we introduced a GC id field on all GC events. This id is a constantly increasing number associated with each GC. I figured we can use the this GC id as a decoration for the HotSpot GC logging similarly to what we do with PrintGCTimeStamps and PrintGCDateStamps. A nice side effect of this is that it will make it easy to map an event in the event tracing to an entry in the GC log. Currently this is impossible to do deterministically since the logging and event tracing use different timing. I propose to add a flag called PrintGCID. Here's an example log entry for GC number 4 running with -XX:+PrintGCID: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] It is a decoration similar to the timestamps. Here's what it looks like if you add -XX:+PrintGCTimeStamps: 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] And here's what it will look like if you add -XX:+PrintGCDateStamps: 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] Here's a little longer example running ParallelGC: #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 secs] #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 secs] Note that for the concurrent GCs this gives a good way of seeing the concurrent action. Here is a G1 example: #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), 0,0383157 secs] #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), 0,0412220 secs] #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), 0,0426144 secs] #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), 0,1467633 secs] #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), 0,1627465 secs] #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), 0,2376728 secs] #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) 255M->269M(1004M), 0,2623779 secs] #7: [GC concurrent-root-region-scan-start] #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] #7: [GC concurrent-mark-start] #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), 0,3007405 secs] #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), 0,4063937 secs] #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), 0,7500609 secs] #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), 1,5464884 secs] #7: [GC concurrent-mark-end, 3,9464599 secs] #7: [GC remark, 0,0240462 secs] #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] #7: [GC concurrent-cleanup-start] #7: [GC concurrent-cleanup-end, 0,0000228 secs] #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), 2,2464423 secs] Notice how the complete concurrent cycle is decorated with id #7. That makes it possible to follow the concurrent work more easily and grep it out etc. Here's a similar thing for CMS where the concurrent cycle #22 has a young GC #23 in the middle of it: #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), 1,4558792 secs] #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), 0,7909961 secs] #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] I think this is very helpful information to have, so I would like to push this with the new flag PrintGCID as true by default. I'm sure some parsers will break, but it is an easy work around to run with -XX:-PrintGCID to get the old format. It should even be pretty easy to do post processing on a log file to remove the GC id information. Having it on by default makes it possible for us to use this information and in particular be able to map it to the JFR events we see in MissionControl. I would also like to backport this to 8u40, but in that case with the default value of PrintGCID as false. Thanks, Bengt From frederic.parain at oracle.com Wed May 21 10:40:22 2014 From: frederic.parain at oracle.com (Frederic Parain) Date: Wed, 21 May 2014 12:40:22 +0200 Subject: RFR 8042727: nsk/jdb/unwatch/unwatch001 crash in InstanceKlass::methods_do(void (*)(Method*)) In-Reply-To: <537C4091.8080607@oracle.com> References: <537BCD6D.2090402@oracle.com> <537C4091.8080607@oracle.com> Message-ID: <537C8296.6070506@oracle.com> Coleen, Looks good. Minor nit: copyright date in compilerOracle.hpp Regards, Fred On 05/21/2014 07:58 AM, David Holmes wrote: > Hi Coleen, > > This seems quite reasonable to me. > > David > > On 21/05/2014 7:47 AM, Coleen Phillimore wrote: >> Summary: Only walk methods in instanceklass if the class is loaded >> >> The bug was caused by walking methods in the CLDG while some classes >> were still being loaded, so you could get null for methods(). I also >> changed print_method_statistics to not unconditionally walk the CLDG >> methods at exit if there are no CompilerOracle print options. >> >> Tested with regression/other/RuntimeTest which crashed within an hour >> with this bug, and never with the fix. >> Also ran jtreg tests, and manually tested .hotspot_compiler prints a >> method requested. I didn't add a test because there should be some >> other compiler print tests already and this would be a small meaningless >> test. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8042727/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8042727 >> >> Thanks, >> Coleen -- Frederic Parain - Oracle Grenoble Engineering Center - France Phone: +33 4 76 18 81 17 Email: Frederic.Parain at oracle.com From fredrik.arvidsson at oracle.com Wed May 21 12:39:47 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Wed, 21 May 2014 14:39:47 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: References: <5375D08C.9070206@oracle.com> <5378D1EE.3020309@oracle.com> Message-ID: <537C9E93.20901@oracle.com> Hi Thomas and thanks for the feedback. Interesting to see that you already have a design that is so close to what we are proposing in this JEP. I have added some comments in-line below. On 2014-05-20 09:59, Thomas St?fe wrote: > Hi all, > > I like you proposal and hope this tracing system turns out well. > > I'm with the SAP JVM; we have had our own homebrewn tracing system for > years, which resembles your proposal a lot. If your solution turns out > well, we may decide to abandon our propietary solution in favor of > your proposal. > > Here are some aspects we found important over the last years with our > tracing system: > > 1) tracing during initialization phase. > When tracing in code which gets executed very early it would be nice > to have a solution which works right from the start. So, argument > parsing for the tracing system should happen as early as possible, or > not rely only on command line arguments. > Often I found myself tracing initialization code only to find that > my output was not visible, so there was always the question "did I not > hit my code or was the tracing system not yet initialized?" - in the > end, due to time, one often falls back to fprintf. > This is of course an implementation specific question, but in my proof of concept implementation I initialize the logger framework very early in the Threads::create_vm() method which is as early as it gets I think. There are some prerequisites for my code that os::init() have been run to have Atomic::cmpxchg() available. > 2) Tracing system should be implemented as "lowest VM layer", without > dependencies to VM infrastructure > This one is clear and also very important. It also affects all > adapters. > For instance, tracing system should not use os::malloc, because I > might want to trace os::malloc. So, it should be implemented as an > independend module, with no dependencies to other infrastructure. > This is also related to (1), as it makes it possible to trace right > from VM start without having to wait for VM initialization > We should strive to make the logger implementation having as few dependencies to the rest of the vm code as possible. I think it will be a balance between having to re-implement and reuse whats in there already. One example is the Atomic CAS function I use to implement a read-writer lock. > 3) inactive trace points should not incurr costs > > In our tracing system, we have a flat vector of "loggers" which are > controlled by a global bit array; this means that at runtime, any > decision to actually trace costs one memory access to a memory > location which hopefully is already cached. Simple and stupid, but it > works, and costs to not trace are at a minimum. > > In your proposal I would have to call "Log::log()". The logger class > has to be resolved, and its state queried, which, depending on how > this is implemented, may cause more than one memory accesses. > The loggers in my implementation are statically declared. Loggers and all related code is 'generated' by just a few xmacro's at compile time and will be low cost to use I hope. A logging call will make one comparison to find out if it needs to log or not. This is something we have to look in to later, and we will have to spend some time on optimization I'm sure. But as you say, a good design from the start lays the foundation for a good implementation. > > 4) Adapters > > This whole thing seems a bit too complicated. > - Do we really need the ability to attach different adapters to > different Loggers? Would one adapter for the whole VM not suffice? > - Do we really need the ability to chain adapters, especially chain > different file loggers? > This seems to me like something the OS does far better than the VM. > > This flexibility would cause a lot of testing effort, and you never > can be sure that you tested all possible combinations. Also timing: > different adapters may show different timing behaviour, and if > different loggers may cause different adapters or combination of > adapters, timing of different trace spots in respect to each other may > differ greatly. > > I am also concerned about the reliability of adapters. In my > experience, the tracing backends are a weak spot in a tracing system, > especially if one allows other teams to extend them by writing new > adapters. > I have asked my colleagues about the need to have multiple adapters and different adapters for different loggers and it seem there are cases when this is desired. I have not yet thought about the test implications, but as you say it will be harder to test this functionality than if we just had one adapter for all loggers. > > 5) Just cosmetics: > > I would prefer the logging calls to be simpler. Instead of writing > > Log::log(Log::vm_init(), LogLevel::info(), "Init took %lu ms to > complete.", time); > > I would prefer something like > > logI(vm_init, "Init took %lu ms to complete.", time); > > or for an error: > > logE(vm_init, "All is lost."); > I have experimented some with a couple of macros making logging less verbose, it is still not ready though. One thing i added to the macro was to check if logging is enabled for the current level on the logger. This saves us one call to the logger passing all parameters etc... still experimental. > > > Some minor proposals: > > 6) Shared memory adapter > > I would propose another adaptor (beside stdout and file), which > writes to a preallocated large shared memory segment; overwriting old > entries if full. That shared memory segment could be read from the > outside with a command line tool. > > (Arguably the same could be done with memory file systems, in an OS > dependend way). This is definitely a candidate for an adapter. Probably pretty easy to implement also. > > 7) LogMark > > Log& Log::().begin_transaction(); > bool Log::().commit_transaction(); > > could be wrapped into something like a LogMark, similar to a > ResourceMark, to end the transaction when the stack unwinds. > I will investigate if I can add some support for this. I have done similar things before when implementing lock/concurrency primitives and they are pretty handy at many times. > > Kind Regards > > Thomas St?fe Cheers /Fredrik From kirk at kodewerk.com Wed May 21 14:12:26 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 21 May 2014 16:12:26 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps In-Reply-To: <537C7264.70200@oracle.com> References: <537C7264.70200@oracle.com> Message-ID: <79D2DC95-29ED-43AD-B772-CB113AD9A1FD@kodewerk.com> Hi Bengt, This change looks very useful. Kind regards, Kirk On May 21, 2014, at 11:31 AM, Bengt Rutisson wrote: > > Hi everyone, > > Can I have some reviews for this patch: > http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ > > https://bugs.openjdk.java.net/browse/JDK-8043607 > > This is mostly a GC change but I am sending it to the broader list since I added a method to ostream.hpp/cpp. We had this code duplicated in many places in the GC code: > > gclog_or_tty->date_stamp(PrintGCDateStamps); > gclog_or_tty->stamp(PrintGCTimeStamps); > > And with my patch I had to add one more section to this code duplication. Instead I wrapped all three calls in a method called gclog_stamp(). It is a bit hard to know where to place that method. ostream seems like the wrong place for GC specific code, but there is already lots of GC specific code there and even gclog_or_tty is declared there. So I added the method there for now: > > void outputStream::gclog_stamp(uint gc_id) { > date_stamp(PrintGCDateStamps); > stamp(PrintGCTimeStamps); > if (PrintGCID) { > print("#%u: ", gc_id); > } > } > > Other than that this is a pure GC change. > > > Background: > > With event based tracing we introduced a GC id field on all GC events. This id is a constantly increasing number associated with each GC. I figured we can use the this GC id as a decoration for the HotSpot GC logging similarly to what we do with PrintGCTimeStamps and PrintGCDateStamps. > > A nice side effect of this is that it will make it easy to map an event in the event tracing to an entry in the GC log. Currently this is impossible to do deterministically since the logging and event tracing use different timing. > > I propose to add a flag called PrintGCID. Here's an example log entry for GC number 4 running with -XX:+PrintGCID: > > #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] > > It is a decoration similar to the timestamps. Here's what it looks like if you add -XX:+PrintGCTimeStamps: > > 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] > > And here's what it will look like if you add -XX:+PrintGCDateStamps: > > 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] > > > Here's a little longer example running ParallelGC: > > #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] > #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] > #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] > #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 secs] > #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] > #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 secs] > > Note that for the concurrent GCs this gives a good way of seeing the concurrent action. Here is a G1 example: > > #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), 0,0383157 secs] > #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), 0,0412220 secs] > #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), 0,0426144 secs] > #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), 0,1467633 secs] > #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), 0,1627465 secs] > #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), 0,2376728 secs] > #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) 255M->269M(1004M), 0,2623779 secs] > #7: [GC concurrent-root-region-scan-start] > #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] > #7: [GC concurrent-mark-start] > #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), 0,3007405 secs] > #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), 0,4063937 secs] > #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), 0,7500609 secs] > #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), 1,5464884 secs] > #7: [GC concurrent-mark-end, 3,9464599 secs] > #7: [GC remark, 0,0240462 secs] > #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] > #7: [GC concurrent-cleanup-start] > #7: [GC concurrent-cleanup-end, 0,0000228 secs] > #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), 2,2464423 secs] > > Notice how the complete concurrent cycle is decorated with id #7. That makes it possible to follow the concurrent work more easily and grep it out etc. > > > Here's a similar thing for CMS where the concurrent cycle #22 has a young GC #23 in the middle of it: > > #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), 1,4558792 secs] > #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] > #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), 0,7909961 secs] > #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] > > > I think this is very helpful information to have, so I would like to push this with the new flag PrintGCID as true by default. I'm sure some parsers will break, but it is an easy work around to run with -XX:-PrintGCID to get the old format. It should even be pretty easy to do post processing on a log file to remove the GC id information. Having it on by default makes it possible for us to use this information and in particular be able to map it to the JFR events we see in MissionControl. > > I would also like to backport this to 8u40, but in that case with the default value of PrintGCID as false. > > Thanks, > Bengt > > > > > > From jvanek at redhat.com Wed May 21 14:48:21 2014 From: jvanek at redhat.com (Jiri Vanek) Date: Wed, 21 May 2014 16:48:21 +0200 Subject: [rfc] hs_err log in temp readable for all In-Reply-To: <537A054F.6060507@redhat.com> References: <537A054F.6060507@redhat.com> Message-ID: <537CBCB5.7080507@redhat.com> ping? On 05/19/2014 03:21 PM, Jiri Vanek wrote: > Hi! > > I run JVM in read only directory in shared server and form time to time it fails. However, I noted that hs_logs which then go to TMP direcotry, are readable to all. > It may contains private information, and I think that this file should be readable only to its owner. > > Attached webrevs fixes it: > for all hs_err logs - http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/minimal/1/webrev/ > for hs_log in tmp only - http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/better/1/webrev/ > > If I will be even more paranoid, then i think that hs_err{pid} name is predictable, and can point to pre-created pipe. Maybe this 0600 file should be placed in 0700 directory with salt in name. Or add the salt to file itself (when created in tmp) > > I will be happy to adapt the patch for your needs. > > Best regards, > J. From christian.thalinger at oracle.com Wed May 21 14:55:31 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 21 May 2014 07:55:31 -0700 Subject: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CEC3EBB@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CEC30DF@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CEC3973@DEWDFEMB12A.global.corp.sap> <5379E919.7010803@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC39F3@DEWDFEMB12A.global.corp.sap> <537A20B3.1040104@oracle.com> <7E3E56CF-46B6-4869-97E0-D3D014844DF6@oracle.com> <21CE3D5B-9E7D-4188-BF14-DC097F4C780C@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC3C36@DEWDFEMB12A.global.corp.sap> <949226D8-9228-4C8C-979C-3A51031981F9@oracle.com> <4295855A5C1DE049A61835A1887419CC2CEC3EBB@DEWDFEMB12A.global.corp.sap> Message-ID: <7864402B-8A1A-43F2-8A67-AB1D287C3126@oracle.com> You?re welcome. On May 21, 2014, at 12:36 AM, Lindenmaier, Goetz wrote: > Hi Christian, > > thanks for pushing the change! > > Best regards, > Goetz. > > -----Original Message----- > From: Christian Thalinger [mailto:christian.thalinger at oracle.com] > Sent: Dienstag, 20. Mai 2014 17:17 > To: Lindenmaier, Goetz > Cc: Stefan Karlsson; hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp > > > On May 20, 2014, at 12:32 AM, Lindenmaier, Goetz wrote: > >> Hi Christian, >> >> I added the Reviewed-by line to the changeset I uploaded, so it jchecks fine. >> Is this what you needed? Can I prepare that some more? > > Yes, that's what I wanted. Thanks. > >> >> I tested the changeset on linuxx86_64, windowsx86_64, solaris sparc 64 & 32 bit, >> darwinintel64, linuxppc64 and aix. Opt & fastdbg build on all (without precompiled headers) >> and dbg build on most of the platforms. >> >> Thanks for your help, >> Goetz. >> >> >> >> -----Original Message----- >> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >> Sent: Montag, 19. Mai 2014 18:50 >> To: Stefan Karlsson >> Cc: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >> >> Actually, since you are a Commiter can you prepare a changeset so I can just take and push it? Saves me some time :-) >> >> On May 19, 2014, at 9:48 AM, Christian Thalinger wrote: >> >>> Alright. I will push this to hs-comp. >>> >>> On May 19, 2014, at 8:18 AM, Stefan Karlsson wrote: >>> >>>> On 2014-05-19 16:43, Lindenmaier, Goetz wrote: >>>>> Hi Stefan, >>>>> >>>>> thanks for the comments! >>>>> I fixed compactibelFreeListSpace.cpp and heapRegion.cpp. >>>>> I also saw another usage of Prefetch in klass.hpp. I removed >>>>> the macro altogether as it's unused. >>>>> >>>>> I kept the include in precompiled.hpp though, as for other >>>>> files it's listed there, too. If you insist, and >>>>> there are no other objections, I'll remove it. >>>> >>>> OK. >>>> >>>>> >>>>> The new webrev is at >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.01 >>>> >>>> Looks good. >>>> >>>> thanks, >>>> StefanK >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> -----Original Message----- >>>>> From: Stefan Karlsson [mailto:stefan.karlsson at oracle.com] >>>>> Sent: Montag, 19. Mai 2014 13:21 >>>>> To: Lindenmaier, Goetz; Christian Thalinger >>>>> Cc: hotspot-dev at openjdk.java.net >>>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>>> >>>>> Hi Goetz, >>>>> >>>>> On 2014-05-19 12:58, Lindenmaier, Goetz wrote: >>>>>> Hi Christian, >>>>>> >>>>>> thanks for the review! >>>>>> >>>>>> Although prefetch is not used that often, I think it's good >>>>>> that all headers in the os_cpu directories are handled >>>>>> similarly now, i.e., they are only included in the respective >>>>>> shared file. (For some it's the file in the /cpu/ dir, like >>>>>> copy_linux_x86.inline.hpp.) >>>>>> >>>>>> Could you sponsor the change? Do I need a second reviewer? >>>>> Thanks for cleaning this up. >>>>> >>>>> Some minor comments: >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp.frames.html >>>>> >>>>> You shouldn't have to include prefetch.inline.hpp from this file. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/memory/space.inline.hpp.frames.html >>>>> >>>>> heapRegion.cpp needs to include space.inline.hpp since it uses the >>>>> SCAN_AND_FORWARD macro. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/src/share/vm/precompiled/precompiled.hpp.frames.html >>>>> >>>>> I don't think its necessary to add the prefetch.inline.hpp to >>>>> precompiled.hpp. >>>>> >>>>> >>>>> Otherwise, this looks good to me. >>>>> >>>>> thanks, >>>>> StefanK >>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Christian Thalinger [mailto:christian.thalinger at oracle.com] >>>>>> Sent: Donnerstag, 15. Mai 2014 17:14 >>>>>> To: Lindenmaier, Goetz >>>>>> Cc: hotspot-dev at openjdk.java.net >>>>>> Subject: Re: RFR(M): 8042737 : Introduce umbrella header prefetch.inline.hpp >>>>>> >>>>>> Looks good to me. I thought that prefetch is used more often but apparently not. >>>>>> >>>>>> On May 15, 2014, at 1:16 AM, Lindenmaier, Goetz wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This change introduces an umbrella header for prefetch__.inline.hpp >>>>>>> files: runtime/prefetch.inline.hpp as proposed in >>>>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013702.html >>>>>>> This follows the scheme applied to atomic.inline.hpp. >>>>>>> >>>>>>> Further this change adds includes of prefetch.inline.hpp in all .cpp >>>>>>> and .inline.hpp files where a method of Prefetch declared 'inline' is >>>>>>> called. >>>>>>> >>>>>>> Finally it moves macros calling inline methods of Prefetch from >>>>>>> space.hpp to the corresponding .inline.hpp file and adds the necessary >>>>>>> includes in files using the moved macros. >>>>>>> >>>>>>> Please review and test this change. I please need a sponsor. >>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8042737-prefIncl/webrev.00/ >>>>>>> >>>>>>> Thanks and best regards, >>>>>>> Goetz. >>>> >>> >> > From jesper.wilhelmsson at oracle.com Wed May 21 17:44:27 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 21 May 2014 19:44:27 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps In-Reply-To: <537C7264.70200@oracle.com> References: <537C7264.70200@oracle.com> Message-ID: <537CE5FB.9060003@oracle.com> Hi Bengt, Good change! Thanks for doing this! A few thoughts: * There are a few places that passes both a GCTimer and now a GCId. Would it make sense to pass an info-object of some kind that contains misc info that is used for logging/tracing? I'm not suggesting that you should implement that in this change, just a thought for the future. * In genCollectdHeap.cpp you increase the gc id only if PrintGCDetails is enabled. I assume this is the only place where a gc id is used when running collectors that use GenCollectedHeap::do_collection(). If not, the id created and used in other places would be different depending on if PrintGCDetails was enabled or not. Anyway, looks good! Ship it! /Jesper Bengt Rutisson skrev 21/5/14 11:31: > > Hi everyone, > > Can I have some reviews for this patch: > http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ > > https://bugs.openjdk.java.net/browse/JDK-8043607 > > This is mostly a GC change but I am sending it to the broader list since I added > a method to ostream.hpp/cpp. We had this code duplicated in many places in the > GC code: > > gclog_or_tty->date_stamp(PrintGCDateStamps); > gclog_or_tty->stamp(PrintGCTimeStamps); > > And with my patch I had to add one more section to this code duplication. > Instead I wrapped all three calls in a method called gclog_stamp(). It is a bit > hard to know where to place that method. ostream seems like the wrong place for > GC specific code, but there is already lots of GC specific code there and even > gclog_or_tty is declared there. So I added the method there for now: > > void outputStream::gclog_stamp(uint gc_id) { > date_stamp(PrintGCDateStamps); > stamp(PrintGCTimeStamps); > if (PrintGCID) { > print("#%u: ", gc_id); > } > } > > Other than that this is a pure GC change. > > > Background: > > With event based tracing we introduced a GC id field on all GC events. This id > is a constantly increasing number associated with each GC. I figured we can use > the this GC id as a decoration for the HotSpot GC logging similarly to what we > do with PrintGCTimeStamps and PrintGCDateStamps. > > A nice side effect of this is that it will make it easy to map an event in the > event tracing to an entry in the GC log. Currently this is impossible to do > deterministically since the logging and event tracing use different timing. > > I propose to add a flag called PrintGCID. Here's an example log entry for GC > number 4 running with -XX:+PrintGCID: > > #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] > > It is a decoration similar to the timestamps. Here's what it looks like if you > add -XX:+PrintGCTimeStamps: > > 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 > secs] > > And here's what it will look like if you add -XX:+PrintGCDateStamps: > > 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation Pause) > (young) 185M->212M(502M), 0,2373420 secs] > > > Here's a little longer example running ParallelGC: > > #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] > #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] > #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] > #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 secs] > #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] > #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 secs] > > Note that for the concurrent GCs this gives a good way of seeing the concurrent > action. Here is a G1 example: > > #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), 0,0383157 secs] > #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), 0,0412220 secs] > #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), 0,0426144 secs] > #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), 0,1467633 secs] > #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), 0,1627465 secs] > #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), 0,2376728 secs] > #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) 255M->269M(1004M), > 0,2623779 secs] > #7: [GC concurrent-root-region-scan-start] > #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] > #7: [GC concurrent-mark-start] > #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), 0,3007405 secs] > #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), 0,4063937 secs] > #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), 0,7500609 secs] > #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), 1,5464884 secs] > #7: [GC concurrent-mark-end, 3,9464599 secs] > #7: [GC remark, 0,0240462 secs] > #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] > #7: [GC concurrent-cleanup-start] > #7: [GC concurrent-cleanup-end, 0,0000228 secs] > #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), 2,2464423 secs] > > Notice how the complete concurrent cycle is decorated with id #7. That makes it > possible to follow the concurrent work more easily and grep it out etc. > > > Here's a similar thing for CMS where the concurrent cycle #22 has a young GC #23 > in the middle of it: > > #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), 1,4558792 secs] > #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] > #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), 0,7909961 secs] > #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] > > > I think this is very helpful information to have, so I would like to push this > with the new flag PrintGCID as true by default. I'm sure some parsers will > break, but it is an easy work around to run with -XX:-PrintGCID to get the old > format. It should even be pretty easy to do post processing on a log file to > remove the GC id information. Having it on by default makes it possible for us > to use this information and in particular be able to map it to the JFR events we > see in MissionControl. > > I would also like to backport this to 8u40, but in that case with the default > value of PrintGCID as false. > > Thanks, > Bengt > > > > > > From coleen.phillimore at oracle.com Wed May 21 18:38:19 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 21 May 2014 14:38:19 -0400 Subject: RFR 8042727: nsk/jdb/unwatch/unwatch001 crash in InstanceKlass::methods_do(void (*)(Method*)) In-Reply-To: <537C8296.6070506@oracle.com> References: <537BCD6D.2090402@oracle.com> <537C4091.8080607@oracle.com> <537C8296.6070506@oracle.com> Message-ID: <537CF29B.1000403@oracle.com> Thanks Frederic and David. I have a script to fix the copyright date when I commit. Coleen On 5/21/14, 6:40 AM, Frederic Parain wrote: > Coleen, > > Looks good. > Minor nit: copyright date in compilerOracle.hpp > > Regards, > > Fred > > On 05/21/2014 07:58 AM, David Holmes wrote: >> Hi Coleen, >> >> This seems quite reasonable to me. >> >> David >> >> On 21/05/2014 7:47 AM, Coleen Phillimore wrote: >>> Summary: Only walk methods in instanceklass if the class is loaded >>> >>> The bug was caused by walking methods in the CLDG while some classes >>> were still being loaded, so you could get null for methods(). I also >>> changed print_method_statistics to not unconditionally walk the CLDG >>> methods at exit if there are no CompilerOracle print options. >>> >>> Tested with regression/other/RuntimeTest which crashed within an hour >>> with this bug, and never with the fix. >>> Also ran jtreg tests, and manually tested .hotspot_compiler prints a >>> method requested. I didn't add a test because there should be some >>> other compiler print tests already and this would be a small >>> meaningless >>> test. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8042727/ >>> bug link https://bugs.openjdk.java.net/browse/JDK-8042727 >>> >>> Thanks, >>> Coleen > From coleen.phillimore at oracle.com Wed May 21 18:43:36 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 21 May 2014 14:43:36 -0400 Subject: [8u20] RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking Message-ID: <537CF3D8.5000403@oracle.com> Summary: Only prune metaspace virtual spaces at safepoint so walking them is safe outside a safepoint. This change has been in hs-rt for a week and backported cleanly. JFR performance in 8u20 needs this fix. open webrev at http://cr.openjdk.java.net/~coleenp/8038212_8u20/ bug link https://bugs.openjdk.java.net/browse/JDK-8038212 Thanks, Coleen From markus.gronlund at oracle.com Wed May 21 18:52:34 2014 From: markus.gronlund at oracle.com (=?iso-8859-1?B?TWFya3VzIEdy9m5sdW5k?=) Date: Wed, 21 May 2014 11:52:34 -0700 (PDT) Subject: [8u20] RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <537CF3D8.5000403@oracle.com> References: <537CF3D8.5000403@oracle.com> Message-ID: <1a7107f0-a93f-429f-bfff-c8625a31340d@default> Looks good. Thanks Markus -----Original Message----- From: Coleen Phillimore Sent: den 21 maj 2014 20:44 To: hotspot-dev developers Subject: [8u20] RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking Summary: Only prune metaspace virtual spaces at safepoint so walking them is safe outside a safepoint. This change has been in hs-rt for a week and backported cleanly. JFR performance in 8u20 needs this fix. open webrev at http://cr.openjdk.java.net/~coleenp/8038212_8u20/ bug link https://bugs.openjdk.java.net/browse/JDK-8038212 Thanks, Coleen From harold.seigel at oracle.com Wed May 21 19:07:16 2014 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 21 May 2014 15:07:16 -0400 Subject: [8u20] RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <1a7107f0-a93f-429f-bfff-c8625a31340d@default> References: <537CF3D8.5000403@oracle.com> <1a7107f0-a93f-429f-bfff-c8625a31340d@default> Message-ID: <537CF964.3010705@oracle.com> Hi Coleen, Looks good to me, also. Harold On 5/21/2014 2:52 PM, Markus Gr?nlund wrote: > Looks good. > > Thanks > Markus > > -----Original Message----- > From: Coleen Phillimore > Sent: den 21 maj 2014 20:44 > To: hotspot-dev developers > Subject: [8u20] RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking > > Summary: Only prune metaspace virtual spaces at safepoint so walking them is safe outside a safepoint. > > This change has been in hs-rt for a week and backported cleanly. JFR performance in 8u20 needs this fix. > > open webrev at http://cr.openjdk.java.net/~coleenp/8038212_8u20/ > bug link https://bugs.openjdk.java.net/browse/JDK-8038212 > > Thanks, > Coleen From coleen.phillimore at oracle.com Wed May 21 19:10:14 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 21 May 2014 15:10:14 -0400 Subject: [8u20] RFR 8038212: Method::is_valid_method() check has performance regression impact for stackwalking In-Reply-To: <537CF964.3010705@oracle.com> References: <537CF3D8.5000403@oracle.com> <1a7107f0-a93f-429f-bfff-c8625a31340d@default> <537CF964.3010705@oracle.com> Message-ID: <537CFA16.7000808@oracle.com> Thanks Markus and Harold. Coleen On 5/21/14, 3:07 PM, harold seigel wrote: > Hi Coleen, > > Looks good to me, also. > > Harold > > On 5/21/2014 2:52 PM, Markus Gr?nlund wrote: >> Looks good. >> >> Thanks >> Markus >> >> -----Original Message----- >> From: Coleen Phillimore >> Sent: den 21 maj 2014 20:44 >> To: hotspot-dev developers >> Subject: [8u20] RFR 8038212: Method::is_valid_method() check has >> performance regression impact for stackwalking >> >> Summary: Only prune metaspace virtual spaces at safepoint so walking >> them is safe outside a safepoint. >> >> This change has been in hs-rt for a week and backported cleanly. JFR >> performance in 8u20 needs this fix. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8038212_8u20/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8038212 >> >> Thanks, >> Coleen > From dmitry.samersoff at oracle.com Wed May 21 20:02:17 2014 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 22 May 2014 00:02:17 +0400 Subject: [rfc] hs_err log in temp readable for all In-Reply-To: <537A054F.6060507@redhat.com> References: <537A054F.6060507@redhat.com> Message-ID: <537D0649.3050202@oracle.com> Jiri, There are plenty of tools that rely on presence of hs_err* file to monitor java app etc. Readability of hs_err file is also important because typically, monitoring software run not under same user as a main Java app. It is probably better to keep hs_err file "group readable" only, but compatibility impact of such decision should be carefully evaluated. If world readable hs_err file is not acceptable for your system you can change umask to have restricted permissions. e.g.: > umask 0555 ... > kill -BUS ---------- 1 dms dms 26419 May 21 23:52 hs_err_pid15822.log -Dmitry On 2014-05-19 17:21, Jiri Vanek wrote: > Hi! > > I run JVM in read only directory in shared server and form time to time > it fails. However, I noted that hs_logs which then go to TMP direcotry, > are readable to all. > It may contains private information, and I think that this file should > be readable only to its owner. > > Attached webrevs fixes it: > for all hs_err logs - > http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/minimal/1/webrev/ > > for hs_log in tmp only - > http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/better/1/webrev/ > > > If I will be even more paranoid, then i think that hs_err{pid} name is > predictable, and can point to pre-created pipe. Maybe this 0600 file > should be placed in 0700 directory with salt in name. Or add the salt to > file itself (when created in tmp) > > I will be happy to adapt the patch for your needs. > > Best regards, > J. -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From vladimir.kozlov at oracle.com Wed May 21 21:31:13 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 21 May 2014 14:31:13 -0700 Subject: [8u20] RFR (M) 8042786: Proper fix for 8032566 Message-ID: <537D1B21.6040106@oracle.com> 8u20 backport request. The fix was pushed into jdk9 5 days ago and nightly testing shows no problems. Changes from jdk9 applied to 8u without conflicts. http://cr.openjdk.java.net/~kvn/8042786/webrev/ https://bugs.openjdk.java.net/browse/JDK-8042786 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/13818eb5243c Thanks, Vladimir From david.r.chase at oracle.com Wed May 21 23:13:41 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 21 May 2014 19:13:41 -0400 Subject: [8u20] RFR (L) 8037816+8043029+8043164 Message-ID: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> Summary: this is the hotspot side of port to Mavericks + clang, including turning on all the format annotations (so it won't go stale again) fixing all the scary bugs flagged by format warnings fixing quite a few of the harmless warnings flagged by format annotations disabling gcc's warnings for some files filled with harmless warnings (but with an ugly flag macro so people can clean it up incrementally) The intent of the backport is to reduce conflicts for other future backports, and to wipe clean all the this-can't-be-right mistakes identified by format warnings (mismatched number of args, that sort of thing). This webrev includes the original large patch, plus two follow-on patches for mistakes that were made. It's been tested as an 8u patch on Mountain Lion, 32- and 64-bit Linux. It has not been tested on Mavericks with clang 5.1 because whatever attempts have been made to port the build for 8 to Mavericks are not there yet (but this should make remove one obstacle, and the warnings enabled should help keep it from backsliding). This backport has tracked the 9 patch for most of its existence, and was diff'd and desk-checked to see that it matches as much as possible. Bugs: https://bugs.openjdk.java.net/browse/JDK-8037816 Fix for 8036122 breaks build with Xcode5/clang https://bugs.openjdk.java.net/browse/JDK-8043029 Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas https://bugs.openjdk.java.net/browse/JDK-8043164 Format warning in traceStream.hpp Webrev: http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-21a/ From vladimir.kozlov at oracle.com Wed May 21 23:47:39 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 21 May 2014 16:47:39 -0700 Subject: [8u20] RFR (L) 8037816+8043029+8043164 In-Reply-To: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> References: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> Message-ID: <537D3B1B.7070506@oracle.com> Looks good for backport but you based it on wrong repository. Should be: http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot Thanks, Vladimir On 5/21/14 4:13 PM, David Chase wrote: > Summary: this is the hotspot side of port to Mavericks + clang, including > turning on all the format annotations (so it won't go stale again) > fixing all the scary bugs flagged by format warnings > fixing quite a few of the harmless warnings flagged by format annotations > disabling gcc's warnings for some files filled with harmless warnings > (but with an ugly flag macro so people can clean it up incrementally) > > The intent of the backport is to reduce conflicts for other future backports, > and to wipe clean all the this-can't-be-right mistakes identified by format > warnings (mismatched number of args, that sort of thing). > > This webrev includes the original large patch, plus two follow-on patches > for mistakes that were made. It's been tested as an 8u patch on Mountain Lion, > 32- and 64-bit Linux. It has not been tested on Mavericks with clang 5.1 > because whatever attempts have been made to port the build for 8 to Mavericks > are not there yet (but this should make remove one obstacle, and the warnings > enabled should help keep it from backsliding). > > This backport has tracked the 9 patch for most of its existence, and was diff'd > and desk-checked to see that it matches as much as possible. > > Bugs: > https://bugs.openjdk.java.net/browse/JDK-8037816 Fix for 8036122 breaks build with Xcode5/clang > https://bugs.openjdk.java.net/browse/JDK-8043029 Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas > https://bugs.openjdk.java.net/browse/JDK-8043164 Format warning in traceStream.hpp > > Webrev: > http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-21a/ > From bengt.rutisson at oracle.com Thu May 22 07:33:05 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 22 May 2014 09:33:05 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps In-Reply-To: <79D2DC95-29ED-43AD-B772-CB113AD9A1FD@kodewerk.com> References: <537C7264.70200@oracle.com> <79D2DC95-29ED-43AD-B772-CB113AD9A1FD@kodewerk.com> Message-ID: <537DA831.20401@oracle.com> Hi Kirk, On 5/21/14 4:12 PM, Kirk Pepperdine wrote: > Hi Bengt, > > This change looks very useful. Thanks for looking at it and providing feedback! I was a bit worried about what you would think about the change :) Bengt > > Kind regards, > Kirk > > On May 21, 2014, at 11:31 AM, Bengt Rutisson wrote: > >> Hi everyone, >> >> Can I have some reviews for this patch: >> http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ >> >> https://bugs.openjdk.java.net/browse/JDK-8043607 >> >> This is mostly a GC change but I am sending it to the broader list since I added a method to ostream.hpp/cpp. We had this code duplicated in many places in the GC code: >> >> gclog_or_tty->date_stamp(PrintGCDateStamps); >> gclog_or_tty->stamp(PrintGCTimeStamps); >> >> And with my patch I had to add one more section to this code duplication. Instead I wrapped all three calls in a method called gclog_stamp(). It is a bit hard to know where to place that method. ostream seems like the wrong place for GC specific code, but there is already lots of GC specific code there and even gclog_or_tty is declared there. So I added the method there for now: >> >> void outputStream::gclog_stamp(uint gc_id) { >> date_stamp(PrintGCDateStamps); >> stamp(PrintGCTimeStamps); >> if (PrintGCID) { >> print("#%u: ", gc_id); >> } >> } >> >> Other than that this is a pure GC change. >> >> >> Background: >> >> With event based tracing we introduced a GC id field on all GC events. This id is a constantly increasing number associated with each GC. I figured we can use the this GC id as a decoration for the HotSpot GC logging similarly to what we do with PrintGCTimeStamps and PrintGCDateStamps. >> >> A nice side effect of this is that it will make it easy to map an event in the event tracing to an entry in the GC log. Currently this is impossible to do deterministically since the logging and event tracing use different timing. >> >> I propose to add a flag called PrintGCID. Here's an example log entry for GC number 4 running with -XX:+PrintGCID: >> >> #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] >> >> It is a decoration similar to the timestamps. Here's what it looks like if you add -XX:+PrintGCTimeStamps: >> >> 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] >> >> And here's what it will look like if you add -XX:+PrintGCDateStamps: >> >> 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] >> >> >> Here's a little longer example running ParallelGC: >> >> #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] >> #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] >> #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] >> #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 secs] >> #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] >> #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 secs] >> >> Note that for the concurrent GCs this gives a good way of seeing the concurrent action. Here is a G1 example: >> >> #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), 0,0383157 secs] >> #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), 0,0412220 secs] >> #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), 0,0426144 secs] >> #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), 0,1467633 secs] >> #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), 0,1627465 secs] >> #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), 0,2376728 secs] >> #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) 255M->269M(1004M), 0,2623779 secs] >> #7: [GC concurrent-root-region-scan-start] >> #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] >> #7: [GC concurrent-mark-start] >> #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), 0,3007405 secs] >> #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), 0,4063937 secs] >> #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), 0,7500609 secs] >> #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), 1,5464884 secs] >> #7: [GC concurrent-mark-end, 3,9464599 secs] >> #7: [GC remark, 0,0240462 secs] >> #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] >> #7: [GC concurrent-cleanup-start] >> #7: [GC concurrent-cleanup-end, 0,0000228 secs] >> #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), 2,2464423 secs] >> >> Notice how the complete concurrent cycle is decorated with id #7. That makes it possible to follow the concurrent work more easily and grep it out etc. >> >> >> Here's a similar thing for CMS where the concurrent cycle #22 has a young GC #23 in the middle of it: >> >> #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), 1,4558792 secs] >> #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] >> #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), 0,7909961 secs] >> #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] >> >> >> I think this is very helpful information to have, so I would like to push this with the new flag PrintGCID as true by default. I'm sure some parsers will break, but it is an easy work around to run with -XX:-PrintGCID to get the old format. It should even be pretty easy to do post processing on a log file to remove the GC id information. Having it on by default makes it possible for us to use this information and in particular be able to map it to the JFR events we see in MissionControl. >> >> I would also like to backport this to 8u40, but in that case with the default value of PrintGCID as false. >> >> Thanks, >> Bengt >> >> >> >> >> >> From jvanek at redhat.com Thu May 22 07:39:29 2014 From: jvanek at redhat.com (Jiri Vanek) Date: Thu, 22 May 2014 09:39:29 +0200 Subject: [rfc] hs_err log in temp readable for all In-Reply-To: <537D0649.3050202@oracle.com> References: <537A054F.6060507@redhat.com> <537D0649.3050202@oracle.com> Message-ID: <537DA9B1.2090507@redhat.com> On 05/21/2014 10:02 PM, Dmitry Samersoff wrote: Hello Dmitry, Thank you for explanation! J. > Jiri, > > There are plenty of tools that rely on presence of hs_err* file to > monitor java app etc. > > Readability of hs_err file is also important because typically, > monitoring software run not under same user as a main Java app. > > It is probably better to keep hs_err file "group readable" only, but > compatibility impact of such decision should be carefully evaluated. > > If world readable hs_err file is not acceptable for your system you can > change umask to have restricted permissions. > > e.g.: > >> umask 0555 > ... >> kill -BUS > > ---------- 1 dms dms 26419 May 21 23:52 hs_err_pid15822.log > > -Dmitry > > On 2014-05-19 17:21, Jiri Vanek wrote: >> Hi! >> >> I run JVM in read only directory in shared server and form time to time >> it fails. However, I noted that hs_logs which then go to TMP direcotry, >> are readable to all. >> It may contains private information, and I think that this file should >> be readable only to its owner. >> >> Attached webrevs fixes it: >> for all hs_err logs - >> http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/minimal/1/webrev/ >> >> for hs_log in tmp only - >> http://jvanek.fedorapeople.org/oracle/jdk9/stricter_permissions/better/1/webrev/ >> >> >> If I will be even more paranoid, then i think that hs_err{pid} name is >> predictable, and can point to pre-created pipe. Maybe this 0600 file >> should be placed in 0700 directory with salt in name. Or add the salt to >> file itself (when created in tmp) >> >> I will be happy to adapt the patch for your needs. >> >> Best regards, >> J. > > From bengt.rutisson at oracle.com Thu May 22 07:40:48 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 22 May 2014 09:40:48 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps In-Reply-To: <537CE5FB.9060003@oracle.com> References: <537C7264.70200@oracle.com> <537CE5FB.9060003@oracle.com> Message-ID: <537DAA00.2010104@oracle.com> Hi Jesper, On 5/21/14 7:44 PM, Jesper Wilhelmsson wrote: > Hi Bengt, > > Good change! Thanks for doing this! Thanks for looking at it. > > A few thoughts: > > * There are a few places that passes both a GCTimer and now a GCId. > Would it make sense to pass an info-object of some kind that contains > misc info that is used for logging/tracing? I'm not suggesting that > you should implement that in this change, just a thought for the future. Yes, I think that may be a good future cleanup to do. But would be good as a separate change, just as you point out. > > > * In genCollectdHeap.cpp you increase the gc id only if PrintGCDetails > is enabled. I assume this is the only place where a gc id is used when > running collectors that use GenCollectedHeap::do_collection(). If not, > the id created and used in other places would be different depending > on if PrintGCDetails was enabled or not. Yes, this is a work around for the strange way the logging for GenCollectedHeap works. We do completely different timing and logging for PrintGC and PrintGCDetails. That the logging is different is just strange. That the the timing is different is in my opinion a bug. The pause time reported is shorter if you run with PrintGC than if you run with PrintGCDetails. This mostly affect CMS. The logging for CMS has several other issues as well. I added one comment about that in the bug report. So, I think it would be good to give the CMS logging an overhaul, but that should definitely be a separate change :) And maybe we should hold off with that until the unified logging framework is in place. > > Anyway, looks good! Thanks! Bengt > Ship it! > /Jesper > > > Bengt Rutisson skrev 21/5/14 11:31: >> >> Hi everyone, >> >> Can I have some reviews for this patch: >> http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ >> >> https://bugs.openjdk.java.net/browse/JDK-8043607 >> >> This is mostly a GC change but I am sending it to the broader list >> since I added >> a method to ostream.hpp/cpp. We had this code duplicated in many >> places in the >> GC code: >> >> gclog_or_tty->date_stamp(PrintGCDateStamps); >> gclog_or_tty->stamp(PrintGCTimeStamps); >> >> And with my patch I had to add one more section to this code >> duplication. >> Instead I wrapped all three calls in a method called gclog_stamp(). >> It is a bit >> hard to know where to place that method. ostream seems like the wrong >> place for >> GC specific code, but there is already lots of GC specific code there >> and even >> gclog_or_tty is declared there. So I added the method there for now: >> >> void outputStream::gclog_stamp(uint gc_id) { >> date_stamp(PrintGCDateStamps); >> stamp(PrintGCTimeStamps); >> if (PrintGCID) { >> print("#%u: ", gc_id); >> } >> } >> >> Other than that this is a pure GC change. >> >> >> Background: >> >> With event based tracing we introduced a GC id field on all GC >> events. This id >> is a constantly increasing number associated with each GC. I figured >> we can use >> the this GC id as a decoration for the HotSpot GC logging similarly >> to what we >> do with PrintGCTimeStamps and PrintGCDateStamps. >> >> A nice side effect of this is that it will make it easy to map an >> event in the >> event tracing to an entry in the GC log. Currently this is impossible >> to do >> deterministically since the logging and event tracing use different >> timing. >> >> I propose to add a flag called PrintGCID. Here's an example log entry >> for GC >> number 4 running with -XX:+PrintGCID: >> >> #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), >> 0,2373420 secs] >> >> It is a decoration similar to the timestamps. Here's what it looks >> like if you >> add -XX:+PrintGCTimeStamps: >> >> 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), >> 0,2373420 >> secs] >> >> And here's what it will look like if you add -XX:+PrintGCDateStamps: >> >> 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation >> Pause) >> (young) 185M->212M(502M), 0,2373420 secs] >> >> >> Here's a little longer example running ParallelGC: >> >> #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] >> #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] >> #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] >> #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 secs] >> #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] >> #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 secs] >> >> Note that for the concurrent GCs this gives a good way of seeing the >> concurrent >> action. Here is a G1 example: >> >> #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), >> 0,0383157 secs] >> #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), >> 0,0412220 secs] >> #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), >> 0,0426144 secs] >> #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), >> 0,1467633 secs] >> #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), >> 0,1627465 secs] >> #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), >> 0,2376728 secs] >> #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) >> 255M->269M(1004M), >> 0,2623779 secs] >> #7: [GC concurrent-root-region-scan-start] >> #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] >> #7: [GC concurrent-mark-start] >> #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), >> 0,3007405 secs] >> #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), >> 0,4063937 secs] >> #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), >> 0,7500609 secs] >> #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), >> 1,5464884 secs] >> #7: [GC concurrent-mark-end, 3,9464599 secs] >> #7: [GC remark, 0,0240462 secs] >> #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] >> #7: [GC concurrent-cleanup-start] >> #7: [GC concurrent-cleanup-end, 0,0000228 secs] >> #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), >> 2,2464423 secs] >> >> Notice how the complete concurrent cycle is decorated with id #7. >> That makes it >> possible to follow the concurrent work more easily and grep it out etc. >> >> >> Here's a similar thing for CMS where the concurrent cycle #22 has a >> young GC #23 >> in the middle of it: >> >> #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), >> 1,4558792 secs] >> #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] >> #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), >> 0,7909961 secs] >> #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] >> >> >> I think this is very helpful information to have, so I would like to >> push this >> with the new flag PrintGCID as true by default. I'm sure some parsers >> will >> break, but it is an easy work around to run with -XX:-PrintGCID to >> get the old >> format. It should even be pretty easy to do post processing on a log >> file to >> remove the GC id information. Having it on by default makes it >> possible for us >> to use this information and in particular be able to map it to the >> JFR events we >> see in MissionControl. >> >> I would also like to backport this to 8u40, but in that case with the >> default >> value of PrintGCID as false. >> >> Thanks, >> Bengt >> >> >> >> >> >> From dan at danny.cz Thu May 22 08:19:04 2014 From: dan at danny.cz (Dan =?UTF-8?B?SG9yw6Fr?=) Date: Thu, 22 May 2014 10:19:04 +0200 Subject: PATCH: swapped usage of idx_t and bm_word_t types in parMarkBitMap.cpp Message-ID: <20140522101904.a9a5d06f13b9b13cf64fca4e@danny.cz> Hello, I've created a patch [1] to fix swapped usage of idx_t and bm_word_t types in hotspot/src/share/vm/gc_implementation/parallelScavenge/parMarkBitMap.cpp The code currently uses a pointer to idx_t as an argument to BitMap::set_map(bm_word_t* map) which fails on s390. For most platforms the idx_t and bm_word_t types are effectively the same, so the interchange is uncaught, but on s390 (32-bit) the size_t is "unsigned long" which makes it incompatible with "unsigned int". This a follow-up of my earlier s390 related patch [2] [1] http://fedora.danny.cz/openjdk/bitmap-1/webrev/ [2] http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-September/010913.html Note: I'm Red Hat employee (dhorak at redhat.com), so I should be covered by Red Hat's CLA for my contributions. With regards Dan From igor.ignatyev at oracle.com Thu May 22 08:47:44 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 22 May 2014 12:47:44 +0400 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot In-Reply-To: <537CC81C.6010905@oracle.com> References: <536B7CF0.6010508@oracle.com> <536B9E36.6090802@oracle.com> <536BA28D.7030808@oracle.com> <3503372.bSN5QEX8PY@mgerdin03> <5371F734.6090809@oracle.com> <537CC81C.6010905@oracle.com> Message-ID: <537DB9B0.8010200@oracle.com> Andrey, 1. You changed dozen of tests, have you tested your changes? 2. Your changes of year in copyright is wrong. it has to be $first_year, [$last_year, ], see Mark's email[1] for details. [1] http://mail.openjdk.java.net/pipermail/jdk7-dev/2010-May/001321.html Igor On 05/21/2014 07:37 PM, Andrey Zakharov wrote: > > On 13.05.2014 14:43, Andrey Zakharov wrote: >> Hi >> So here is trivial patch - >> removing ClassFileInstaller sun.hotspot.WhiteBox and adding >> main/othervm/bootclasspath >> where this needed >> >> Also, some tests are modified as >> - "-Xbootclasspath/a:.", >> + "-Xbootclasspath/a:" + System.getProperty("test.classes"), >> >> Thanks. > webrev: http://cr.openjdk.java.net/~jwilhelm/8011397/webrev.02/ > bug: https://bugs.openjdk.java.net/browse/JDK-8011397 > Thanks. > >> >> On 09.05.2014 12:13, Mikael Gerdin wrote: >>> On Thursday 08 May 2014 19.28.13 Igor Ignatyev wrote: >>>> // cc'ing hotspot-dev instaed of compiler, runtime and gc lists. >>>> >>>> On 05/08/2014 07:09 PM, Filipp Zhinkin wrote: >>>>> Andrey, >>>>> >>>>> I've CC'ed compiler and runtime mailing list, because you're changes >>>>> affect test for other components as too. >>>>> >>>>> I don't like your solution (but I'm not a reviewer, so treat my words >>>>> just as suggestion), >>>>> because we'll have to write more meta information for each test and it >>>>> is very easy to >>>>> forget to install WhiteBoxPermission if you don't test your test with >>>>> some security manager. >>>>> >>>>> From my point of view, it will be better to extend >>>>> ClassFileInstaller >>>>> >>>>> so it will copy not only >>>>> a class whose name was passed as an arguments, but also all inner >>>>> classes of that class. >>>>> And if someone want copy only specified class without inner classes, >>>>> then some option >>>>> could be added to ClassFileInstaller to force such behaviour. >>> Perhaps this is a good time to get rid of ClassFileInstaller altogether? >>> >>> https://bugs.openjdk.java.net/browse/JDK-8009117 >>> >>> The reason for its existence is that the WhiteBox class needs to be >>> on the >>> boot class path. >>> If we can live with having all the test's classes on the boot class >>> path then >>> we could use the /bootclasspath option in jtreg as stated in the RFE. >>> >>> /Mikael >>> >>>>> Thanks, >>>>> Filipp. >>>>> >>>>> On 05/08/2014 04:47 PM, Andrey Zakharov wrote: >>>>>> Hi! >>>>>> Suggesting patch with fixes for >>>>>> https://bugs.openjdk.java.net/browse/JDK-8011397 >>>>>> >>>>>> webrev: >>>>>> https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz >>>>>> patch: >>>>>> https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPer >>>>>> >>>>>> mission >>>>>> >>>>>> >>>>>> >>>>>> Thanks. >> > From dan at danny.cz Thu May 22 08:59:03 2014 From: dan at danny.cz (Dan =?UTF-8?B?SG9yw6Fr?=) Date: Thu, 22 May 2014 10:59:03 +0200 Subject: PATCH: max_heap_for_compressed_oops() declared with size_t, but defined with uintx Message-ID: <20140522105903.97cf9b229b44ba4dd42ee651@danny.cz> Hello, I've created a patch [1] to fix usage of uintx where size_t type is expected in hotspot/src/share/vm/runtime/arguments.cpp The Arguments::max_heap_for_compressed_oops() method is declared in arguments.hpp as size_t max_heap_for_compressed_oops() but later defined in arguments.cpp as uintx max_heap_for_compressed_oops() For most platforms the uintx and size_t types are effectively the same, so the interchange is uncaught, but on s390 (32-bit) the size_t is "unsigned long" which makes it incompatible with "unsigned int". This a follow-up of my earlier s390 related patch [2] [1] http://fedora.danny.cz/openjdk/argument/webrev/ [2] http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-September/010913.html Note: I'm Red Hat employee (dhorak at redhat.com), so I should be covered by Red Hat's CLA for my contributions. With regards Dan From thomas.schatzl at oracle.com Thu May 22 09:23:48 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 22 May 2014 11:23:48 +0200 Subject: PATCH: swapped usage of idx_t and bm_word_t types in parMarkBitMap.cpp In-Reply-To: <20140522101904.a9a5d06f13b9b13cf64fca4e@danny.cz> References: <20140522101904.a9a5d06f13b9b13cf64fca4e@danny.cz> Message-ID: <1400750628.2617.11.camel@cirrus> Hi Dan, On Thu, 2014-05-22 at 10:19 +0200, Dan Hor?k wrote: > Hello, > > I've created a patch [1] to fix swapped usage of idx_t and bm_word_t > types in > hotspot/src/share/vm/gc_implementation/parallelScavenge/parMarkBitMap.cpp > > The code currently uses a pointer to idx_t as an argument to > BitMap::set_map(bm_word_t* map) which fails on s390. > > For most platforms the idx_t and bm_word_t types are effectively the > same, so the interchange is uncaught, but on s390 (32-bit) the size_t is > "unsigned long" which makes it incompatible with "unsigned int". > > This a follow-up of my earlier s390 related patch [2] > > [1] http://fedora.danny.cz/openjdk/bitmap-1/webrev/ > [2] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-September/010913.html > > Note: I'm Red Hat employee (dhorak at redhat.com), so I should be covered > by Red Hat's CLA for my contributions. > Created https://bugs.openjdk.java.net/browse/JDK-8043722 . I roughly looked at other similar changes, shouldn't line 129 in g1/concurrentMark.cpp changed too? 129 _bm.set_map((uintptr_t*)_virtual_space.low()); @Compiler team: what about the instantiations of MethodLivenessResult? I.e. in methodLiveness.cpp:476 and 1001 there are two other uintptr_t* casts. Thanks, Thomas From thomas.schatzl at oracle.com Thu May 22 09:24:16 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 22 May 2014 11:24:16 +0200 Subject: PATCH: max_heap_for_compressed_oops() declared with size_t, but defined with uintx In-Reply-To: <20140522105903.97cf9b229b44ba4dd42ee651@danny.cz> References: <20140522105903.97cf9b229b44ba4dd42ee651@danny.cz> Message-ID: <1400750656.2617.12.camel@cirrus> Hi Dan, On Thu, 2014-05-22 at 10:59 +0200, Dan Hor?k wrote: > Hello, > > I've created a patch [1] to fix usage of uintx where size_t type is > expected in hotspot/src/share/vm/runtime/arguments.cpp > > The Arguments::max_heap_for_compressed_oops() method is declared in > arguments.hpp as > size_t max_heap_for_compressed_oops() > but later defined in arguments.cpp as > uintx max_heap_for_compressed_oops() > > For most platforms the uintx and size_t types are effectively the > same, so the interchange is uncaught, but on s390 (32-bit) the size_t is > "unsigned long" which makes it incompatible with "unsigned int". > > This a follow-up of my earlier s390 related patch [2] > > [1] http://fedora.danny.cz/openjdk/argument/webrev/ > [2] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-September/010913.html > > Note: I'm Red Hat employee (dhorak at redhat.com), so I should be covered > by Red Hat's CLA for my contributions. created https://bugs.openjdk.java.net/browse/JDK-8043723 . Reviewed. I can push the change. Looks good, Thomas From dan at danny.cz Thu May 22 09:59:47 2014 From: dan at danny.cz (Dan =?UTF-8?B?SG9yw6Fr?=) Date: Thu, 22 May 2014 11:59:47 +0200 Subject: PATCH: swapped usage of idx_t and bm_word_t types in parMarkBitMap.cpp In-Reply-To: <1400750628.2617.11.camel@cirrus> References: <20140522101904.a9a5d06f13b9b13cf64fca4e@danny.cz> <1400750628.2617.11.camel@cirrus> Message-ID: <20140522115947.5ce5ad198a35e8155a9c1f59@danny.cz> On Thu, 22 May 2014 11:23:48 +0200 Thomas Schatzl wrote: > Hi Dan, > > On Thu, 2014-05-22 at 10:19 +0200, Dan Hor?k wrote: > > Hello, > > > > I've created a patch [1] to fix swapped usage of idx_t and bm_word_t > > types in > > hotspot/src/share/vm/gc_implementation/parallelScavenge/parMarkBitMap.cpp > > > > The code currently uses a pointer to idx_t as an argument to > > BitMap::set_map(bm_word_t* map) which fails on s390. > > > > For most platforms the idx_t and bm_word_t types are effectively the > > same, so the interchange is uncaught, but on s390 (32-bit) the > > size_t is "unsigned long" which makes it incompatible with > > "unsigned int". > > > > This a follow-up of my earlier s390 related patch [2] > > > > [1] http://fedora.danny.cz/openjdk/bitmap-1/webrev/ > > [2] > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-September/010913.html > > > > Note: I'm Red Hat employee (dhorak at redhat.com), so I should be > > covered by Red Hat's CLA for my contributions. > > > > Created https://bugs.openjdk.java.net/browse/JDK-8043722 . > > I roughly looked at other similar changes, shouldn't line 129 in > g1/concurrentMark.cpp changed too? > > 129 _bm.set_map((uintptr_t*)_virtual_space.low()); > > @Compiler team: what about the instantiations of MethodLivenessResult? > > I.e. in methodLiveness.cpp:476 and 1001 there are two other uintptr_t* > casts. they don't fail the build because bm_word_t is uintptr_t, but in my opinion they should be changed too, at least for consistency Dan From stefan.johansson at oracle.com Thu May 22 11:23:52 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 22 May 2014 13:23:52 +0200 Subject: PATCH: max_heap_for_compressed_oops() declared with size_t, but defined with uintx In-Reply-To: <1400750656.2617.12.camel@cirrus> References: <20140522105903.97cf9b229b44ba4dd42ee651@danny.cz> <1400750656.2617.12.camel@cirrus> Message-ID: <537DDE48.2000507@oracle.com> Looks good, Stefan On 2014-05-22 11:24, Thomas Schatzl wrote: > Hi Dan, > > On Thu, 2014-05-22 at 10:59 +0200, Dan Hor?k wrote: >> Hello, >> >> I've created a patch [1] to fix usage of uintx where size_t type is >> expected in hotspot/src/share/vm/runtime/arguments.cpp >> >> The Arguments::max_heap_for_compressed_oops() method is declared in >> arguments.hpp as >> size_t max_heap_for_compressed_oops() >> but later defined in arguments.cpp as >> uintx max_heap_for_compressed_oops() >> >> For most platforms the uintx and size_t types are effectively the >> same, so the interchange is uncaught, but on s390 (32-bit) the size_t is >> "unsigned long" which makes it incompatible with "unsigned int". >> >> This a follow-up of my earlier s390 related patch [2] >> >> [1] http://fedora.danny.cz/openjdk/argument/webrev/ >> [2] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-September/010913.html >> >> Note: I'm Red Hat employee (dhorak at redhat.com), so I should be covered >> by Red Hat's CLA for my contributions. > created https://bugs.openjdk.java.net/browse/JDK-8043723 . > > Reviewed. I can push the change. > > Looks good, > Thomas > From david.r.chase at oracle.com Thu May 22 12:40:26 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 22 May 2014 08:40:26 -0400 Subject: [8u20] RFR (L) 8037816+8043029+8043164 In-Reply-To: <537D3B1B.7070506@oracle.com> References: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> <537D3B1B.7070506@oracle.com> Message-ID: And how many reviews are needed for a backport (especially of this size)? On 2014-05-21, at 7:47 PM, Vladimir Kozlov wrote: > Looks good for backport but you based it on wrong repository. Should be: > > http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot > > Thanks, > Vladimir > > On 5/21/14 4:13 PM, David Chase wrote: >> Summary: this is the hotspot side of port to Mavericks + clang, including >> turning on all the format annotations (so it won't go stale again) >> fixing all the scary bugs flagged by format warnings >> fixing quite a few of the harmless warnings flagged by format annotations >> disabling gcc's warnings for some files filled with harmless warnings >> (but with an ugly flag macro so people can clean it up incrementally) >> >> The intent of the backport is to reduce conflicts for other future backports, >> and to wipe clean all the this-can't-be-right mistakes identified by format >> warnings (mismatched number of args, that sort of thing). >> >> This webrev includes the original large patch, plus two follow-on patches >> for mistakes that were made. It's been tested as an 8u patch on Mountain Lion, >> 32- and 64-bit Linux. It has not been tested on Mavericks with clang 5.1 >> because whatever attempts have been made to port the build for 8 to Mavericks >> are not there yet (but this should make remove one obstacle, and the warnings >> enabled should help keep it from backsliding). >> >> This backport has tracked the 9 patch for most of its existence, and was diff'd >> and desk-checked to see that it matches as much as possible. >> >> Bugs: >> https://bugs.openjdk.java.net/browse/JDK-8037816 Fix for 8036122 breaks build with Xcode5/clang >> https://bugs.openjdk.java.net/browse/JDK-8043029 Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas >> https://bugs.openjdk.java.net/browse/JDK-8043164 Format warning in traceStream.hpp >> >> Webrev: >> http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-21a/ >> From daniel.daugherty at oracle.com Thu May 22 14:28:59 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 22 May 2014 08:28:59 -0600 Subject: [8u20] RFR (L) 8037816+8043029+8043164 In-Reply-To: References: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> <537D3B1B.7070506@oracle.com> Message-ID: <537E09AB.7070806@oracle.com> For large or risky backports, I usually request a total of three reviewers. You have one (R)eviewer so the others can be either. Dan On 5/22/14 6:40 AM, David Chase wrote: > And how many reviews are needed for a backport (especially of this size)? > > On 2014-05-21, at 7:47 PM, Vladimir Kozlov wrote: > >> Looks good for backport but you based it on wrong repository. Should be: >> >> http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot >> >> Thanks, >> Vladimir >> >> On 5/21/14 4:13 PM, David Chase wrote: >>> Summary: this is the hotspot side of port to Mavericks + clang, including >>> turning on all the format annotations (so it won't go stale again) >>> fixing all the scary bugs flagged by format warnings >>> fixing quite a few of the harmless warnings flagged by format annotations >>> disabling gcc's warnings for some files filled with harmless warnings >>> (but with an ugly flag macro so people can clean it up incrementally) >>> >>> The intent of the backport is to reduce conflicts for other future backports, >>> and to wipe clean all the this-can't-be-right mistakes identified by format >>> warnings (mismatched number of args, that sort of thing). >>> >>> This webrev includes the original large patch, plus two follow-on patches >>> for mistakes that were made. It's been tested as an 8u patch on Mountain Lion, >>> 32- and 64-bit Linux. It has not been tested on Mavericks with clang 5.1 >>> because whatever attempts have been made to port the build for 8 to Mavericks >>> are not there yet (but this should make remove one obstacle, and the warnings >>> enabled should help keep it from backsliding). >>> >>> This backport has tracked the 9 patch for most of its existence, and was diff'd >>> and desk-checked to see that it matches as much as possible. >>> >>> Bugs: >>> https://bugs.openjdk.java.net/browse/JDK-8037816 Fix for 8036122 breaks build with Xcode5/clang >>> https://bugs.openjdk.java.net/browse/JDK-8043029 Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas >>> https://bugs.openjdk.java.net/browse/JDK-8043164 Format warning in traceStream.hpp >>> >>> Webrev: >>> http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-21a/ >>> From mikael.vidstedt at oracle.com Thu May 22 17:36:05 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 22 May 2014 10:36:05 -0700 Subject: RFR (XS): 8043206: Fix signed vs. unsigned comparison warning in copy_sparc.hpp Message-ID: <537E3585.3050705@oracle.com> Please review the following small fix which adds a cast to resolve a unsigned/signed comparison (and remove the corresponding warning gcc emits). Bug: https://bugs.openjdk.java.net/browse/JDK-8043206 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8043206/webrev.00/webrev/ Cheers, Mikael From lois.foltan at oracle.com Thu May 22 17:42:23 2014 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 22 May 2014 13:42:23 -0400 Subject: RFR (XS): 8043206: Fix signed vs. unsigned comparison warning in copy_sparc.hpp In-Reply-To: <537E3585.3050705@oracle.com> References: <537E3585.3050705@oracle.com> Message-ID: <537E36FF.6060401@oracle.com> Looks good, just the copyright needs to be updated. Lois On 5/22/2014 1:36 PM, Mikael Vidstedt wrote: > > Please review the following small fix which adds a cast to resolve a > unsigned/signed comparison (and remove the corresponding warning gcc > emits). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8043206 > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8043206/webrev.00/webrev/ > > Cheers, > Mikael > From coleen.phillimore at oracle.com Thu May 22 17:45:48 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 22 May 2014 13:45:48 -0400 Subject: RFR (XS): 8043206: Fix signed vs. unsigned comparison warning in copy_sparc.hpp In-Reply-To: <537E3585.3050705@oracle.com> References: <537E3585.3050705@oracle.com> Message-ID: <537E37CC.40100@oracle.com> Looks good. Coleen On 5/22/14, 1:36 PM, Mikael Vidstedt wrote: > > Please review the following small fix which adds a cast to resolve a > unsigned/signed comparison (and remove the corresponding warning gcc > emits). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8043206 > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8043206/webrev.00/webrev/ > > Cheers, > Mikael > From vladimir.kozlov at oracle.com Thu May 22 17:49:38 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 22 May 2014 10:49:38 -0700 Subject: RFR (XS): 8043206: Fix signed vs. unsigned comparison warning in copy_sparc.hpp In-Reply-To: <537E3585.3050705@oracle.com> References: <537E3585.3050705@oracle.com> Message-ID: <537E38B2.1020809@oracle.com> Good. Vladimir On 5/22/14 10:36 AM, Mikael Vidstedt wrote: > > Please review the following small fix which adds a cast to resolve a > unsigned/signed comparison (and remove the corresponding warning gcc > emits). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8043206 > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8043206/webrev.00/webrev/ > > Cheers, > Mikael > From mikael.vidstedt at oracle.com Thu May 22 18:39:57 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 22 May 2014 11:39:57 -0700 Subject: [8u20] RFR (S): 8042059: Various fixes to linux/sparc Message-ID: <537E447D.6010601@oracle.com> Please review this 8u20 backport request. The fix was pushed to jdk9 a few weeks ago and has not shown any problems. The change from jdk9 applies to jdk8u/hs-dev without conflicts. Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8042059-8uhsdev/webrev.00/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8042059 jdk9 change: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/c884ec3ea87f Thanks, Mikael From christian.thalinger at oracle.com Thu May 22 18:50:46 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 22 May 2014 11:50:46 -0700 Subject: [8u20] RFR (L) 8037816+8043029+8043164 In-Reply-To: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> References: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> Message-ID: <8B272ACF-BBE5-40E5-8F1A-216C5CFE7056@oracle.com> Looks good. I filed a few RFEs for cleanups. On May 21, 2014, at 4:13 PM, David Chase wrote: > Summary: this is the hotspot side of port to Mavericks + clang, including > turning on all the format annotations (so it won't go stale again) > fixing all the scary bugs flagged by format warnings > fixing quite a few of the harmless warnings flagged by format annotations > disabling gcc's warnings for some files filled with harmless warnings > (but with an ugly flag macro so people can clean it up incrementally) > > The intent of the backport is to reduce conflicts for other future backports, > and to wipe clean all the this-can't-be-right mistakes identified by format > warnings (mismatched number of args, that sort of thing). > > This webrev includes the original large patch, plus two follow-on patches > for mistakes that were made. It's been tested as an 8u patch on Mountain Lion, > 32- and 64-bit Linux. It has not been tested on Mavericks with clang 5.1 > because whatever attempts have been made to port the build for 8 to Mavericks > are not there yet (but this should make remove one obstacle, and the warnings > enabled should help keep it from backsliding). > > This backport has tracked the 9 patch for most of its existence, and was diff'd > and desk-checked to see that it matches as much as possible. > > Bugs: > https://bugs.openjdk.java.net/browse/JDK-8037816 Fix for 8036122 breaks build with Xcode5/clang > https://bugs.openjdk.java.net/browse/JDK-8043029 Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas > https://bugs.openjdk.java.net/browse/JDK-8043164 Format warning in traceStream.hpp > > Webrev: > http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-21a/ > From christian.thalinger at oracle.com Thu May 22 18:54:33 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 22 May 2014 11:54:33 -0700 Subject: [8u20] RFR (S): 8042059: Various fixes to linux/sparc In-Reply-To: <537E447D.6010601@oracle.com> References: <537E447D.6010601@oracle.com> Message-ID: Good. On May 22, 2014, at 11:39 AM, Mikael Vidstedt wrote: > > Please review this 8u20 backport request. The fix was pushed to jdk9 a few weeks ago and has not shown any problems. > > The change from jdk9 applies to jdk8u/hs-dev without conflicts. > > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8042059-8uhsdev/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8042059 > jdk9 change: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/c884ec3ea87f > > Thanks, > Mikael > From coleen.phillimore at oracle.com Thu May 22 19:14:00 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 22 May 2014 15:14:00 -0400 Subject: [8u20] RFR (L) 8037816+8043029+8043164 In-Reply-To: <8B272ACF-BBE5-40E5-8F1A-216C5CFE7056@oracle.com> References: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> <8B272ACF-BBE5-40E5-8F1A-216C5CFE7056@oracle.com> Message-ID: <537E4C78.70503@oracle.com> This looks good. Coleen On 5/22/14, 2:50 PM, Christian Thalinger wrote: > Looks good. I filed a few RFEs for cleanups. > > On May 21, 2014, at 4:13 PM, David Chase wrote: > >> Summary: this is the hotspot side of port to Mavericks + clang, including >> turning on all the format annotations (so it won't go stale again) >> fixing all the scary bugs flagged by format warnings >> fixing quite a few of the harmless warnings flagged by format annotations >> disabling gcc's warnings for some files filled with harmless warnings >> (but with an ugly flag macro so people can clean it up incrementally) >> >> The intent of the backport is to reduce conflicts for other future backports, >> and to wipe clean all the this-can't-be-right mistakes identified by format >> warnings (mismatched number of args, that sort of thing). >> >> This webrev includes the original large patch, plus two follow-on patches >> for mistakes that were made. It's been tested as an 8u patch on Mountain Lion, >> 32- and 64-bit Linux. It has not been tested on Mavericks with clang 5.1 >> because whatever attempts have been made to port the build for 8 to Mavericks >> are not there yet (but this should make remove one obstacle, and the warnings >> enabled should help keep it from backsliding). >> >> This backport has tracked the 9 patch for most of its existence, and was diff'd >> and desk-checked to see that it matches as much as possible. >> >> Bugs: >> https://bugs.openjdk.java.net/browse/JDK-8037816 Fix for 8036122 breaks build with Xcode5/clang >> https://bugs.openjdk.java.net/browse/JDK-8043029 Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas >> https://bugs.openjdk.java.net/browse/JDK-8043164 Format warning in traceStream.hpp >> >> Webrev: >> http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-21a/ >> From zhengyu.gu at oracle.com Thu May 22 19:19:14 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Thu, 22 May 2014 15:19:14 -0400 Subject: RFR (L) 8028541: Native Memory Tracking enhancement Message-ID: <537E4DB2.5020602@oracle.com> This is significant rework of native memory tracking introduced in earlier releases. The goal of this enhancement is to improve scalability, from both tracking memory and CPU usage perspectives, so it can scale well with increased memory allocation in large applications. The enhancement is mainly focused on malloc memory tracking, whose activities are several magnitude higher than virtual memory, and was the main bottleneck in early implementation. Instead of using book keeping records for tracking malloc activities, new implementation co-locates tracking data along side with user data by using a prefixed header. The header size is 8 bytes on 32-bit systems and 16 bytes on 64-bit systems, which ensure that user data also align properly. Virtual memory tracking still uses book keeping records, and ThreadCritical lock is always acquired to alter the records and related data structures. Summary tracking data is maintained in static data structures, via atomic operations. Malloc detail tracking call stacks are maintained in a lock free hashtable. The key improvements: 1. Up-to-date tracking report. 2. Detail tracking now shows multiple call frames. Number of frames is compilation time decision, currently default to 4. 3. Malloc tracking is lock free. 4. Tracking summary is reported in hs_err file when native memory tracking is enabled. 5. Query is faster, uses little memory and need a very little process. The drawback is that, malloc tracking header is always needed if native memory tracking has ever been enabled, even after tracking is shutdown. Impacts: The most noticeable impact for JVM developers, is that Arena now also take memory type as constructor parameter, besides the new operators. Arena* a = new (mtCode) Arena() => Arena* a = new (mtCode) Arena(mtCode) The webrev shows modification of about 60 files, but most of them are due to tracking API changes, mainly due to tracking stack, now, is an object, vs. a single pc. The most important files for this implementations are: memTracker.hpp/cpp mallocTracker.hpp/cpp and mallocTracker.inline.hpp virtualMemoryTracker.hpp/cpp mallocSiteTable.hpp/cpp allocationSite.hpp nativeCallStack.hpp/cpp linkedlist.hpp Tests: - JPRT - NMT test suite - vm.quick.testlist - Kitchensink stability test for 16+ days - FMW Bug: https://bugs.openjdk.java.net/browse/JDK-8028541 Webrev: http://cr.openjdk.java.net/~zgu/8028541/webrev.00/ Thanks, -Zhengyu From igor.veresov at oracle.com Thu May 22 19:22:41 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Thu, 22 May 2014 09:22:41 -1000 Subject: [8u20] RFR (L) 8037816+8043029+8043164 In-Reply-To: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> References: <1782EAF9-D66E-4088-97EF-DDB43B22D8DB@oracle.com> Message-ID: <8E892D5C-F27A-40E5-92AF-83A50D9E32E9@oracle.com> Looks alright! igor On May 21, 2014, at 1:13 PM, David Chase wrote: > Summary: this is the hotspot side of port to Mavericks + clang, including > turning on all the format annotations (so it won't go stale again) > fixing all the scary bugs flagged by format warnings > fixing quite a few of the harmless warnings flagged by format annotations > disabling gcc's warnings for some files filled with harmless warnings > (but with an ugly flag macro so people can clean it up incrementally) > > The intent of the backport is to reduce conflicts for other future backports, > and to wipe clean all the this-can't-be-right mistakes identified by format > warnings (mismatched number of args, that sort of thing). > > This webrev includes the original large patch, plus two follow-on patches > for mistakes that were made. It's been tested as an 8u patch on Mountain Lion, > 32- and 64-bit Linux. It has not been tested on Mavericks with clang 5.1 > because whatever attempts have been made to port the build for 8 to Mavericks > are not there yet (but this should make remove one obstacle, and the warnings > enabled should help keep it from backsliding). > > This backport has tracked the 9 patch for most of its existence, and was diff'd > and desk-checked to see that it matches as much as possible. > > Bugs: > https://bugs.openjdk.java.net/browse/JDK-8037816 Fix for 8036122 breaks build with Xcode5/clang > https://bugs.openjdk.java.net/browse/JDK-8043029 Change 8037816 breaks HS build with older GCC versions which don't support diagnostic pragmas > https://bugs.openjdk.java.net/browse/JDK-8043164 Format warning in traceStream.hpp > > Webrev: > http://cr.openjdk.java.net/~drchase/8037816/webrev-8u.05-21a/ > From mikael.vidstedt at oracle.com Thu May 22 19:46:35 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 22 May 2014 12:46:35 -0700 Subject: [8u20] RFR (XS): 8022070: Compilation error in stubGenerator_sparc.cpp with some compilers Message-ID: <537E541B.7060706@oracle.com> Please review this 8u20 backport request. The fix was pushed to jdk9 a few weeks ago and has not shown any problems. The change from jdk9 applies to jdk8u/hs-dev without conflicts. Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8022070-8uhsdev/webrev.00/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8022070 jdk9 change: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e3a56821c4e3 Thanks, Mikael From lois.foltan at oracle.com Thu May 22 20:11:13 2014 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 22 May 2014 16:11:13 -0400 Subject: [8u20] RFR (XS): 8022070: Compilation error in stubGenerator_sparc.cpp with some compilers In-Reply-To: <537E541B.7060706@oracle.com> References: <537E541B.7060706@oracle.com> Message-ID: <537E59E1.7060302@oracle.com> Looks good. Lois On 5/22/2014 3:46 PM, Mikael Vidstedt wrote: > > Please review this 8u20 backport request. The fix was pushed to jdk9 a > few weeks ago and has not shown any problems. > > The change from jdk9 applies to jdk8u/hs-dev without conflicts. > > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8022070-8uhsdev/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8022070 > jdk9 change: > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e3a56821c4e3 > > Thanks, > Mikael > From vladimir.kozlov at oracle.com Thu May 22 21:06:54 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 22 May 2014 14:06:54 -0700 Subject: [8u20] RFR (XS): 8022070: Compilation error in stubGenerator_sparc.cpp with some compilers In-Reply-To: <537E541B.7060706@oracle.com> References: <537E541B.7060706@oracle.com> Message-ID: <537E66EE.8090609@oracle.com> Good. Vladimir On 5/22/14 12:46 PM, Mikael Vidstedt wrote: > > Please review this 8u20 backport request. The fix was pushed to jdk9 a > few weeks ago and has not shown any problems. > > The change from jdk9 applies to jdk8u/hs-dev without conflicts. > > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8022070-8uhsdev/webrev.00/webrev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8022070 > jdk9 change: > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/e3a56821c4e3 > > Thanks, > Mikael > From daniel.daugherty at oracle.com Fri May 23 04:18:34 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 22 May 2014 22:18:34 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537A0E1C.4070605@oracle.com> References: <537A0E1C.4070605@oracle.com> Message-ID: <537ECC1A.1030409@oracle.com> Zhengyu is tied up with some other work so I've taken on this fix. Here's the webrev URL for the next round: http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ The fix has been tested with vm.quick on all Aurora Adhoc platforms. The new test has been run with the fix via JPRT and passes on all JPRT platforms. The new test has also been run without the fix and fails on most platforms. Since the default sample size is just 30, it is possible to get 30 runs in a row without failing. Thanks, in advance, for any comments, questions or suggestions. Dan On 5/19/14 7:58 AM, Zhengyu Gu wrote: > This is a simple fix for incorrect lock state. > > The timing on setting thread's pending monitor can result stack trace > dump reporting incorrect lock state. The solution is to check the > monitor's owner, if the owner is other than the current thread, then > the monitor, is or is in process of being, set the pending monitor of > current thread. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 > Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ > > > Thanks, > > -Zhengyu > From bengt.rutisson at oracle.com Fri May 23 05:00:44 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 23 May 2014 07:00:44 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps In-Reply-To: <537DAA00.2010104@oracle.com> References: <537C7264.70200@oracle.com> <537CE5FB.9060003@oracle.com> <537DAA00.2010104@oracle.com> Message-ID: <537ED5FC.6080702@oracle.com> Hi Jesper and everyone else, After explaining the problem with GenCollectedHeap in my last email I realized that a different approach would be to just assume that the next GC id will be the right one when we start the logging before we have incremented the GC id. I think that is a nicer way of doing it than "wasting" a GC id as I did before. Here's an updated webrev with that changed: http://cr.openjdk.java.net/~brutisso/8043607/webrev.01/ Here's the diff compared to the last webrev: http://cr.openjdk.java.net/~brutisso/8043607/webrev.00-01-diff/ Thanks, Bengt On 5/22/14 9:40 AM, Bengt Rutisson wrote: > > Hi Jesper, > > On 5/21/14 7:44 PM, Jesper Wilhelmsson wrote: >> Hi Bengt, >> >> Good change! Thanks for doing this! > > Thanks for looking at it. > >> >> A few thoughts: >> >> * There are a few places that passes both a GCTimer and now a GCId. >> Would it make sense to pass an info-object of some kind that contains >> misc info that is used for logging/tracing? I'm not suggesting that >> you should implement that in this change, just a thought for the future. > > Yes, I think that may be a good future cleanup to do. But would be > good as a separate change, just as you point out. > >> >> >> * In genCollectdHeap.cpp you increase the gc id only if >> PrintGCDetails is enabled. I assume this is the only place where a gc >> id is used when running collectors that use >> GenCollectedHeap::do_collection(). If not, the id created and used in >> other places would be different depending on if PrintGCDetails was >> enabled or not. > > Yes, this is a work around for the strange way the logging for > GenCollectedHeap works. We do completely different timing and logging > for PrintGC and PrintGCDetails. That the logging is different is just > strange. That the the timing is different is in my opinion a bug. The > pause time reported is shorter if you run with PrintGC than if you run > with PrintGCDetails. > > This mostly affect CMS. The logging for CMS has several other issues > as well. I added one comment about that in the bug report. So, I think > it would be good to give the CMS logging an overhaul, but that should > definitely be a separate change :) And maybe we should hold off with > that until the unified logging framework is in place. > >> >> Anyway, looks good! > Thanks! > > Bengt > >> Ship it! >> /Jesper >> >> >> Bengt Rutisson skrev 21/5/14 11:31: >>> >>> Hi everyone, >>> >>> Can I have some reviews for this patch: >>> http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ >>> >>> https://bugs.openjdk.java.net/browse/JDK-8043607 >>> >>> This is mostly a GC change but I am sending it to the broader list >>> since I added >>> a method to ostream.hpp/cpp. We had this code duplicated in many >>> places in the >>> GC code: >>> >>> gclog_or_tty->date_stamp(PrintGCDateStamps); >>> gclog_or_tty->stamp(PrintGCTimeStamps); >>> >>> And with my patch I had to add one more section to this code >>> duplication. >>> Instead I wrapped all three calls in a method called gclog_stamp(). >>> It is a bit >>> hard to know where to place that method. ostream seems like the >>> wrong place for >>> GC specific code, but there is already lots of GC specific code >>> there and even >>> gclog_or_tty is declared there. So I added the method there for now: >>> >>> void outputStream::gclog_stamp(uint gc_id) { >>> date_stamp(PrintGCDateStamps); >>> stamp(PrintGCTimeStamps); >>> if (PrintGCID) { >>> print("#%u: ", gc_id); >>> } >>> } >>> >>> Other than that this is a pure GC change. >>> >>> >>> Background: >>> >>> With event based tracing we introduced a GC id field on all GC >>> events. This id >>> is a constantly increasing number associated with each GC. I figured >>> we can use >>> the this GC id as a decoration for the HotSpot GC logging similarly >>> to what we >>> do with PrintGCTimeStamps and PrintGCDateStamps. >>> >>> A nice side effect of this is that it will make it easy to map an >>> event in the >>> event tracing to an entry in the GC log. Currently this is >>> impossible to do >>> deterministically since the logging and event tracing use different >>> timing. >>> >>> I propose to add a flag called PrintGCID. Here's an example log >>> entry for GC >>> number 4 running with -XX:+PrintGCID: >>> >>> #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), >>> 0,2373420 secs] >>> >>> It is a decoration similar to the timestamps. Here's what it looks >>> like if you >>> add -XX:+PrintGCTimeStamps: >>> >>> 3,921: #4: [GC pause (G1 Evacuation Pause) (young) >>> 185M->212M(502M), 0,2373420 >>> secs] >>> >>> And here's what it will look like if you add -XX:+PrintGCDateStamps: >>> >>> 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation >>> Pause) >>> (young) 185M->212M(502M), 0,2373420 secs] >>> >>> >>> Here's a little longer example running ParallelGC: >>> >>> #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] >>> #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] >>> #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] >>> #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 >>> secs] >>> #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] >>> #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 >>> secs] >>> >>> Note that for the concurrent GCs this gives a good way of seeing the >>> concurrent >>> action. Here is a G1 example: >>> >>> #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), >>> 0,0383157 secs] >>> #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), >>> 0,0412220 secs] >>> #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), >>> 0,0426144 secs] >>> #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), >>> 0,1467633 secs] >>> #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), >>> 0,1627465 secs] >>> #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), >>> 0,2376728 secs] >>> #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) >>> 255M->269M(1004M), >>> 0,2623779 secs] >>> #7: [GC concurrent-root-region-scan-start] >>> #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] >>> #7: [GC concurrent-mark-start] >>> #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), >>> 0,3007405 secs] >>> #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), >>> 0,4063937 secs] >>> #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), >>> 0,7500609 secs] >>> #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), >>> 1,5464884 secs] >>> #7: [GC concurrent-mark-end, 3,9464599 secs] >>> #7: [GC remark, 0,0240462 secs] >>> #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] >>> #7: [GC concurrent-cleanup-start] >>> #7: [GC concurrent-cleanup-end, 0,0000228 secs] >>> #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), >>> 2,2464423 secs] >>> >>> Notice how the complete concurrent cycle is decorated with id #7. >>> That makes it >>> possible to follow the concurrent work more easily and grep it out etc. >>> >>> >>> Here's a similar thing for CMS where the concurrent cycle #22 has a >>> young GC #23 >>> in the middle of it: >>> >>> #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), >>> 1,4558792 secs] >>> #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] >>> #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), >>> 0,7909961 secs] >>> #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] >>> >>> >>> I think this is very helpful information to have, so I would like to >>> push this >>> with the new flag PrintGCID as true by default. I'm sure some >>> parsers will >>> break, but it is an easy work around to run with -XX:-PrintGCID to >>> get the old >>> format. It should even be pretty easy to do post processing on a log >>> file to >>> remove the GC id information. Having it on by default makes it >>> possible for us >>> to use this information and in particular be able to map it to the >>> JFR events we >>> see in MissionControl. >>> >>> I would also like to backport this to 8u40, but in that case with >>> the default >>> value of PrintGCID as false. >>> >>> Thanks, >>> Bengt >>> >>> >>> >>> >>> >>> > From david.holmes at oracle.com Fri May 23 07:39:54 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 May 2014 17:39:54 +1000 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537ECC1A.1030409@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> Message-ID: <537EFB4A.1000507@oracle.com> Hi Dan, Functional changes look good. For the test I thought the preferred approach these days was to use the testlibrary utilities to launch the test in another VM and also launch the tool. That way the test isn't restricted to needs_jdk. Cheers, David On 23/05/2014 2:18 PM, Daniel D. Daugherty wrote: > Zhengyu is tied up with some other work so I've taken on this fix. > > Here's the webrev URL for the next round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ > > The fix has been tested with vm.quick on all Aurora Adhoc platforms. > The new test has been run with the fix via JPRT and passes on all > JPRT platforms. The new test has also been run without the fix and > fails on most platforms. Since the default sample size is just 30, > it is possible to get 30 runs in a row without failing. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/19/14 7:58 AM, Zhengyu Gu wrote: >> This is a simple fix for incorrect lock state. >> >> The timing on setting thread's pending monitor can result stack trace >> dump reporting incorrect lock state. The solution is to check the >> monitor's owner, if the owner is other than the current thread, then >> the monitor, is or is in process of being, set the pending monitor of >> current thread. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >> >> >> Thanks, >> >> -Zhengyu >> > From daniel.daugherty at oracle.com Fri May 23 13:56:27 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 23 May 2014 07:56:27 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537EFB4A.1000507@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <537EFB4A.1000507@oracle.com> Message-ID: <537F538B.2030109@oracle.com> Thanks for the quick review! On 5/23/14 1:39 AM, David Holmes wrote: > Hi Dan, > > Functional changes look good. Thanks. > For the test I thought the preferred approach these days was to use > the testlibrary utilities to launch the test in another VM and also > launch the tool. That way the test isn't restricted to needs_jdk. Don't really know the right answer here. I usually write tests like this as a Java program with a shell script wrapper, but I know we're trying not to do that anymore. When I grep'ed for jstack in the hotspot tests, I ran across one: $ rgrep jstack test test/runtime/7194254/Test7194254.java: * whether jstack reports correct priorities for them. test/runtime/7194254/Test7194254.java: String jstack = System.getProperty("java.home") + "/../bin/jstack"; test/runtime/7194254/Test7194254.java: Process process = new ProcessBuilder(jstack, pid) It turns out that your test for 7194254 was pretty much perfect for me to adapt for what I was trying to test. For now, I'm inclined to go with the current test if you're OK with that. Dan > > Cheers, > David > > On 23/05/2014 2:18 PM, Daniel D. Daugherty wrote: >> Zhengyu is tied up with some other work so I've taken on this fix. >> >> Here's the webrev URL for the next round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >> >> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >> The new test has been run with the fix via JPRT and passes on all >> JPRT platforms. The new test has also been run without the fix and >> fails on most platforms. Since the default sample size is just 30, >> it is possible to get 30 runs in a row without failing. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack trace >>> dump reporting incorrect lock state. The solution is to check the >>> monitor's owner, if the owner is other than the current thread, then >>> the monitor, is or is in process of being, set the pending monitor of >>> current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >> From paul.sandoz at oracle.com Fri May 23 15:03:56 2014 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 23 May 2014 17:03:56 +0200 Subject: Hotspot build broken on Mac 10.8.5 Xcode 4.6.3 Message-ID: <44C64E74-A862-4B75-ABDF-C36A560ACDFC@oracle.com> Hi, I can no longer build jdk9/dev as hotspot fails to build: dtrace headers generated Generating precompiled header precompiled.hpp.gch cc1plus: warnings being treated as errors In file included from ../generated/tracefiles/traceEventClasses.hpp:19, from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/tracing.hpp:28, from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/compile.hpp:43, from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/node.hpp:29, from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/addnode.hpp:28, from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/precompiled/precompiled.hpp:252: /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp: In member function ?void TraceStream::print_val(const char*, s8)?: /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp:69: warning: format ?%lld? expects type ?long long int?, but argument 4 has type ?s8? make[8]: *** [precompiled.hpp.gch] Error 1 If fails for the tip: hotspot $ hg sum parent: 6436:be9e24a9d6c6 tip Merge branch: default commit: (clean) update: (current) And passes for the previous changeset: hotspot $ hg update -r -2 hotspot $ hg sum parent: 6435:18f86da62a40 Added tag jdk9-b14 for changeset bd333491bb6c branch: default commit: (clean) update: 25 new changesets (update) Do we have a bad merge? http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/be9e24a9d6c6 Paul. From nils.eliasson at oracle.com Fri May 23 16:07:11 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 23 May 2014 09:07:11 -0700 (PDT) Subject: [8u20] RFR(S): 8031475: Missing oopmap in patching stubs Message-ID: Please review this backport of JDK-8031745 from 9 to 8u20. Applies cleanly to http://hg.openjdk.java.net/jdk8u/hs-dev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8042059 jkd9: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6448 webrev 8u20: http://cr.openjdk.java.net/~neliasso/8031475_8u20/webrev.01/ Kind Regards, Nils Eliasson From vladimir.kozlov at oracle.com Fri May 23 16:08:42 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 23 May 2014 09:08:42 -0700 Subject: [8u20] Request for approval to backport JDK-8031475: Missing oopmap in patching stubs In-Reply-To: References: Message-ID: <537F728A.10107@oracle.com> Hotspot backports requests should be sent to hotspot-dev. You did not list jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/279b06e56692 Changes are good to backport. Don't forget to use the main bug id. Use hg export/hg import. Thanks, Vladimir On 5/23/14 4:10 AM, Nils Eliasson wrote: > Please approve the backport of JDK-8031745 from 9 to 8u20. > > 8031475: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6448 > http://cr.openjdk.java.net/~neliasso/8031475/webrev.06/ > > Applies cleanly to http://hg.openjdk.java.net/jdk8u/hs-dev/ > > Kind Regards, > Nils Eliasson > From vladimir.kozlov at oracle.com Fri May 23 16:09:38 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 23 May 2014 09:09:38 -0700 Subject: [8u20] RFR(S): 8031475: Missing oopmap in patching stubs In-Reply-To: References: Message-ID: <537F72C2.5090009@oracle.com> I replied to other mail. Changes are good. Thanks, Vladimir On 5/23/14 9:07 AM, Nils Eliasson wrote: > Please review this backport of JDK-8031745 from 9 to 8u20. > > Applies cleanly to http://hg.openjdk.java.net/jdk8u/hs-dev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8042059 > jkd9: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6448 > webrev 8u20: http://cr.openjdk.java.net/~neliasso/8031475_8u20/webrev.01/ > > Kind Regards, > Nils Eliasson > From david.r.chase at oracle.com Fri May 23 16:28:02 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 23 May 2014 12:28:02 -0400 Subject: Hotspot build broken on Mac 10.8.5 Xcode 4.6.3 In-Reply-To: <44C64E74-A862-4B75-ABDF-C36A560ACDFC@oracle.com> References: <44C64E74-A862-4B75-ABDF-C36A560ACDFC@oracle.com> Message-ID: <4391523E-B689-4F09-BEB8-612A2D09CB5C@oracle.com> Supposedly fixed, I'll try to figure out where it went. This is the bug: https://bugs.openjdk.java.net/browse/JDK-8043164 This is the changeset: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/fdcbdafe2073 David On 2014-05-23, at 11:03 AM, Paul Sandoz wrote: > Hi, > > I can no longer build jdk9/dev as hotspot fails to build: > > dtrace headers generated > Generating precompiled header precompiled.hpp.gch > cc1plus: warnings being treated as errors > In file included from ../generated/tracefiles/traceEventClasses.hpp:19, > from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/tracing.hpp:28, > from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/compile.hpp:43, > from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/node.hpp:29, > from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/addnode.hpp:28, > from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/precompiled/precompiled.hpp:252: > /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp: In member function ?void TraceStream::print_val(const char*, s8)?: > /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp:69: warning: format ?%lld? expects type ?long long int?, but argument 4 has type ?s8? > make[8]: *** [precompiled.hpp.gch] Error 1 > > > If fails for the tip: > > hotspot $ hg sum > parent: 6436:be9e24a9d6c6 tip > Merge > branch: default > commit: (clean) > update: (current) > > And passes for the previous changeset: > > hotspot $ hg update -r -2 > hotspot $ hg sum > parent: 6435:18f86da62a40 > Added tag jdk9-b14 for changeset bd333491bb6c > branch: default > commit: (clean) > update: 25 new changesets (update) > > Do we have a bad merge? > > http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/be9e24a9d6c6 > > Paul. From david.r.chase at oracle.com Fri May 23 16:58:53 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 23 May 2014 12:58:53 -0400 Subject: Hotspot build broken on Mac 10.8.5 Xcode 4.6.3 In-Reply-To: <4391523E-B689-4F09-BEB8-612A2D09CB5C@oracle.com> References: <44C64E74-A862-4B75-ABDF-C36A560ACDFC@oracle.com> <4391523E-B689-4F09-BEB8-612A2D09CB5C@oracle.com> Message-ID: I think that somehow hs-rt just didn't get merged in to dev yet, and that the failure is also dependent on flag settings that are not always active (because it didn't get caught by my own testing on the same OS+compiler, nor did JPRT catch it). So it's not a bad merge, just a missing merge that has not happened yet. David On 2014-05-23, at 12:28 PM, David Chase wrote: > Supposedly fixed, I'll try to figure out where it went. > > This is the bug: https://bugs.openjdk.java.net/browse/JDK-8043164 > This is the changeset: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/fdcbdafe2073 > > David > > On 2014-05-23, at 11:03 AM, Paul Sandoz wrote: > >> Hi, >> >> I can no longer build jdk9/dev as hotspot fails to build: >> >> dtrace headers generated >> Generating precompiled header precompiled.hpp.gch >> cc1plus: warnings being treated as errors >> In file included from ../generated/tracefiles/traceEventClasses.hpp:19, >> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/tracing.hpp:28, >> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/compile.hpp:43, >> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/node.hpp:29, >> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/addnode.hpp:28, >> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/precompiled/precompiled.hpp:252: >> /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp: In member function ?void TraceStream::print_val(const char*, s8)?: >> /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp:69: warning: format ?%lld? expects type ?long long int?, but argument 4 has type ?s8? >> make[8]: *** [precompiled.hpp.gch] Error 1 >> >> >> If fails for the tip: >> >> hotspot $ hg sum >> parent: 6436:be9e24a9d6c6 tip >> Merge >> branch: default >> commit: (clean) >> update: (current) >> >> And passes for the previous changeset: >> >> hotspot $ hg update -r -2 >> hotspot $ hg sum >> parent: 6435:18f86da62a40 >> Added tag jdk9-b14 for changeset bd333491bb6c >> branch: default >> commit: (clean) >> update: 25 new changesets (update) >> >> Do we have a bad merge? >> >> http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/be9e24a9d6c6 >> >> Paul. > From daniel.daugherty at oracle.com Fri May 23 17:24:55 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 23 May 2014 11:24:55 -0600 Subject: Hotspot build broken on Mac 10.8.5 Xcode 4.6.3 In-Reply-To: References: <44C64E74-A862-4B75-ABDF-C36A560ACDFC@oracle.com> <4391523E-B689-4F09-BEB8-612A2D09CB5C@oracle.com> Message-ID: <537F8467.4010306@oracle.com> The changeset for: 8043164: Format warning in traceStream.hpp was pushed to RT_Baseline on 2014.05.14 (Wednesday last week). RT_Baseline typically snapshots Tuesday night and pushes on a Wednesday. This means that the fix for 8043164 did not get pushed to Main_Baseline until 2014.05.21 (Wednesday this week). If you have a push to RT_Baseline that does not make the normal Tuesday night cut-off, you have to let the GK know that they need to either wait or do a second push. Dan On 5/23/14 10:58 AM, David Chase wrote: > I think that somehow hs-rt just didn't get merged in to dev yet, and that the > failure is also dependent on flag settings that are not always active (because > it didn't get caught by my own testing on the same OS+compiler, nor did JPRT > catch it). > > So it's not a bad merge, just a missing merge that has not happened yet. > > David > > On 2014-05-23, at 12:28 PM, David Chase wrote: > >> Supposedly fixed, I'll try to figure out where it went. >> >> This is the bug: https://bugs.openjdk.java.net/browse/JDK-8043164 >> This is the changeset: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/fdcbdafe2073 >> >> David >> >> On 2014-05-23, at 11:03 AM, Paul Sandoz wrote: >> >>> Hi, >>> >>> I can no longer build jdk9/dev as hotspot fails to build: >>> >>> dtrace headers generated >>> Generating precompiled header precompiled.hpp.gch >>> cc1plus: warnings being treated as errors >>> In file included from ../generated/tracefiles/traceEventClasses.hpp:19, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/tracing.hpp:28, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/compile.hpp:43, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/node.hpp:29, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/addnode.hpp:28, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/precompiled/precompiled.hpp:252: >>> /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp: In member function ?void TraceStream::print_val(const char*, s8)?: >>> /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp:69: warning: format ?%lld? expects type ?long long int?, but argument 4 has type ?s8? >>> make[8]: *** [precompiled.hpp.gch] Error 1 >>> >>> >>> If fails for the tip: >>> >>> hotspot $ hg sum >>> parent: 6436:be9e24a9d6c6 tip >>> Merge >>> branch: default >>> commit: (clean) >>> update: (current) >>> >>> And passes for the previous changeset: >>> >>> hotspot $ hg update -r -2 >>> hotspot $ hg sum >>> parent: 6435:18f86da62a40 >>> Added tag jdk9-b14 for changeset bd333491bb6c >>> branch: default >>> commit: (clean) >>> update: 25 new changesets (update) >>> >>> Do we have a bad merge? >>> >>> http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/be9e24a9d6c6 >>> >>> Paul. > > > From coleen.phillimore at oracle.com Fri May 23 17:43:54 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 23 May 2014 13:43:54 -0400 Subject: Hotspot build broken on Mac 10.8.5 Xcode 4.6.3 In-Reply-To: References: <44C64E74-A862-4B75-ABDF-C36A560ACDFC@oracle.com> <4391523E-B689-4F09-BEB8-612A2D09CB5C@oracle.com> Message-ID: <537F88DA.7010902@oracle.com> Right, hs-rt merged on the 14th with the tested bits from the 13th. Coleen On 5/23/14, 12:58 PM, David Chase wrote: > I think that somehow hs-rt just didn't get merged in to dev yet, and that the > failure is also dependent on flag settings that are not always active (because > it didn't get caught by my own testing on the same OS+compiler, nor did JPRT > catch it). > > So it's not a bad merge, just a missing merge that has not happened yet. > > David > > On 2014-05-23, at 12:28 PM, David Chase wrote: > >> Supposedly fixed, I'll try to figure out where it went. >> >> This is the bug: https://bugs.openjdk.java.net/browse/JDK-8043164 >> This is the changeset: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/fdcbdafe2073 >> >> David >> >> On 2014-05-23, at 11:03 AM, Paul Sandoz wrote: >> >>> Hi, >>> >>> I can no longer build jdk9/dev as hotspot fails to build: >>> >>> dtrace headers generated >>> Generating precompiled header precompiled.hpp.gch >>> cc1plus: warnings being treated as errors >>> In file included from ../generated/tracefiles/traceEventClasses.hpp:19, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/tracing.hpp:28, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/compile.hpp:43, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/node.hpp:29, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/opto/addnode.hpp:28, >>> from /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/precompiled/precompiled.hpp:252: >>> /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp: In member function ?void TraceStream::print_val(const char*, s8)?: >>> /Users/sandoz/Projects/jdk9/dev-clone/hotspot/src/share/vm/trace/traceStream.hpp:69: warning: format ?%lld? expects type ?long long int?, but argument 4 has type ?s8? >>> make[8]: *** [precompiled.hpp.gch] Error 1 >>> >>> >>> If fails for the tip: >>> >>> hotspot $ hg sum >>> parent: 6436:be9e24a9d6c6 tip >>> Merge >>> branch: default >>> commit: (clean) >>> update: (current) >>> >>> And passes for the previous changeset: >>> >>> hotspot $ hg update -r -2 >>> hotspot $ hg sum >>> parent: 6435:18f86da62a40 >>> Added tag jdk9-b14 for changeset bd333491bb6c >>> branch: default >>> commit: (clean) >>> update: 25 new changesets (update) >>> >>> Do we have a bad merge? >>> >>> http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/be9e24a9d6c6 >>> >>> Paul. From fredrik.arvidsson at oracle.com Fri May 16 12:24:37 2014 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Fri, 16 May 2014 14:24:37 +0200 Subject: Need comments on JEP-158: Unified JVM Logging In-Reply-To: <1400240489.32759.48.camel@galactica.localdomain> References: <5375D08C.9070206@oracle.com> <1400240489.32759.48.camel@galactica.localdomain> Message-ID: <53760385.3000608@oracle.com> Hi Mario, thanks for the feedback! Pleas see my comments in-lined below. On 2014-05-16 13:41, Mario Torre wrote: > On Fri, 2014-05-16 at 11:45 +0100, Richard Warburton wrote: >> Hi, > Hi all, > > I share this same doubts, and I'm also a bit confused. >> I'm not saying that an abstract framework should depend upon the details of >> the different components but if you had component definable markers for the >> type of information in order to offer more fine grained control I think it >> would be a most appreciated. That way something like a GC logger could >> define its own markers for details, cause, application stopped time, >> safepoints etc. An end user who is trying to diagnose a specific problem >> could leave their gc logger at warning, but also then has the option to >> enable a specific marker for something like application stopped time which >> is probably a lower level than warning. I posted an answer to Richards mail earlier, I don't know if you?ve had time to read it yet. I hope my answer describes an alternative way of having fine grained control of logging using sub loggers/components/categories. > This would be very nice. > > I think this JEP pretty much works in the same direction as 167. I > believe the ultimate goal should be to have some kind of DTRACE/STAP > feature where user could enable/disable a specific probe/set of probes > at runtime and get events from those, while JVM/Serviceability developer > could easily add probes and instrumentation to the code in a standard > way with a proper structure (which should be efficient also). > > If I understand correctly, this JEP tries to implement the logging part > of such infrastructure (as you say in Non-Goals). > > I would not do this, though: > > "Logging messages are in human-readable plain text" The 'human-readable' is a part of the goal for this JEP. This JEP set up a goal to have an few adapters implemented. StdOut, StdErr and file. Other types of adapters might have other ways of representing the logged information though. > > This should be part of the logging backend, which may be part of the > framework and so still be a goal in a sense, but I would rather keep it > separated from the logging infrastructure itself, which is what I would > spend more time defining. > > Looks to me that the concept of adapters is very interesting, but > applies to the "act of logging" itself. For instance, if you log a > thread state change event, this would end up in the adapter, whose job > is to put it, say, in a file, rather than in the syslog, etc. Adapters are logging backends and the API between the logger and the adapter does not dictate in what way, or in which form the logged information or any decorations will be output. It is up to the adapter implementation to decide if the logged information should be put in an asynchronous FIFO queue before output or if to do it in a blocking fashion like the file-adapter. > I would go a bit further though, defining a specific protocol where > those events are accumulated before being consumed by the adapter, this > protocol should allow for minimum overhead, for instance, all events > will be directed to a memory buffer, a separate adapter thread will read > this buffer and take away in FIFO order, etc.. Asynchronous logging is not always something you want in the scope of this JEP. When doing post-mortem analysis for example you most probably want to be as sure as possible that all logs have been flushed to an external media when a process crashes or happens to end up exiting on an assert. With the adapter design it is however up to the implementer to decide what, where and when to flush things. Even if we will go to great lengths trying to keep this implementation as non-intrusive and lean as possible it is not intended to be a trace logging framework having extremely low overhead like the JFR/JEP-167 solution. > > Not sure if this is part of this JEP, but sounds a more interesting and > useful thing to do (I was hoping to see this in JEP 167, but apparently > this is not what happens there). > > Cheers, > Mario > Cheers /Fredrik // Adding back hotspot-dev at openjdk.java.net // From andrey.x.zakharov at oracle.com Wed May 21 15:37:00 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Wed, 21 May 2014 19:37:00 +0400 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot In-Reply-To: <5371F734.6090809@oracle.com> References: <536B7CF0.6010508@oracle.com> <536B9E36.6090802@oracle.com> <536BA28D.7030808@oracle.com> <3503372.bSN5QEX8PY@mgerdin03> <5371F734.6090809@oracle.com> Message-ID: <537CC81C.6010905@oracle.com> On 13.05.2014 14:43, Andrey Zakharov wrote: > Hi > So here is trivial patch - > removing ClassFileInstaller sun.hotspot.WhiteBox and adding > main/othervm/bootclasspath > where this needed > > Also, some tests are modified as > - "-Xbootclasspath/a:.", > + "-Xbootclasspath/a:" + System.getProperty("test.classes"), > > Thanks. webrev: http://cr.openjdk.java.net/~jwilhelm/8011397/webrev.02/ bug: https://bugs.openjdk.java.net/browse/JDK-8011397 Thanks. > > On 09.05.2014 12:13, Mikael Gerdin wrote: >> On Thursday 08 May 2014 19.28.13 Igor Ignatyev wrote: >>> // cc'ing hotspot-dev instaed of compiler, runtime and gc lists. >>> >>> On 05/08/2014 07:09 PM, Filipp Zhinkin wrote: >>>> Andrey, >>>> >>>> I've CC'ed compiler and runtime mailing list, because you're changes >>>> affect test for other components as too. >>>> >>>> I don't like your solution (but I'm not a reviewer, so treat my words >>>> just as suggestion), >>>> because we'll have to write more meta information for each test and it >>>> is very easy to >>>> forget to install WhiteBoxPermission if you don't test your test with >>>> some security manager. >>>> >>>> From my point of view, it will be better to extend >>>> ClassFileInstaller >>>> >>>> so it will copy not only >>>> a class whose name was passed as an arguments, but also all inner >>>> classes of that class. >>>> And if someone want copy only specified class without inner classes, >>>> then some option >>>> could be added to ClassFileInstaller to force such behaviour. >> Perhaps this is a good time to get rid of ClassFileInstaller altogether? >> >> https://bugs.openjdk.java.net/browse/JDK-8009117 >> >> The reason for its existence is that the WhiteBox class needs to be >> on the >> boot class path. >> If we can live with having all the test's classes on the boot class >> path then >> we could use the /bootclasspath option in jtreg as stated in the RFE. >> >> /Mikael >> >>>> Thanks, >>>> Filipp. >>>> >>>> On 05/08/2014 04:47 PM, Andrey Zakharov wrote: >>>>> Hi! >>>>> Suggesting patch with fixes for >>>>> https://bugs.openjdk.java.net/browse/JDK-8011397 >>>>> >>>>> webrev: >>>>> https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz >>>>> patch: >>>>> https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPer >>>>> >>>>> mission >>>>> >>>>> >>>>> >>>>> Thanks. > From andrey.x.zakharov at oracle.com Fri May 23 11:40:59 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Fri, 23 May 2014 15:40:59 +0400 Subject: RFR: 8011397: JTREG needs to copy additional WhiteBox class file to JTwork/scratch/sun/hotspot In-Reply-To: <537DB9B0.8010200@oracle.com> References: <536B7CF0.6010508@oracle.com> <536B9E36.6090802@oracle.com> <536BA28D.7030808@oracle.com> <3503372.bSN5QEX8PY@mgerdin03> <5371F734.6090809@oracle.com> <537CC81C.6010905@oracle.com> <537DB9B0.8010200@oracle.com> Message-ID: <537F33CB.5050505@oracle.com> On 22.05.2014 12:47, Igor Ignatyev wrote: > Andrey, > > 1. You changed dozen of tests, have you tested your changes? Locally, aurora on the way. > > 2. Your changes of year in copyright is wrong. it has to be > $first_year, [$last_year, ], see Mark's email[1] for details. > > [1] http://mail.openjdk.java.net/pipermail/jdk7-dev/2010-May/001321.html Thanks, fixed. will be uploaded soon. > > Igor > > On 05/21/2014 07:37 PM, Andrey Zakharov wrote: >> >> On 13.05.2014 14:43, Andrey Zakharov wrote: >>> Hi >>> So here is trivial patch - >>> removing ClassFileInstaller sun.hotspot.WhiteBox and adding >>> main/othervm/bootclasspath >>> where this needed >>> >>> Also, some tests are modified as >>> - "-Xbootclasspath/a:.", >>> + "-Xbootclasspath/a:" + System.getProperty("test.classes"), >>> >>> Thanks. >> webrev: http://cr.openjdk.java.net/~jwilhelm/8011397/webrev.02/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8011397 >> Thanks. >> >>> >>> On 09.05.2014 12:13, Mikael Gerdin wrote: >>>> On Thursday 08 May 2014 19.28.13 Igor Ignatyev wrote: >>>>> // cc'ing hotspot-dev instaed of compiler, runtime and gc lists. >>>>> >>>>> On 05/08/2014 07:09 PM, Filipp Zhinkin wrote: >>>>>> Andrey, >>>>>> >>>>>> I've CC'ed compiler and runtime mailing list, because you're changes >>>>>> affect test for other components as too. >>>>>> >>>>>> I don't like your solution (but I'm not a reviewer, so treat my >>>>>> words >>>>>> just as suggestion), >>>>>> because we'll have to write more meta information for each test >>>>>> and it >>>>>> is very easy to >>>>>> forget to install WhiteBoxPermission if you don't test your test >>>>>> with >>>>>> some security manager. >>>>>> >>>>>> From my point of view, it will be better to extend >>>>>> ClassFileInstaller >>>>>> >>>>>> so it will copy not only >>>>>> a class whose name was passed as an arguments, but also all inner >>>>>> classes of that class. >>>>>> And if someone want copy only specified class without inner classes, >>>>>> then some option >>>>>> could be added to ClassFileInstaller to force such behaviour. >>>> Perhaps this is a good time to get rid of ClassFileInstaller >>>> altogether? >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8009117 >>>> >>>> The reason for its existence is that the WhiteBox class needs to be >>>> on the >>>> boot class path. >>>> If we can live with having all the test's classes on the boot class >>>> path then >>>> we could use the /bootclasspath option in jtreg as stated in the RFE. >>>> >>>> /Mikael >>>> >>>>>> Thanks, >>>>>> Filipp. >>>>>> >>>>>> On 05/08/2014 04:47 PM, Andrey Zakharov wrote: >>>>>>> Hi! >>>>>>> Suggesting patch with fixes for >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8011397 >>>>>>> >>>>>>> webrev: >>>>>>> https://bugs.openjdk.java.net/secure/attachment/20275/8011397.tgz >>>>>>> patch: >>>>>>> https://bugs.openjdk.java.net/secure/attachment/20274/8011397.WhiteBoxPer >>>>>>> >>>>>>> >>>>>>> mission >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks. >>> >> From philip.race at oracle.com Fri May 23 19:56:57 2014 From: philip.race at oracle.com (Phil Race) Date: Fri, 23 May 2014 12:56:57 -0700 Subject: Hotspot build broken on Mac 10.8.5 Xcode 4.6.3 Message-ID: <537FA809.1080500@oracle.com> I just pulled changes into my repo which include this fix :- 8037816: Fix for 8036122 breaks build with Xcode5/clang Summary: Repaired or selectively disabled offending formats; future-proofed with additional checking Reviewed-by: kvn, jrose, stefank and I'm seeing the exact same issue discussed here :- http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-May/014051.html Any idea when this'll get resolved, or if there's a workaround I can apply ? -phil. From serguei.spitsyn at oracle.com Fri May 23 23:08:31 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 23 May 2014 16:08:31 -0700 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537ECC1A.1030409@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> Message-ID: <537FD4EF.2030406@oracle.com> Hi Dan, The fix itself looks good. The test deserves a refactoring as the main method is too big. Would it be possible to convert some of the blocks of the while loop into methods? Even the try block (variant: the while loop) is a good candidate to be a separate method. Also the definitions of the patterns can be moved out of the main and made static final fields. Thanks, Serguei On 5/22/14 9:18 PM, Daniel D. Daugherty wrote: > Zhengyu is tied up with some other work so I've taken on this fix. > > Here's the webrev URL for the next round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ > > The fix has been tested with vm.quick on all Aurora Adhoc platforms. > The new test has been run with the fix via JPRT and passes on all > JPRT platforms. The new test has also been run without the fix and > fails on most platforms. Since the default sample size is just 30, > it is possible to get 30 runs in a row without failing. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/19/14 7:58 AM, Zhengyu Gu wrote: >> This is a simple fix for incorrect lock state. >> >> The timing on setting thread's pending monitor can result stack trace >> dump reporting incorrect lock state. The solution is to check the >> monitor's owner, if the owner is other than the current thread, then >> the monitor, is or is in process of being, set the pending monitor of >> current thread. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >> >> >> Thanks, >> >> -Zhengyu >> > From david.holmes at oracle.com Sat May 24 00:04:02 2014 From: david.holmes at oracle.com (David Holmes) Date: Sat, 24 May 2014 10:04:02 +1000 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537F538B.2030109@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <537EFB4A.1000507@oracle.com> <537F538B.2030109@oracle.com> Message-ID: <537FE1F2.3050408@oracle.com> On 23/05/2014 11:56 PM, Daniel D. Daugherty wrote: > Thanks for the quick review! > > > On 5/23/14 1:39 AM, David Holmes wrote: >> Hi Dan, >> >> Functional changes look good. > > Thanks. > > >> For the test I thought the preferred approach these days was to use >> the testlibrary utilities to launch the test in another VM and also >> launch the tool. That way the test isn't restricted to needs_jdk. > > Don't really know the right answer here. I usually write tests > like this as a Java program with a shell script wrapper, but I > know we're trying not to do that anymore. > > When I grep'ed for jstack in the hotspot tests, I ran across one: > > $ rgrep jstack test > test/runtime/7194254/Test7194254.java: * whether jstack reports > correct priorities for them. > test/runtime/7194254/Test7194254.java: String jstack = > System.getProperty("java.home") + "/../bin/jstack"; > test/runtime/7194254/Test7194254.java: Process process = new > ProcessBuilder(jstack, pid) > > It turns out that your test for 7194254 was pretty much perfect > for me to adapt for what I was trying to test. For now, I'm > inclined to go with the current test if you're OK with that. That was then, this is now :) testlibrary gives you everything you need without having to resort to java.home hacks. And the added benefit is that the test is not restricted to being a JDK-only test. Your call - you are the one that has to do the work. Cheers, David > Dan > > >> >> Cheers, >> David >> >> On 23/05/2014 2:18 PM, Daniel D. Daugherty wrote: >>> Zhengyu is tied up with some other work so I've taken on this fix. >>> >>> Here's the webrev URL for the next round: >>> >>> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >>> >>> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >>> The new test has been run with the fix via JPRT and passes on all >>> JPRT platforms. The new test has also been run without the fix and >>> fails on most platforms. Since the default sample size is just 30, >>> it is possible to get 30 runs in a row without failing. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >>> >>> >>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>> This is a simple fix for incorrect lock state. >>>> >>>> The timing on setting thread's pending monitor can result stack trace >>>> dump reporting incorrect lock state. The solution is to check the >>>> monitor's owner, if the owner is other than the current thread, then >>>> the monitor, is or is in process of being, set the pending monitor of >>>> current thread. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>> > From nils.eliasson at oracle.com Mon May 26 08:26:07 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Mon, 26 May 2014 10:26:07 +0200 Subject: [8u20] RFR(S): 8031475: Missing oopmap in patching stubs In-Reply-To: <537F72C2.5090009@oracle.com> References: <537F72C2.5090009@oracle.com> Message-ID: <5382FA9F.2050201@oracle.com> Thank you Vladimir! //Nils On 2014-05-23 18:09, Vladimir Kozlov wrote: > I replied to other mail. Changes are good. > > Thanks, > Vladimir > > On 5/23/14 9:07 AM, Nils Eliasson wrote: >> Please review this backport of JDK-8031745 from 9 to 8u20. >> >> Applies cleanly to http://hg.openjdk.java.net/jdk8u/hs-dev/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8042059 >> jkd9: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/6448 >> webrev 8u20: >> http://cr.openjdk.java.net/~neliasso/8031475_8u20/webrev.01/ >> >> Kind Regards, >> Nils Eliasson >> From thomas.schatzl at oracle.com Mon May 26 09:12:33 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 26 May 2014 11:12:33 +0200 Subject: PATCH: swapped usage of idx_t and bm_word_t types in parMarkBitMap.cpp In-Reply-To: <20140522115947.5ce5ad198a35e8155a9c1f59@danny.cz> References: <20140522101904.a9a5d06f13b9b13cf64fca4e@danny.cz> <1400750628.2617.11.camel@cirrus> <20140522115947.5ce5ad198a35e8155a9c1f59@danny.cz> Message-ID: <1401095553.2642.4.camel@cirrus> Hi, On Thu, 2014-05-22 at 11:59 +0200, Dan Hor?k wrote: > On Thu, 22 May 2014 11:23:48 +0200 > Thomas Schatzl wrote: > > > Hi Dan, > > > > On Thu, 2014-05-22 at 10:19 +0200, Dan Hor?k wrote: > > > Hello, > > > [...] > > > This a follow-up of my earlier s390 related patch [2] > > > > > > [1] http://fedora.danny.cz/openjdk/bitmap-1/webrev/ > > > [2] > > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-September/010913.html > > > > > > Note: I'm Red Hat employee (dhorak at redhat.com), so I should be > > > covered by Red Hat's CLA for my contributions. > > > > > > > Created https://bugs.openjdk.java.net/browse/JDK-8043722 . > > > > I roughly looked at other similar changes, shouldn't line 129 in > > g1/concurrentMark.cpp changed too? > > > > 129 _bm.set_map((uintptr_t*)_virtual_space.low()); > > > > @Compiler team: what about the instantiations of MethodLivenessResult? > > > > I.e. in methodLiveness.cpp:476 and 1001 there are two other uintptr_t* > > casts. > > they don't fail the build because bm_word_t is uintptr_t, but in my > opinion they should be changed too, at least for consistency Can you check the patch at http://cr.openjdk.java.net/~tschatzl/8043722/webrev/ for S390? It builds on all Oracle platforms. If so, your change has been reviewed by me. Thanks, Thomas From mikael.vidstedt at oracle.com Tue May 27 22:22:47 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 27 May 2014 15:22:47 -0700 Subject: RFR (XS): 8044071: Print format/argument warnings Message-ID: <53851037.8040801@oracle.com> Please review the below change which addresses a number of print format/argument mismatches when compiling linux/sparc using gcc. Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ Most of these are straightforward changes to INTPTR_FORMAT & adding p2i. A couple of changes worth bringing up: * nativeInst_sparc.cpp I changed the format of the offset in mov related prints (two occurrences) to %d, since it's really a signed decimal int. This will hopefully reduce confusion when the offset is negative, where previously it would have looked like a relatively large positive offset. * macroAssembler_sparc.cpp The new code is in line with other similar patterns in the same file, but the ugly cast should IMHO really be rewritten to cast through a union. RFE? Cheers, Mikael From mikael.vidstedt at oracle.com Tue May 27 22:58:37 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 27 May 2014 15:58:37 -0700 Subject: RFR (XS): 8043206: Fix signed vs. unsigned comparison warning in copy_sparc.hpp In-Reply-To: <537E38B2.1020809@oracle.com> References: <537E3585.3050705@oracle.com> <537E38B2.1020809@oracle.com> Message-ID: <5385189D.1010105@oracle.com> Coleen/Lois/Vladimir - thanks for the reviews! Cheers, Mikael On 2014-05-22 10:49, Vladimir Kozlov wrote: > Good. > > Vladimir > > On 5/22/14 10:36 AM, Mikael Vidstedt wrote: >> >> Please review the following small fix which adds a cast to resolve a >> unsigned/signed comparison (and remove the corresponding warning gcc >> emits). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8043206 >> Webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8043206/webrev.00/webrev/ >> >> Cheers, >> Mikael >> From david.r.chase at oracle.com Wed May 28 03:10:05 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 27 May 2014 23:10:05 -0400 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <53851037.8040801@oracle.com> References: <53851037.8040801@oracle.com> Message-ID: Looks good to me, though I am a reviewer, not a Reviewer. David On 2014-05-27, at 6:22 PM, Mikael Vidstedt wrote: > > Please review the below change which addresses a number of print format/argument mismatches when compiling linux/sparc using gcc. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ > > Most of these are straightforward changes to INTPTR_FORMAT & adding p2i. A couple of changes worth bringing up: > > * nativeInst_sparc.cpp > > I changed the format of the offset in mov related prints (two occurrences) to %d, since it's really a signed decimal int. This will hopefully reduce confusion when the offset is negative, where previously it would have looked like a relatively large positive offset. > > * macroAssembler_sparc.cpp > > The new code is in line with other similar patterns in the same file, but the ugly cast should IMHO really be rewritten to cast through a union. RFE? > > Cheers, > Mikael > From david.holmes at oracle.com Wed May 28 05:38:59 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 May 2014 15:38:59 +1000 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <53851037.8040801@oracle.com> References: <53851037.8040801@oracle.com> Message-ID: <53857673.9040905@oracle.com> Looks okay. Can't comment on the changed formats in nativeInst_sparc.cpp. The ugly cast has precedent so I wouldn't lose sleep over it. Did the change in src/os_cpu/linux_sparc/vm/vm_version_linux_sparc.cpp fix a warning or do you just prefer using the raw array name? David H. On 28/05/2014 8:22 AM, Mikael Vidstedt wrote: > > Please review the below change which addresses a number of print > format/argument mismatches when compiling linux/sparc using gcc. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ > > Most of these are straightforward changes to INTPTR_FORMAT & adding p2i. > A couple of changes worth bringing up: > > * nativeInst_sparc.cpp > > I changed the format of the offset in mov related prints (two > occurrences) to %d, since it's really a signed decimal int. This will > hopefully reduce confusion when the offset is negative, where previously > it would have looked like a relatively large positive offset. > > * macroAssembler_sparc.cpp > > The new code is in line with other similar patterns in the same file, > but the ugly cast should IMHO really be rewritten to cast through a > union. RFE? > > Cheers, > Mikael > From ivan at azulsystems.com Wed May 28 08:56:31 2014 From: ivan at azulsystems.com (Ivan Krylov) Date: Wed, 28 May 2014 08:56:31 +0000 Subject: Fwd: JEP-147 implementation details References: <2C82F743BAC10D4D843822D7B0146F3BC58565@MBX021-W3-CA-2.exch021.domain.local> Message-ID: <7A646166-7158-4646-A244-3BB0BA67DDBB@azulsystems.com> Forwarding this to hotspot-dev for wider audience. Ivan Begin forwarded message: From: Fedor Burdun > Subject: intgresting in JEP-147 Date: 27 May 2014 18:46:14 GMT+4 To: "hotspot-runtime-dev at openjdk.java.net" > Hello, All. I'm interesting in what was done for JEP-147 (on Java8 M6 (b75) ). I found only next bugs JDK-8005895, JDK-8006280 probably related to it. However they are seemed very small part of techniques described in JEP. Also I have an interest in success metrics used for this JEP (measuring regression of startup and runtime, memory-reduce?) Thanks, Fedor From tobias.hartmann at oracle.com Wed May 28 09:49:45 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 28 May 2014 11:49:45 +0200 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <76B72B6C-B078-42F0-80DE-77DD56D92F92@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> <53750694.3080702@univ-mlv.fr> <5375AA6E.6030402@oracle.com> <5375FD00.90603@oracle.com> <76B72B6C-B078-42F0-80DE-77DD56D92F92@oracle.com> Message-ID: <5385B139.80808@oracle.com> Hi, thanks everyone for the feedback! @Remi: I agree with Paul. This is not a problem because if the normal read sees an outdated null value, a new LambdaForm is created and setCachedLambdaForm(...) is executed. This will guarantee that the non-null value is seen and used. The unnecessary creation of a new LamdaForm is not a problem either. @John: I added the code that you suggested to simulate CAS. Please find the new webrev at: http://cr.openjdk.java.net/~anoll/8005873/webrev.02/ Sorry for the delay, I was on vacation. Thanks, Tobias On 19.05.2014 20:31, John Rose wrote: > On May 16, 2014, at 4:56 AM, Tobias Hartmann wrote: > >> Is it sufficient then to use synchronized (lambdaForms) { ... } in setCachedLambdaForm(..) and a normal read in cachedLambdaForm(..)? > Yes, that is how I see it. The fast path is a racy non-volatile read of a safely-published structure. > > (If safe publication via arrays were broken, java.lang.String would be broken. But the JMM is carefully designed to support safe publication of array elements, and through array elements.) > > ? John From vladimir.x.ivanov at oracle.com Wed May 28 10:48:38 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 28 May 2014 14:48:38 +0400 Subject: [9] RFR(S): 8005873: JRuby test_respond_to.rb asserts with: MT-unsafe modification of inline cache In-Reply-To: <5385B139.80808@oracle.com> References: <53732FFB.7050407@oracle.com> <145fa3069c0.2761.5ce0156bc2f1864459fd77229eff3fe3@univ-mlv.fr> <53734692.6080409@oracle.com> <537349CA.1090904@oracle.com> <78DF1B85-B2FF-4C10-A721-2A565096983F@oracle.com> <53736C7B.2040303@oracle.com> <5373D0D7.40407@univ-mlv.fr> <5373D726.10008@oracle.com> <53747C93.3030508@univ-mlv.fr> <5374832F.5000800@oracle.com> <5374A0A8.1000805@oracle.com> <5374A50D.3080704@oracle.com> <19F4915C-F2E7-46EF-808E-89060BA2B5AD@oracle.com> <53750694.3080702@univ-mlv.fr> <5375AA6E.6030402@oracle.com> <5375FD00.90603@oracle.com> <76B72B6C-B078-42F0-80DE-77DD56D92F92@oracle.com> <5385B139.80808@oracle.com> Message-ID: <5385BF06.5090808@oracle.com> Looks good. It should be safe to sync on MTF instance since it's not accessible outside (MTF and MT.form() are package-private). Best regards, Vladimir Ivanov On 5/28/14 1:49 PM, Tobias Hartmann wrote: > Hi, > > thanks everyone for the feedback! > > @Remi: I agree with Paul. This is not a problem because if the normal > read sees an outdated null value, a new LambdaForm is created and > setCachedLambdaForm(...) is executed. This will guarantee that the > non-null value is seen and used. The unnecessary creation of a new > LamdaForm is not a problem either. > > @John: I added the code that you suggested to simulate CAS. Please find > the new webrev at: > > http://cr.openjdk.java.net/~anoll/8005873/webrev.02/ > > Sorry for the delay, I was on vacation. > > Thanks, > Tobias > > On 19.05.2014 20:31, John Rose wrote: >> On May 16, 2014, at 4:56 AM, Tobias Hartmann >> wrote: >> >>> Is it sufficient then to use synchronized (lambdaForms) { ... } in >>> setCachedLambdaForm(..) and a normal read in cachedLambdaForm(..)? >> Yes, that is how I see it. The fast path is a racy non-volatile read >> of a safely-published structure. >> >> (If safe publication via arrays were broken, java.lang.String would be >> broken. But the JMM is carefully designed to support safe publication >> of array elements, and through array elements.) >> >> ? John > From erik.helin at oracle.com Wed May 28 11:10:33 2014 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 28 May 2014 13:10:33 +0200 Subject: 8042933: assert(capacity_until_gc >= committed_bytes) failed Message-ID: <4471789.3Gp5iV3Z2a@ehelin-desktop> Hi all, this is the second attemp to fix JDK-8034852 [0], but since I backed out the previous attempt, I have to use a new bug [1] (you can't use the same bug-id for two different committs). For a background to the original problem, please see the old email thread [2]. This patch is just and updated version of the original patch [3]. The issue with the original path is that MetaspaceGC::allowed_expansion can, during special circumstances, allow the Metaspace high-water mark (HWM aka MetaspaceGC::_capacity_until_GC) to become larger than the amount of committed memory. Keeping the HWM less than or equal to how much we have committed is an invariant we would like to be true in Metaspace at all times to make it easier to reason about the sizing heuristics. This patch changes MetaspaceGC::allowed_expansion to always enforce the invariant that HWM <= committed_bytes. However, we still want to allow the VM to allocate as much metadata it wants during initialization. To achieve this, we set the HWM to MaxMetaspaceSize when initializing Metaspace and then reset the HWM to max(MetaspaceSize, HWM) once the initialization is done. Thanks StefanK for coming up with this approach! MetaspaceGC::allowed_expansion previously also allowed the VM to allocate as much metadata it wants when the GC lock is active and we are awaiting a GC. The new behaviour will be to try to expand the HWM and then allocate (the expand_and_allocate call in CollectorPolicy::satisfy_failed_metadata_allocation). Under very special circumstances this might cause a thread to now await the GC in GC_locker::stall_until_clear instead of getting its metadata and continue executing. Webrev: http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/ Testing: - JPRT - Ad-hoc testing (on all platforms): - Kitchensink - runThese - Parallel Class Loading testlist - Metaspace testlist - GC nightly testlist - OOM testlist - Quick testlist - JTREG tests in both VM and JDK Thanks, Erik [0]: https://bugs.openjdk.java.net/browse/JDK-8034852 [1]: https://bugs.openjdk.java.net/browse/JDK-8042933 [2]: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013716.html [3]: http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ From lois.foltan at oracle.com Wed May 28 11:44:00 2014 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 28 May 2014 07:44:00 -0400 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <53851037.8040801@oracle.com> References: <53851037.8040801@oracle.com> Message-ID: <5385CC00.70800@oracle.com> Hi Mikael, Looks good. One question concerning a change in nativeInst_sparc.cpp. It seems that in most cases "0x%x" was changed in favor of INTPTR_FORMAT, but in NativeMovConstRegPatching::print() you did the opposite change, INTPTR_FORMAT to "0x%x". Just curious if there was a reason for that? It would be more consistent for example, with your change to NativeMoveConstReg::print(), if it was left as an INTPTR_FORMAT. Minor - copyrights on all files need to be updated. Thanks, Lois On 5/27/2014 6:22 PM, Mikael Vidstedt wrote: > > Please review the below change which addresses a number of print > format/argument mismatches when compiling linux/sparc using gcc. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ > > Most of these are straightforward changes to INTPTR_FORMAT & adding > p2i. A couple of changes worth bringing up: > > * nativeInst_sparc.cpp > > I changed the format of the offset in mov related prints (two > occurrences) to %d, since it's really a signed decimal int. This will > hopefully reduce confusion when the offset is negative, where > previously it would have looked like a relatively large positive offset. > > * macroAssembler_sparc.cpp > > The new code is in line with other similar patterns in the same file, > but the ugly cast should IMHO really be rewritten to cast through a > union. RFE? > > Cheers, > Mikael > From stefan.karlsson at oracle.com Wed May 28 11:56:48 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 28 May 2014 13:56:48 +0200 Subject: 8042933: assert(capacity_until_gc >= committed_bytes) failed In-Reply-To: <4471789.3Gp5iV3Z2a@ehelin-desktop> References: <4471789.3Gp5iV3Z2a@ehelin-desktop> Message-ID: <5385CF00.2070605@oracle.com> On 2014-05-28 13:10, Erik Helin wrote: > Hi all, > > this is the second attemp to fix JDK-8034852 [0], but since I backed > out the previous attempt, I have to use a new bug [1] (you can't use > the same bug-id for two different committs). > > For a background to the original problem, please see the old email > thread [2]. This patch is just and updated version of the original > patch [3]. The issue with the original path is that > MetaspaceGC::allowed_expansion can, during special circumstances, > allow the Metaspace high-water mark (HWM aka > MetaspaceGC::_capacity_until_GC) to become larger than the amount of > committed memory. Keeping the HWM less than or equal to how much we > have committed is an invariant we would like to be true in Metaspace > at all times to make it easier to reason about the sizing heuristics. > > This patch changes MetaspaceGC::allowed_expansion to always enforce > the invariant that HWM <= committed_bytes. However, we still want to > allow the VM to allocate as much metadata it wants during > initialization. To achieve this, we set the HWM to MaxMetaspaceSize > when initializing Metaspace and then reset the HWM to > max(MetaspaceSize, HWM) once the initialization is done. Thanks > StefanK for coming up with this approach! > > MetaspaceGC::allowed_expansion previously also allowed the VM to > allocate as much metadata it wants when the GC lock is active and we > are awaiting a GC. The new behaviour will be to try to expand the HWM > and then allocate (the expand_and_allocate call in > CollectorPolicy::satisfy_failed_metadata_allocation). Under very > special circumstances this might cause a thread to now await the GC in > GC_locker::stall_until_clear instead of getting its metadata and > continue executing. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/ http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/src/share/vm/memory/metaspace.hpp.udiff.html Could you add a comment explaining why we use MaxMetaspaceSize here, instead of the MetaspaceSize: - static void initialize() { _capacity_until_GC = MetaspaceSize; } + static void initialize() { _capacity_until_GC = MaxMetaspaceSize; } + static void post_initialize(); Maybe it would be better to move the implementation of initialize() to metaspace.cpp and place it together with the post_initialize() function? I think that would make it more apparent that we just setcapacity_until_GC temporarily to MaxMetaspaceSize. http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/src/share/vm/runtime/thread.cpp.udiff.html Could add a blank line between these two functions, so that they are not grouped together? // Set flag that basic initialization has completed. Used by exceptions and various // debug stuff, that does not work until all basic classes have been initialized. set_init_completed(); + Metaspace::post_initialize(); Otherwise, this looks good. thanks, StefanK > > Testing: > - JPRT > - Ad-hoc testing (on all platforms): > - Kitchensink > - runThese > - Parallel Class Loading testlist > - Metaspace testlist > - GC nightly testlist > - OOM testlist > - Quick testlist > - JTREG tests in both VM and JDK > > Thanks, > Erik > > [0]: https://bugs.openjdk.java.net/browse/JDK-8034852 > [1]: https://bugs.openjdk.java.net/browse/JDK-8042933 > [2]: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013716.html > [3]: http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ > From erik.helin at oracle.com Wed May 28 12:29:31 2014 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 28 May 2014 14:29:31 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps In-Reply-To: <537ED5FC.6080702@oracle.com> References: <537C7264.70200@oracle.com> <537DAA00.2010104@oracle.com> <537ED5FC.6080702@oracle.com> Message-ID: <2198629.dbW8eWbo0l@ehelin-desktop> Hi Bengt, just a first quick general comment: would it make sense for GCId to be in its own file or class, so the GC logging don't have to depend on the GC trace framework? Thanks, Erik On Friday 23 May 2014 07.00.44 Bengt Rutisson wrote: > Hi Jesper and everyone else, > > After explaining the problem with GenCollectedHeap in my last email I > realized that a different approach would be to just assume that the next > GC id will be the right one when we start the logging before we have > incremented the GC id. I think that is a nicer way of doing it than > "wasting" a GC id as I did before. > > Here's an updated webrev with that changed: > http://cr.openjdk.java.net/~brutisso/8043607/webrev.01/ > > Here's the diff compared to the last webrev: > http://cr.openjdk.java.net/~brutisso/8043607/webrev.00-01-diff/ > > Thanks, > Bengt > > On 5/22/14 9:40 AM, Bengt Rutisson wrote: > > Hi Jesper, > > > > On 5/21/14 7:44 PM, Jesper Wilhelmsson wrote: > >> Hi Bengt, > >> > >> Good change! Thanks for doing this! > > > > Thanks for looking at it. > > > >> A few thoughts: > >> > >> * There are a few places that passes both a GCTimer and now a GCId. > >> Would it make sense to pass an info-object of some kind that contains > >> misc info that is used for logging/tracing? I'm not suggesting that > >> you should implement that in this change, just a thought for the future. > > > > Yes, I think that may be a good future cleanup to do. But would be > > good as a separate change, just as you point out. > > > >> * In genCollectdHeap.cpp you increase the gc id only if > >> PrintGCDetails is enabled. I assume this is the only place where a gc > >> id is used when running collectors that use > >> GenCollectedHeap::do_collection(). If not, the id created and used in > >> other places would be different depending on if PrintGCDetails was > >> enabled or not. > > > > Yes, this is a work around for the strange way the logging for > > GenCollectedHeap works. We do completely different timing and logging > > for PrintGC and PrintGCDetails. That the logging is different is just > > strange. That the the timing is different is in my opinion a bug. The > > pause time reported is shorter if you run with PrintGC than if you run > > with PrintGCDetails. > > > > This mostly affect CMS. The logging for CMS has several other issues > > as well. I added one comment about that in the bug report. So, I think > > it would be good to give the CMS logging an overhaul, but that should > > definitely be a separate change :) And maybe we should hold off with > > that until the unified logging framework is in place. > > > >> Anyway, looks good! > > > > Thanks! > > > > Bengt > > > >> Ship it! > >> /Jesper > >> > >> Bengt Rutisson skrev 21/5/14 11:31: > >>> Hi everyone, > >>> > >>> Can I have some reviews for this patch: > >>> http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8043607 > >>> > >>> This is mostly a GC change but I am sending it to the broader list > >>> since I added > >>> a method to ostream.hpp/cpp. We had this code duplicated in many > >>> places in the > >>> GC code: > >>> > >>> gclog_or_tty->date_stamp(PrintGCDateStamps); > >>> gclog_or_tty->stamp(PrintGCTimeStamps); > >>> > >>> And with my patch I had to add one more section to this code > >>> duplication. > >>> Instead I wrapped all three calls in a method called gclog_stamp(). > >>> It is a bit > >>> hard to know where to place that method. ostream seems like the > >>> wrong place for > >>> GC specific code, but there is already lots of GC specific code > >>> there and even > >>> gclog_or_tty is declared there. So I added the method there for now: > >>> > >>> void outputStream::gclog_stamp(uint gc_id) { > >>> > >>> date_stamp(PrintGCDateStamps); > >>> stamp(PrintGCTimeStamps); > >>> if (PrintGCID) { > >>> > >>> print("#%u: ", gc_id); > >>> > >>> } > >>> > >>> } > >>> > >>> Other than that this is a pure GC change. > >>> > >>> > >>> Background: > >>> > >>> With event based tracing we introduced a GC id field on all GC > >>> events. This id > >>> is a constantly increasing number associated with each GC. I figured > >>> we can use > >>> the this GC id as a decoration for the HotSpot GC logging similarly > >>> to what we > >>> do with PrintGCTimeStamps and PrintGCDateStamps. > >>> > >>> A nice side effect of this is that it will make it easy to map an > >>> event in the > >>> event tracing to an entry in the GC log. Currently this is > >>> impossible to do > >>> deterministically since the logging and event tracing use different > >>> timing. > >>> > >>> I propose to add a flag called PrintGCID. Here's an example log > >>> entry for GC > >>> number 4 running with -XX:+PrintGCID: > >>> > >>> #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), > >>> 0,2373420 secs] > >>> > >>> It is a decoration similar to the timestamps. Here's what it looks > >>> like if you > >>> add -XX:+PrintGCTimeStamps: > >>> > >>> 3,921: #4: [GC pause (G1 Evacuation Pause) (young) > >>> 185M->212M(502M), 0,2373420 > >>> secs] > >>> > >>> And here's what it will look like if you add -XX:+PrintGCDateStamps: > >>> > >>> 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation > >>> Pause) > >>> (young) 185M->212M(502M), 0,2373420 secs] > >>> > >>> > >>> Here's a little longer example running ParallelGC: > >>> > >>> #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] > >>> #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] > >>> #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] > >>> #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 > >>> secs] > >>> #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] > >>> #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 > >>> secs] > >>> > >>> Note that for the concurrent GCs this gives a good way of seeing the > >>> concurrent > >>> action. Here is a G1 example: > >>> > >>> #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), > >>> 0,0383157 secs] > >>> #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), > >>> 0,0412220 secs] > >>> #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), > >>> 0,0426144 secs] > >>> #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), > >>> 0,1467633 secs] > >>> #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), > >>> 0,1627465 secs] > >>> #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), > >>> 0,2376728 secs] > >>> #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) > >>> 255M->269M(1004M), > >>> 0,2623779 secs] > >>> #7: [GC concurrent-root-region-scan-start] > >>> #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] > >>> #7: [GC concurrent-mark-start] > >>> #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), > >>> 0,3007405 secs] > >>> #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), > >>> 0,4063937 secs] > >>> #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), > >>> 0,7500609 secs] > >>> #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), > >>> 1,5464884 secs] > >>> #7: [GC concurrent-mark-end, 3,9464599 secs] > >>> #7: [GC remark, 0,0240462 secs] > >>> #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] > >>> #7: [GC concurrent-cleanup-start] > >>> #7: [GC concurrent-cleanup-end, 0,0000228 secs] > >>> #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), > >>> 2,2464423 secs] > >>> > >>> Notice how the complete concurrent cycle is decorated with id #7. > >>> That makes it > >>> possible to follow the concurrent work more easily and grep it out etc. > >>> > >>> > >>> Here's a similar thing for CMS where the concurrent cycle #22 has a > >>> young GC #23 > >>> in the middle of it: > >>> > >>> #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), > >>> 1,4558792 secs] > >>> #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] > >>> #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), > >>> 0,7909961 secs] > >>> #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] > >>> > >>> > >>> I think this is very helpful information to have, so I would like to > >>> push this > >>> with the new flag PrintGCID as true by default. I'm sure some > >>> parsers will > >>> break, but it is an easy work around to run with -XX:-PrintGCID to > >>> get the old > >>> format. It should even be pretty easy to do post processing on a log > >>> file to > >>> remove the GC id information. Having it on by default makes it > >>> possible for us > >>> to use this information and in particular be able to map it to the > >>> JFR events we > >>> see in MissionControl. > >>> > >>> I would also like to backport this to 8u40, but in that case with > >>> the default > >>> value of PrintGCID as false. > >>> > >>> Thanks, > >>> Bengt From jesper.wilhelmsson at oracle.com Wed May 28 12:42:51 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 28 May 2014 14:42:51 +0200 Subject: RFR (M): JDK-8043607: Add a GC id as a log decoration similar to PrintGCTimeStamps In-Reply-To: <537ED5FC.6080702@oracle.com> References: <537C7264.70200@oracle.com> <537CE5FB.9060003@oracle.com> <537DAA00.2010104@oracle.com> <537ED5FC.6080702@oracle.com> Message-ID: <5385D9CB.5070604@oracle.com> I'm fine with this change. There is a typo present in both new comments in genCollectedHeap.cpp, "thet" should be "the". /Jesper Bengt Rutisson skrev 23/5/14 07:00: > > Hi Jesper and everyone else, > > After explaining the problem with GenCollectedHeap in my last email I realized > that a different approach would be to just assume that the next GC id will be > the right one when we start the logging before we have incremented the GC id. I > think that is a nicer way of doing it than "wasting" a GC id as I did before. > > Here's an updated webrev with that changed: > http://cr.openjdk.java.net/~brutisso/8043607/webrev.01/ > > Here's the diff compared to the last webrev: > http://cr.openjdk.java.net/~brutisso/8043607/webrev.00-01-diff/ > > Thanks, > Bengt > > On 5/22/14 9:40 AM, Bengt Rutisson wrote: >> >> Hi Jesper, >> >> On 5/21/14 7:44 PM, Jesper Wilhelmsson wrote: >>> Hi Bengt, >>> >>> Good change! Thanks for doing this! >> >> Thanks for looking at it. >> >>> >>> A few thoughts: >>> >>> * There are a few places that passes both a GCTimer and now a GCId. Would it >>> make sense to pass an info-object of some kind that contains misc info that >>> is used for logging/tracing? I'm not suggesting that you should implement >>> that in this change, just a thought for the future. >> >> Yes, I think that may be a good future cleanup to do. But would be good as a >> separate change, just as you point out. >> >>> >>> >>> * In genCollectdHeap.cpp you increase the gc id only if PrintGCDetails is >>> enabled. I assume this is the only place where a gc id is used when running >>> collectors that use GenCollectedHeap::do_collection(). If not, the id created >>> and used in other places would be different depending on if PrintGCDetails >>> was enabled or not. >> >> Yes, this is a work around for the strange way the logging for >> GenCollectedHeap works. We do completely different timing and logging for >> PrintGC and PrintGCDetails. That the logging is different is just strange. >> That the the timing is different is in my opinion a bug. The pause time >> reported is shorter if you run with PrintGC than if you run with PrintGCDetails. >> >> This mostly affect CMS. The logging for CMS has several other issues as well. >> I added one comment about that in the bug report. So, I think it would be good >> to give the CMS logging an overhaul, but that should definitely be a separate >> change :) And maybe we should hold off with that until the unified logging >> framework is in place. >> >>> >>> Anyway, looks good! >> Thanks! >> >> Bengt >> >>> Ship it! >>> /Jesper >>> >>> >>> Bengt Rutisson skrev 21/5/14 11:31: >>>> >>>> Hi everyone, >>>> >>>> Can I have some reviews for this patch: >>>> http://cr.openjdk.java.net/~brutisso/8043607/webrev.00/ >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8043607 >>>> >>>> This is mostly a GC change but I am sending it to the broader list since I >>>> added >>>> a method to ostream.hpp/cpp. We had this code duplicated in many places in the >>>> GC code: >>>> >>>> gclog_or_tty->date_stamp(PrintGCDateStamps); >>>> gclog_or_tty->stamp(PrintGCTimeStamps); >>>> >>>> And with my patch I had to add one more section to this code duplication. >>>> Instead I wrapped all three calls in a method called gclog_stamp(). It is a bit >>>> hard to know where to place that method. ostream seems like the wrong place for >>>> GC specific code, but there is already lots of GC specific code there and even >>>> gclog_or_tty is declared there. So I added the method there for now: >>>> >>>> void outputStream::gclog_stamp(uint gc_id) { >>>> date_stamp(PrintGCDateStamps); >>>> stamp(PrintGCTimeStamps); >>>> if (PrintGCID) { >>>> print("#%u: ", gc_id); >>>> } >>>> } >>>> >>>> Other than that this is a pure GC change. >>>> >>>> >>>> Background: >>>> >>>> With event based tracing we introduced a GC id field on all GC events. This id >>>> is a constantly increasing number associated with each GC. I figured we can use >>>> the this GC id as a decoration for the HotSpot GC logging similarly to what we >>>> do with PrintGCTimeStamps and PrintGCDateStamps. >>>> >>>> A nice side effect of this is that it will make it easy to map an event in the >>>> event tracing to an entry in the GC log. Currently this is impossible to do >>>> deterministically since the logging and event tracing use different timing. >>>> >>>> I propose to add a flag called PrintGCID. Here's an example log entry for GC >>>> number 4 running with -XX:+PrintGCID: >>>> >>>> #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 secs] >>>> >>>> It is a decoration similar to the timestamps. Here's what it looks like if you >>>> add -XX:+PrintGCTimeStamps: >>>> >>>> 3,921: #4: [GC pause (G1 Evacuation Pause) (young) 185M->212M(502M), 0,2373420 >>>> secs] >>>> >>>> And here's what it will look like if you add -XX:+PrintGCDateStamps: >>>> >>>> 2014-05-09T13:03:59.384+0200: 3,921: #4: [GC pause (G1 Evacuation Pause) >>>> (young) 185M->212M(502M), 0,2373420 secs] >>>> >>>> >>>> Here's a little longer example running ParallelGC: >>>> >>>> #0: [GC (Allocation Failure) 129024K->2980K(493056K), 0,1050244 secs] >>>> #1: [GC (Allocation Failure) 132004K->3731K(622080K), 0,0389474 secs] >>>> #2: [GC (Allocation Failure) 261779K->27153K(622080K), 0,2057294 secs] >>>> #3: [GC (Allocation Failure) 285201K->224350K(880128K), 0,2410144 secs] >>>> #4: [Full GC (Ergonomics) 224350K->219198K(1154560K), 2,7104481 secs] >>>> #5: [GC (Allocation Failure) 735294K->638073K(863744K), 0,7707819 secs] >>>> >>>> Note that for the concurrent GCs this gives a good way of seeing the concurrent >>>> action. Here is a G1 example: >>>> >>>> #0: [GC pause (G1 Evacuation Pause) (young) 24M->2881K(502M), 0,0383157 secs] >>>> #1: [GC pause (G1 Evacuation Pause) (young) 48M->4745K(502M), 0,0412220 secs] >>>> #2: [GC pause (G1 Evacuation Pause) (young) 136M->9618K(502M), 0,0426144 secs] >>>> #3: [GC pause (G1 Evacuation Pause) (young) 248M->143M(502M), 0,1467633 secs] >>>> #4: [GC pause (G1 Evacuation Pause) (young) 210M->199M(502M), 0,1627465 secs] >>>> #5: [GC pause (G1 Evacuation Pause) (young) 213M->235M(502M), 0,2376728 secs] >>>> #6: [GC pause (G1 Evacuation Pause) (young) (initial-mark) 255M->269M(1004M), >>>> 0,2623779 secs] >>>> #7: [GC concurrent-root-region-scan-start] >>>> #7: [GC concurrent-root-region-scan-end, 0,0158488 secs] >>>> #7: [GC concurrent-mark-start] >>>> #8: [GC pause (G1 Evacuation Pause) (young) 315M->330M(2008M), 0,3007405 secs] >>>> #9: [GC pause (G1 Evacuation Pause) (young) 422M->418M(3212M), 0,4063937 secs] >>>> #10: [GC pause (G1 Evacuation Pause) (young) 564M->572M(4176M), 0,7500609 >>>> secs] >>>> #11: [GC pause (G1 Evacuation Pause) (young) 760M->756M(4946M), 1,5464884 >>>> secs] >>>> #7: [GC concurrent-mark-end, 3,9464599 secs] >>>> #7: [GC remark, 0,0240462 secs] >>>> #7: [GC cleanup 801M->800M(4946M), 0,0108146 secs] >>>> #7: [GC concurrent-cleanup-start] >>>> #7: [GC concurrent-cleanup-end, 0,0000228 secs] >>>> #12: [GC pause (G1 Evacuation Pause) (young) 977M->978M(5562M), 2,2464423 >>>> secs] >>>> >>>> Notice how the complete concurrent cycle is decorated with id #7. That makes it >>>> possible to follow the concurrent work more easily and grep it out etc. >>>> >>>> >>>> Here's a similar thing for CMS where the concurrent cycle #22 has a young GC >>>> #23 >>>> in the middle of it: >>>> >>>> #21: [GC (Allocation Failure) 1621551K->1107977K(2395632K), 1,4558792 secs] >>>> #22: [GC (CMS Initial Mark) 1336515K(2452584K), 0,1107938 secs] >>>> #23: [GC (Allocation Failure) 1768969K->1184500K(2452584K), 0,7909961 secs] >>>> #22: [GC (CMS Final Remark) 1304871K(2452584K), 0,2353840 secs] >>>> >>>> >>>> I think this is very helpful information to have, so I would like to push this >>>> with the new flag PrintGCID as true by default. I'm sure some parsers will >>>> break, but it is an easy work around to run with -XX:-PrintGCID to get the old >>>> format. It should even be pretty easy to do post processing on a log file to >>>> remove the GC id information. Having it on by default makes it possible for us >>>> to use this information and in particular be able to map it to the JFR >>>> events we >>>> see in MissionControl. >>>> >>>> I would also like to backport this to 8u40, but in that case with the default >>>> value of PrintGCID as false. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>> >>>> >>>> >>>> >> > From staffan.larsen at oracle.com Wed May 28 14:50:28 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 28 May 2014 16:50:28 +0200 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders Message-ID: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> This change adds a new Diagnostic Command to list all ClassLoaders and some statistics for each classloader. The command is called ?GC.classloader_stats? and the information listed is: * An id for the ClassLoader. This is the pointer to the Klass of the ClassLoader. The reason for using the Klass* (instead of the oop) is that it is stable across invocations. * The id of the ClassLoader?s parent ClassLoader. * The pointer to the ClassLoaderData structure in the VM. This can be useful for debugging. * The number of classes loaded by the ClassLoader. * The total size of all allocated metaspace chunks for the ClassLoader. * The total size of all allocated metaspace blocks for the ClassLoader. If there are anonymous classes (invokedynamic classes) attributed to the ClassLoader, the following additional information is listed: * The number of anonymous classes loaded by the ClassLoader. * The total size of all allocated metaspace chunks for anonymous classes in the ClassLoader. * The total size of all allocated metaspace blocks for anonymous classes in the ClassLoader. The information is gathered during a safe point to guarantee that the data structures are consistent. I have added a small test and have run this through jprt. A CCC request has been filed. webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8044107 Example output: ClassLoader Parent CLData* Classes ChunkSz BlockSz Type 0x00000007c002d908 0x0000000000000000 0x0000000000000000 0 0 0 sun.misc.Launcher$ExtClassLoader 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 4694016 4241312 37 75776 50928 + invokedynamic classes 0x00000007c0061028 0x00000007c0036878 0x00007fb239c2de60 1 6144 1976 ClassLoaderStatsTest$DummyClassLoader 1 2048 1288 + invokedynamic classes 0x00000007c0036878 0x00000007c002d908 0x00007fb239e10fc0 8 88064 31544 sun.misc.Launcher$AppClassLoader Total = 4 808 4866048 4327048 ChunkSz: Total size of all allocated metaspace chunks BlockSz: Total size of all allocated metaspace blocks (each chunk has several blocks) Thanks, /Staffan From mikael.vidstedt at oracle.com Wed May 28 15:05:11 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 28 May 2014 08:05:11 -0700 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <53857673.9040905@oracle.com> References: <53851037.8040801@oracle.com> <53857673.9040905@oracle.com> Message-ID: <5385FB27.4080503@oracle.com> On 2014-05-27 22:38, David Holmes wrote: > Looks okay. Can't comment on the changed formats in > nativeInst_sparc.cpp. The ugly cast has precedent so I wouldn't lose > sleep over it. > > Did the change in src/os_cpu/linux_sparc/vm/vm_version_linux_sparc.cpp > fix a warning or do you just prefer using the raw array name? It fixes an actual warning saying the format expects a "char *" but gets a "char (*)[128]". Cheers, Mikael > > David H. > > On 28/05/2014 8:22 AM, Mikael Vidstedt wrote: >> >> Please review the below change which addresses a number of print >> format/argument mismatches when compiling linux/sparc using gcc. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 >> Webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ >> >> Most of these are straightforward changes to INTPTR_FORMAT & adding p2i. >> A couple of changes worth bringing up: >> >> * nativeInst_sparc.cpp >> >> I changed the format of the offset in mov related prints (two >> occurrences) to %d, since it's really a signed decimal int. This will >> hopefully reduce confusion when the offset is negative, where previously >> it would have looked like a relatively large positive offset. >> >> * macroAssembler_sparc.cpp >> >> The new code is in line with other similar patterns in the same file, >> but the ugly cast should IMHO really be rewritten to cast through a >> union. RFE? >> >> Cheers, >> Mikael >> From mikael.vidstedt at oracle.com Wed May 28 15:09:26 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 28 May 2014 08:09:26 -0700 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <5385CC00.70800@oracle.com> References: <53851037.8040801@oracle.com> <5385CC00.70800@oracle.com> Message-ID: <5385FC26.1050408@oracle.com> On 2014-05-28 04:44, Lois Foltan wrote: > Hi Mikael, > > Looks good. One question concerning a change in > nativeInst_sparc.cpp. It seems that in most cases "0x%x" was changed > in favor of INTPTR_FORMAT, but in NativeMovConstRegPatching::print() > you did the opposite change, INTPTR_FORMAT to "0x%x". Just curious if > there was a reason for that? It would be more consistent for example, > with your change to NativeMoveConstReg::print(), if it was left as an > INTPTR_FORMAT. In the cases where I changed INTPTR_FORMAT to 0x%x it is because the argument is in fact an int. While I could upcast to an intptr_t instead I personally think that is more confusing than it is helpful. Cheers, Mikael > > Minor - copyrights on all files need to be updated. > > Thanks, > Lois > > On 5/27/2014 6:22 PM, Mikael Vidstedt wrote: >> >> Please review the below change which addresses a number of print >> format/argument mismatches when compiling linux/sparc using gcc. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 >> Webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ >> >> Most of these are straightforward changes to INTPTR_FORMAT & adding >> p2i. A couple of changes worth bringing up: >> >> * nativeInst_sparc.cpp >> >> I changed the format of the offset in mov related prints (two >> occurrences) to %d, since it's really a signed decimal int. This will >> hopefully reduce confusion when the offset is negative, where >> previously it would have looked like a relatively large positive offset. >> >> * macroAssembler_sparc.cpp >> >> The new code is in line with other similar patterns in the same file, >> but the ugly cast should IMHO really be rewritten to cast through a >> union. RFE? >> >> Cheers, >> Mikael >> > From lois.foltan at oracle.com Wed May 28 15:40:43 2014 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 28 May 2014 11:40:43 -0400 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <5385FC26.1050408@oracle.com> References: <53851037.8040801@oracle.com> <5385CC00.70800@oracle.com> <5385FC26.1050408@oracle.com> Message-ID: <5386037B.90203@oracle.com> On 5/28/2014 11:09 AM, Mikael Vidstedt wrote: > > On 2014-05-28 04:44, Lois Foltan wrote: >> Hi Mikael, >> >> Looks good. One question concerning a change in >> nativeInst_sparc.cpp. It seems that in most cases "0x%x" was changed >> in favor of INTPTR_FORMAT, but in NativeMovConstRegPatching::print() >> you did the opposite change, INTPTR_FORMAT to "0x%x". Just curious >> if there was a reason for that? It would be more consistent for >> example, with your change to NativeMoveConstReg::print(), if it was >> left as an INTPTR_FORMAT. > > In the cases where I changed INTPTR_FORMAT to 0x%x it is because the > argument is in fact an int. While I could upcast to an intptr_t > instead I personally think that is more confusing than it is helpful. Got it, thank you for the explanation. Looks good. Lois > > Cheers, > Mikael > >> >> Minor - copyrights on all files need to be updated. >> >> Thanks, >> Lois >> >> On 5/27/2014 6:22 PM, Mikael Vidstedt wrote: >>> >>> Please review the below change which addresses a number of print >>> format/argument mismatches when compiling linux/sparc using gcc. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 >>> Webrev: >>> http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ >>> >>> Most of these are straightforward changes to INTPTR_FORMAT & adding >>> p2i. A couple of changes worth bringing up: >>> >>> * nativeInst_sparc.cpp >>> >>> I changed the format of the offset in mov related prints (two >>> occurrences) to %d, since it's really a signed decimal int. This >>> will hopefully reduce confusion when the offset is negative, where >>> previously it would have looked like a relatively large positive >>> offset. >>> >>> * macroAssembler_sparc.cpp >>> >>> The new code is in line with other similar patterns in the same >>> file, but the ugly cast should IMHO really be rewritten to cast >>> through a union. RFE? >>> >>> Cheers, >>> Mikael >>> >> > From george.triantafillou at oracle.com Wed May 28 17:10:28 2014 From: george.triantafillou at oracle.com (George Triantafillou) Date: Wed, 28 May 2014 13:10:28 -0400 Subject: RFR (L) 8044140: [TESTBUG] Create NMT (Native Memory Tracking) tests for NMT2 Message-ID: <53861884.1050407@oracle.com> Please review these new and modified tests for the redesigned Native Memory Tracking (NMT) functionality (8028541: Native Memory Tracking enhancement ). NMT was redesigned internally to address scalability issues in the first implementation. Existing functionality has not changed, with the exception of the removal of the "-AutoshutdownNMT" flag. Please note that the following new stress tests are excluded with the JTREG @ignore tag: test/runtime/NMT/MallocSiteHashOverflow.java test/runtime/NMT/MallocStressTest.java These tests can be run by using the jtreg override option "-ignore:run". The new key "@key stress" was also added to these tests. JBS: https://bugs.openjdk.java.net/browse/JDK-8044140 Webrev: http://cr.openjdk.java.net/~gtriantafill/bug_8044140/webrev.00/ Testing: JTREG to exercise the added tests Thank you, George From zhengyu.gu at oracle.com Wed May 28 17:25:52 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Wed, 28 May 2014 13:25:52 -0400 Subject: RFR (L) 8044140: [TESTBUG] Create NMT (Native Memory Tracking) tests for NMT2 In-Reply-To: <53861884.1050407@oracle.com> References: <53861884.1050407@oracle.com> Message-ID: <53861C20.60303@oracle.com> Hi, JcmdScale.java: There is no change, should not update copyright ... AutoshutdownNMT.java line #42 42 output.shouldContain("Unrecognized VM option 'AutoshutdownNMT'"); Should be 42 output.shouldContain("Unrecognized VM option 'AutoShutdownNMT'"); Also -XX:-AutoshutdownNMT => -XX:-AutoShutdownNMT -Zhengyu On 5/28/2014 1:10 PM, George Triantafillou wrote: > Please review these new and modified tests for the redesigned Native > Memory Tracking (NMT) functionality (8028541: Native Memory Tracking > enhancement ). NMT > was redesigned internally to address scalability issues in the first > implementation. Existing functionality has not changed, with the > exception of the removal of the "-AutoshutdownNMT" flag. > > Please note that the following new stress tests are excluded with the > JTREG @ignore tag: > > test/runtime/NMT/MallocSiteHashOverflow.java > test/runtime/NMT/MallocStressTest.java > > These tests can be run by using the jtreg override option > "-ignore:run". The new key "@key stress" was also added to these tests. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8044140 > Webrev: > http://cr.openjdk.java.net/~gtriantafill/bug_8044140/webrev.00/ > > Testing: JTREG to exercise the added tests > > Thank you, > > George From christian.thalinger at oracle.com Wed May 28 19:39:30 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 28 May 2014 12:39:30 -0700 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> Message-ID: On May 28, 2014, at 7:50 AM, Staffan Larsen wrote: > This change adds a new Diagnostic Command to list all ClassLoaders and some statistics for each classloader. The command is called ?GC.classloader_stats? and the information listed is: > > * An id for the ClassLoader. This is the pointer to the Klass of the ClassLoader. The reason for using the Klass* (instead of the oop) is that it is stable across invocations. > * The id of the ClassLoader?s parent ClassLoader. > * The pointer to the ClassLoaderData structure in the VM. This can be useful for debugging. > * The number of classes loaded by the ClassLoader. > * The total size of all allocated metaspace chunks for the ClassLoader. > * The total size of all allocated metaspace blocks for the ClassLoader. > > If there are anonymous classes (invokedynamic classes) attributed to the ClassLoader, the following additional information is listed: > * The number of anonymous classes loaded by the ClassLoader. > * The total size of all allocated metaspace chunks for anonymous classes in the ClassLoader. > * The total size of all allocated metaspace blocks for anonymous classes in the ClassLoader. > > The information is gathered during a safe point to guarantee that the data structures are consistent. > > I have added a small test and have run this through jprt. A CCC request has been filed. > > webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8044107 > > Example output: > > ClassLoader Parent CLData* Classes ChunkSz BlockSz Type > 0x00000007c002d908 0x0000000000000000 0x0000000000000000 0 0 0 sun.misc.Launcher$ExtClassLoader > 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 4694016 4241312 > 37 75776 50928 + invokedynamic classes You shouldn?t name these ?invokedynamic classes?; they can be used for different purposes. ?anonymous classes? is the correct name. > 0x00000007c0061028 0x00000007c0036878 0x00007fb239c2de60 1 6144 1976 ClassLoaderStatsTest$DummyClassLoader > 1 2048 1288 + invokedynamic classes > 0x00000007c0036878 0x00000007c002d908 0x00007fb239e10fc0 8 88064 31544 sun.misc.Launcher$AppClassLoader > Total = 4 808 4866048 4327048 > ChunkSz: Total size of all allocated metaspace chunks > BlockSz: Total size of all allocated metaspace blocks (each chunk has several blocks) > > > Thanks, > /Staffan From staffan.larsen at oracle.com Wed May 28 19:46:27 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 28 May 2014 21:46:27 +0200 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> Message-ID: On 28 maj 2014, at 21:39, Christian Thalinger wrote: > > On May 28, 2014, at 7:50 AM, Staffan Larsen wrote: > >> This change adds a new Diagnostic Command to list all ClassLoaders and some statistics for each classloader. The command is called ?GC.classloader_stats? and the information listed is: >> >> * An id for the ClassLoader. This is the pointer to the Klass of the ClassLoader. The reason for using the Klass* (instead of the oop) is that it is stable across invocations. >> * The id of the ClassLoader?s parent ClassLoader. >> * The pointer to the ClassLoaderData structure in the VM. This can be useful for debugging. >> * The number of classes loaded by the ClassLoader. >> * The total size of all allocated metaspace chunks for the ClassLoader. >> * The total size of all allocated metaspace blocks for the ClassLoader. >> >> If there are anonymous classes (invokedynamic classes) attributed to the ClassLoader, the following additional information is listed: >> * The number of anonymous classes loaded by the ClassLoader. >> * The total size of all allocated metaspace chunks for anonymous classes in the ClassLoader. >> * The total size of all allocated metaspace blocks for anonymous classes in the ClassLoader. >> >> The information is gathered during a safe point to guarantee that the data structures are consistent. >> >> I have added a small test and have run this through jprt. A CCC request has been filed. >> >> webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8044107 >> >> Example output: >> >> ClassLoader Parent CLData* Classes ChunkSz BlockSz Type >> 0x00000007c002d908 0x0000000000000000 0x0000000000000000 0 0 0 sun.misc.Launcher$ExtClassLoader >> 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 4694016 4241312 >> 37 75776 50928 + invokedynamic classes > > You shouldn?t name these ?invokedynamic classes?; they can be used for different purposes. ?anonymous classes? is the correct name. Hmm, yeah. Not a good name, I know. The problem with ?anonymous classes? is that for a Java developer, that term is already taken and means something completely different. This is very unfortunate and creates a lot of confusion. My first version called them ?lambdas?, but that was even more wrong. Better names are welcome! /Staffan > >> 0x00000007c0061028 0x00000007c0036878 0x00007fb239c2de60 1 6144 1976 ClassLoaderStatsTest$DummyClassLoader >> 1 2048 1288 + invokedynamic classes >> 0x00000007c0036878 0x00000007c002d908 0x00007fb239e10fc0 8 88064 31544 sun.misc.Launcher$AppClassLoader >> Total = 4 808 4866048 4327048 >> ChunkSz: Total size of all allocated metaspace chunks >> BlockSz: Total size of all allocated metaspace blocks (each chunk has several blocks) >> >> >> Thanks, >> /Staffan From christian.thalinger at oracle.com Wed May 28 19:48:14 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 28 May 2014 12:48:14 -0700 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <53851037.8040801@oracle.com> References: <53851037.8040801@oracle.com> Message-ID: <434ACD4F-CAD7-426D-9D91-224D7556D470@oracle.com> This made me curious because I was expecting a cast too: void NativeMovRegMemPatching::print() { if (is_immediate()) { ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %x]", instruction_address(), offset()); ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %d]", p2i(instruction_address()), offset()); Turns out that NativeMovConstRegPatching::data() returns an int: int NativeMovConstRegPatching::data() const { #ifdef _LP64 return data64(addr_at(sethi_offset), long_at(add_offset)); #else return data32(long_at(sethi_offset), long_at(add_offset)); #endif } but on 64-bit we truncate an intptr_t: static intptr_t data64( address pc, int arith_insn ) { That?s odd because in NativeMovRegMem we use intptr_t for offset exactly because of the non-immediate case. On May 27, 2014, at 3:22 PM, Mikael Vidstedt wrote: > > Please review the below change which addresses a number of print format/argument mismatches when compiling linux/sparc using gcc. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ > > Most of these are straightforward changes to INTPTR_FORMAT & adding p2i. A couple of changes worth bringing up: > > * nativeInst_sparc.cpp > > I changed the format of the offset in mov related prints (two occurrences) to %d, since it's really a signed decimal int. This will hopefully reduce confusion when the offset is negative, where previously it would have looked like a relatively large positive offset. > > * macroAssembler_sparc.cpp > > The new code is in line with other similar patterns in the same file, but the ugly cast should IMHO really be rewritten to cast through a union. RFE? > > Cheers, > Mikael > From christian.thalinger at oracle.com Wed May 28 19:49:57 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 28 May 2014 12:49:57 -0700 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> Message-ID: <7390BCB6-5572-481F-B984-A2AABB76280B@oracle.com> On May 28, 2014, at 12:46 PM, Staffan Larsen wrote: > > On 28 maj 2014, at 21:39, Christian Thalinger wrote: > >> >> On May 28, 2014, at 7:50 AM, Staffan Larsen wrote: >> >>> This change adds a new Diagnostic Command to list all ClassLoaders and some statistics for each classloader. The command is called ?GC.classloader_stats? and the information listed is: >>> >>> * An id for the ClassLoader. This is the pointer to the Klass of the ClassLoader. The reason for using the Klass* (instead of the oop) is that it is stable across invocations. >>> * The id of the ClassLoader?s parent ClassLoader. >>> * The pointer to the ClassLoaderData structure in the VM. This can be useful for debugging. >>> * The number of classes loaded by the ClassLoader. >>> * The total size of all allocated metaspace chunks for the ClassLoader. >>> * The total size of all allocated metaspace blocks for the ClassLoader. >>> >>> If there are anonymous classes (invokedynamic classes) attributed to the ClassLoader, the following additional information is listed: >>> * The number of anonymous classes loaded by the ClassLoader. >>> * The total size of all allocated metaspace chunks for anonymous classes in the ClassLoader. >>> * The total size of all allocated metaspace blocks for anonymous classes in the ClassLoader. >>> >>> The information is gathered during a safe point to guarantee that the data structures are consistent. >>> >>> I have added a small test and have run this through jprt. A CCC request has been filed. >>> >>> webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8044107 >>> >>> Example output: >>> >>> ClassLoader Parent CLData* Classes ChunkSz BlockSz Type >>> 0x00000007c002d908 0x0000000000000000 0x0000000000000000 0 0 0 sun.misc.Launcher$ExtClassLoader >>> 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 4694016 4241312 >>> 37 75776 50928 + invokedynamic classes >> >> You shouldn?t name these ?invokedynamic classes?; they can be used for different purposes. ?anonymous classes? is the correct name. > > Hmm, yeah. Not a good name, I know. The problem with ?anonymous classes? is that for a Java developer, that term is already taken and means something completely different. This is very unfortunate and creates a lot of confusion. My first version called them ?lambdas?, but that was even more wrong. Better names are welcome! These classes are defined via: public native Class defineAnonymousClass(Class hostClass, byte[] data, Object[] cpPatches); so this is what they are called. Sorry :-) > > /Staffan > >> >>> 0x00000007c0061028 0x00000007c0036878 0x00007fb239c2de60 1 6144 1976 ClassLoaderStatsTest$DummyClassLoader >>> 1 2048 1288 + invokedynamic classes >>> 0x00000007c0036878 0x00000007c002d908 0x00007fb239e10fc0 8 88064 31544 sun.misc.Launcher$AppClassLoader >>> Total = 4 808 4866048 4327048 >>> ChunkSz: Total size of all allocated metaspace chunks >>> BlockSz: Total size of all allocated metaspace blocks (each chunk has several blocks) >>> >>> >>> Thanks, >>> /Staffan From coleen.phillimore at oracle.com Wed May 28 19:51:55 2014 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 28 May 2014 15:51:55 -0400 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: <7390BCB6-5572-481F-B984-A2AABB76280B@oracle.com> References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> <7390BCB6-5572-481F-B984-A2AABB76280B@oracle.com> Message-ID: <53863E5B.6050205@oracle.com> On 5/28/14, 3:49 PM, Christian Thalinger wrote: > On May 28, 2014, at 12:46 PM, Staffan Larsen wrote: > >> On 28 maj 2014, at 21:39, Christian Thalinger wrote: >> >>> On May 28, 2014, at 7:50 AM, Staffan Larsen wrote: >>> >>>> This change adds a new Diagnostic Command to list all ClassLoaders and some statistics for each classloader. The command is called ?GC.classloader_stats? and the information listed is: >>>> >>>> * An id for the ClassLoader. This is the pointer to the Klass of the ClassLoader. The reason for using the Klass* (instead of the oop) is that it is stable across invocations. >>>> * The id of the ClassLoader?s parent ClassLoader. >>>> * The pointer to the ClassLoaderData structure in the VM. This can be useful for debugging. >>>> * The number of classes loaded by the ClassLoader. >>>> * The total size of all allocated metaspace chunks for the ClassLoader. >>>> * The total size of all allocated metaspace blocks for the ClassLoader. >>>> >>>> If there are anonymous classes (invokedynamic classes) attributed to the ClassLoader, the following additional information is listed: >>>> * The number of anonymous classes loaded by the ClassLoader. >>>> * The total size of all allocated metaspace chunks for anonymous classes in the ClassLoader. >>>> * The total size of all allocated metaspace blocks for anonymous classes in the ClassLoader. >>>> >>>> The information is gathered during a safe point to guarantee that the data structures are consistent. >>>> >>>> I have added a small test and have run this through jprt. A CCC request has been filed. >>>> >>>> webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8044107 >>>> >>>> Example output: >>>> >>>> ClassLoader Parent CLData* Classes ChunkSz BlockSz Type >>>> 0x00000007c002d908 0x0000000000000000 0x0000000000000000 0 0 0 sun.misc.Launcher$ExtClassLoader >>>> 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 4694016 4241312 >>>> 37 75776 50928 + invokedynamic classes >>> You shouldn?t name these ?invokedynamic classes?; they can be used for different purposes. ?anonymous classes? is the correct name. >> Hmm, yeah. Not a good name, I know. The problem with ?anonymous classes? is that for a Java developer, that term is already taken and means something completely different. This is very unfortunate and creates a lot of confusion. My first version called them ?lambdas?, but that was even more wrong. Better names are welcome! > These classes are defined via: > > public native Class defineAnonymousClass(Class hostClass, byte[] data, Object[] cpPatches); > > so this is what they are called. Sorry :-) It's still a bad name. I call them jsr292-anonymous classes. maybe "unsafe anonymous classes" ? Coleen > >> /Staffan >> >>>> 0x00000007c0061028 0x00000007c0036878 0x00007fb239c2de60 1 6144 1976 ClassLoaderStatsTest$DummyClassLoader >>>> 1 2048 1288 + invokedynamic classes >>>> 0x00000007c0036878 0x00000007c002d908 0x00007fb239e10fc0 8 88064 31544 sun.misc.Launcher$AppClassLoader >>>> Total = 4 808 4866048 4327048 >>>> ChunkSz: Total size of all allocated metaspace chunks >>>> BlockSz: Total size of all allocated metaspace blocks (each chunk has several blocks) >>>> >>>> >>>> Thanks, >>>> /Staffan From forax at univ-mlv.fr Wed May 28 20:26:40 2014 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 28 May 2014 22:26:40 +0200 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: <53863E5B.6050205@oracle.com> References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> <7390BCB6-5572-481F-B984-A2AABB76280B@oracle.com> <53863E5B.6050205@oracle.com> Message-ID: <53864680.106@univ-mlv.fr> On 05/28/2014 09:51 PM, Coleen Phillimore wrote: > > On 5/28/14, 3:49 PM, Christian Thalinger wrote: >> On May 28, 2014, at 12:46 PM, Staffan Larsen >> wrote: >> >>> On 28 maj 2014, at 21:39, Christian Thalinger >>> wrote: >>> >>>> On May 28, 2014, at 7:50 AM, Staffan Larsen >>>> wrote: >>>> >>>>> This change adds a new Diagnostic Command to list all ClassLoaders >>>>> and some statistics for each classloader. The command is called >>>>> ?GC.classloader_stats? and the information listed is: >>>>> >>>>> * An id for the ClassLoader. This is the pointer to the Klass of >>>>> the ClassLoader. The reason for using the Klass* (instead of the >>>>> oop) is that it is stable across invocations. >>>>> * The id of the ClassLoader?s parent ClassLoader. >>>>> * The pointer to the ClassLoaderData structure in the VM. This can >>>>> be useful for debugging. >>>>> * The number of classes loaded by the ClassLoader. >>>>> * The total size of all allocated metaspace chunks for the >>>>> ClassLoader. >>>>> * The total size of all allocated metaspace blocks for the >>>>> ClassLoader. >>>>> >>>>> If there are anonymous classes (invokedynamic classes) attributed >>>>> to the ClassLoader, the following additional information is listed: >>>>> * The number of anonymous classes loaded by the ClassLoader. >>>>> * The total size of all allocated metaspace chunks for anonymous >>>>> classes in the ClassLoader. >>>>> * The total size of all allocated metaspace blocks for anonymous >>>>> classes in the ClassLoader. >>>>> >>>>> The information is gathered during a safe point to guarantee that >>>>> the data structures are consistent. >>>>> >>>>> I have added a small test and have run this through jprt. A CCC >>>>> request has been filed. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8044107 >>>>> >>>>> Example output: >>>>> >>>>> ClassLoader Parent CLData* Classes >>>>> ChunkSz BlockSz Type >>>>> 0x00000007c002d908 0x0000000000000000 0x0000000000000000 >>>>> 0 0 0 sun.misc.Launcher$ExtClassLoader >>>>> 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 >>>>> 761 4694016 4241312 >>>>> 37 75776 50928 + invokedynamic classes >>>> You shouldn?t name these ?invokedynamic classes?; they can be used >>>> for different purposes. ?anonymous classes? is the correct name. >>> Hmm, yeah. Not a good name, I know. The problem with ?anonymous >>> classes? is that for a Java developer, that term is already taken >>> and means something completely different. This is very unfortunate >>> and creates a lot of confusion. My first version called them >>> ?lambdas?, but that was even more wrong. Better names are welcome! >> These classes are defined via: >> >> public native Class defineAnonymousClass(Class hostClass, >> byte[] data, Object[] cpPatches); >> >> so this is what they are called. Sorry :-) > > It's still a bad name. I call them jsr292-anonymous classes. maybe > "unsafe anonymous classes" ? > > Coleen John Rose has coined the name 'anonymous klasses' which is not that bad. R?mi >> >>> /Staffan >>> >>>>> 0x00000007c0061028 0x00000007c0036878 0x00007fb239c2de60 1 >>>>> 6144 1976 ClassLoaderStatsTest$DummyClassLoader >>>>> 1 2048 1288 + invokedynamic classes >>>>> 0x00000007c0036878 0x00000007c002d908 0x00007fb239e10fc0 >>>>> 8 88064 31544 sun.misc.Launcher$AppClassLoader >>>>> Total = 4 808 4866048 4327048 >>>>> ChunkSz: Total size of all allocated metaspace chunks >>>>> BlockSz: Total size of all allocated metaspace blocks (each chunk >>>>> has several blocks) >>>>> >>>>> >>>>> Thanks, >>>>> /Staffan > From dean.long at oracle.com Wed May 28 23:15:43 2014 From: dean.long at oracle.com (Dean Long) Date: Wed, 28 May 2014 16:15:43 -0700 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <434ACD4F-CAD7-426D-9D91-224D7556D470@oracle.com> References: <53851037.8040801@oracle.com> <434ACD4F-CAD7-426D-9D91-224D7556D470@oracle.com> Message-ID: <53866E1F.8050104@oracle.com> NativeMovRegMemPatching might be dead code. I can't find where it's used. dl On 5/28/2014 12:48 PM, Christian Thalinger wrote: > This made me curious because I was expecting a cast too: > void NativeMovRegMemPatching::print() { > if (is_immediate()) { > ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %x]", instruction_address(), offset()); > ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %d]", p2i(instruction_address()), offset()); > Turns out that NativeMovConstRegPatching::data() returns an int: > > int NativeMovConstRegPatching::data() const { > #ifdef _LP64 > return data64(addr_at(sethi_offset), long_at(add_offset)); > #else > return data32(long_at(sethi_offset), long_at(add_offset)); > #endif > } > > but on 64-bit we truncate an intptr_t: > > static intptr_t data64( address pc, int arith_insn ) { > > That?s odd because in NativeMovRegMem we use intptr_t for offset exactly because of the non-immediate case. > > On May 27, 2014, at 3:22 PM, Mikael Vidstedt wrote: > >> Please review the below change which addresses a number of print format/argument mismatches when compiling linux/sparc using gcc. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 >> Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ >> >> Most of these are straightforward changes to INTPTR_FORMAT & adding p2i. A couple of changes worth bringing up: >> >> * nativeInst_sparc.cpp >> >> I changed the format of the offset in mov related prints (two occurrences) to %d, since it's really a signed decimal int. This will hopefully reduce confusion when the offset is negative, where previously it would have looked like a relatively large positive offset. >> >> * macroAssembler_sparc.cpp >> >> The new code is in line with other similar patterns in the same file, but the ugly cast should IMHO really be rewritten to cast through a union. RFE? >> >> Cheers, >> Mikael >> From mikael.vidstedt at oracle.com Thu May 29 01:03:30 2014 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 28 May 2014 18:03:30 -0700 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <53866E1F.8050104@oracle.com> References: <53851037.8040801@oracle.com> <434ACD4F-CAD7-426D-9D91-224D7556D470@oracle.com> <53866E1F.8050104@oracle.com> Message-ID: <53868762.9040400@oracle.com> I think you're right. The only use appears to be in the unit test code itself. I just tried #if 0'ing out everything related to NativeMovRegMemPatching and it builds just fine. I filed an RFE to cover the removal of the class: https://bugs.openjdk.java.net/browse/JDK-8044242 Cheers, Mikael On 2014-05-28 16:15, Dean Long wrote: > NativeMovRegMemPatching might be dead code. I can't find where it's > used. > > dl > > On 5/28/2014 12:48 PM, Christian Thalinger wrote: >> This made me curious because I was expecting a cast too: >> void NativeMovRegMemPatching::print() { >> if (is_immediate()) { >> ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %x]", >> instruction_address(), offset()); >> ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %d]", >> p2i(instruction_address()), offset()); >> Turns out that NativeMovConstRegPatching::data() returns an int: >> >> int NativeMovConstRegPatching::data() const { >> #ifdef _LP64 >> return data64(addr_at(sethi_offset), long_at(add_offset)); >> #else >> return data32(long_at(sethi_offset), long_at(add_offset)); >> #endif >> } >> >> but on 64-bit we truncate an intptr_t: >> >> static intptr_t data64( address pc, int arith_insn ) { >> >> That?s odd because in NativeMovRegMem we use intptr_t for offset >> exactly because of the non-immediate case. >> >> On May 27, 2014, at 3:22 PM, Mikael Vidstedt >> wrote: >> >>> Please review the below change which addresses a number of print >>> format/argument mismatches when compiling linux/sparc using gcc. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 >>> Webrev: >>> http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ >>> >>> Most of these are straightforward changes to INTPTR_FORMAT & adding >>> p2i. A couple of changes worth bringing up: >>> >>> * nativeInst_sparc.cpp >>> >>> I changed the format of the offset in mov related prints (two >>> occurrences) to %d, since it's really a signed decimal int. This >>> will hopefully reduce confusion when the offset is negative, where >>> previously it would have looked like a relatively large positive >>> offset. >>> >>> * macroAssembler_sparc.cpp >>> >>> The new code is in line with other similar patterns in the same >>> file, but the ugly cast should IMHO really be rewritten to cast >>> through a union. RFE? >>> >>> Cheers, >>> Mikael >>> > From erik.helin at oracle.com Thu May 29 12:43:52 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 29 May 2014 14:43:52 +0200 Subject: 8042933: assert(capacity_until_gc >= committed_bytes) failed In-Reply-To: <5385CF00.2070605@oracle.com> References: <4471789.3Gp5iV3Z2a@ehelin-desktop> <5385CF00.2070605@oracle.com> Message-ID: <1502229.ArWomoX4nV@ehelin-laptop> Hi StefanK, thanks for reviewing! Please see new patches at: Incremental: http://cr.openjdk.java.net/~ehelin/8042933/webrev.00-01/ Full: http://cr.openjdk.java.net/~ehelin/8042933/webrev.01/ and comments inline. On Wednesday 28 May 2014 13:56:48 PM Stefan Karlsson wrote: > On 2014-05-28 13:10, Erik Helin wrote: > > Hi all, > > > > this is the second attemp to fix JDK-8034852 [0], but since I backed > > out the previous attempt, I have to use a new bug [1] (you can't use > > the same bug-id for two different committs). > > > > For a background to the original problem, please see the old email > > thread [2]. This patch is just and updated version of the original > > patch [3]. The issue with the original path is that > > MetaspaceGC::allowed_expansion can, during special circumstances, > > allow the Metaspace high-water mark (HWM aka > > MetaspaceGC::_capacity_until_GC) to become larger than the amount of > > committed memory. Keeping the HWM less than or equal to how much we > > have committed is an invariant we would like to be true in Metaspace > > at all times to make it easier to reason about the sizing heuristics. > > > > This patch changes MetaspaceGC::allowed_expansion to always enforce > > the invariant that HWM <= committed_bytes. However, we still want to > > allow the VM to allocate as much metadata it wants during > > initialization. To achieve this, we set the HWM to MaxMetaspaceSize > > when initializing Metaspace and then reset the HWM to > > max(MetaspaceSize, HWM) once the initialization is done. Thanks > > StefanK for coming up with this approach! > > > > MetaspaceGC::allowed_expansion previously also allowed the VM to > > allocate as much metadata it wants when the GC lock is active and we > > are awaiting a GC. The new behaviour will be to try to expand the HWM > > and then allocate (the expand_and_allocate call in > > CollectorPolicy::satisfy_failed_metadata_allocation). Under very > > special circumstances this might cause a thread to now await the GC in > > GC_locker::stall_until_clear instead of getting its metadata and > > continue executing. > > > > Webrev: > > http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/ > > http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/src/share/vm/memory/met > aspace.hpp.udiff.html > > Could you add a comment explaining why we use MaxMetaspaceSize here, > instead of the MetaspaceSize: Sure, done. > - static void initialize() { _capacity_until_GC = MetaspaceSize; } > + static void initialize() { _capacity_until_GC = MaxMetaspaceSize; } > + static void post_initialize(); > > Maybe it would be better to move the implementation of initialize() to > metaspace.cpp and place it together with the post_initialize() function? > I think that would make it more apparent that we just > setcapacity_until_GC temporarily to MaxMetaspaceSize. Agree, fixed. > http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/src/share/vm/runtime/th > read.cpp.udiff.html > > Could add a blank line between these two functions, so that they are not > grouped together? Yep, updated. Thanks, Erik > // Set flag that basic initialization has completed. Used by exceptions > and various // debug stuff, that does not work until all basic classes have > been initialized. set_init_completed(); > + Metaspace::post_initialize(); > > Otherwise, this looks good. > > thanks, > StefanK > > > Testing: > > - JPRT > > > > - Ad-hoc testing (on all platforms): > > - Kitchensink > > - runThese > > - Parallel Class Loading testlist > > - Metaspace testlist > > - GC nightly testlist > > - OOM testlist > > - Quick testlist > > - JTREG tests in both VM and JDK > > > > Thanks, > > Erik > > > > [0]: https://bugs.openjdk.java.net/browse/JDK-8034852 > > [1]: https://bugs.openjdk.java.net/browse/JDK-8042933 > > [2]: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013716.html > > [3]: http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ From daniel.daugherty at oracle.com Thu May 29 14:38:05 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 29 May 2014 08:38:05 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537FD4EF.2030406@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <537FD4EF.2030406@oracle.com> Message-ID: <5387464D.3070805@oracle.com> Serguei, Finally getting back to this thread. New webrev will be out shortly. Thanks for the careful review of the test. Dan On 5/23/14 5:08 PM, serguei.spitsyn at oracle.com wrote: > Hi Dan, > > The fix itself looks good. > > The test deserves a refactoring as the main method is too big. > Would it be possible to convert some of the blocks of the while loop > into methods? > Even the try block (variant: the while loop) is a good candidate to be > a separate method. > Also the definitions of the patterns can be moved out of the main and > made static final fields. > > Thanks, > Serguei > > > On 5/22/14 9:18 PM, Daniel D. Daugherty wrote: >> Zhengyu is tied up with some other work so I've taken on this fix. >> >> Here's the webrev URL for the next round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >> >> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >> The new test has been run with the fix via JPRT and passes on all >> JPRT platforms. The new test has also been run without the fix and >> fails on most platforms. Since the default sample size is just 30, >> it is possible to get 30 runs in a row without failing. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack >>> trace dump reporting incorrect lock state. The solution is to check >>> the monitor's owner, if the owner is other than the current thread, >>> then the monitor, is or is in process of being, set the pending >>> monitor of current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >> > From daniel.daugherty at oracle.com Thu May 29 14:44:44 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 29 May 2014 08:44:44 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537FE1F2.3050408@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <537EFB4A.1000507@oracle.com> <537F538B.2030109@oracle.com> <537FE1F2.3050408@oracle.com> Message-ID: <538747DC.8040006@oracle.com> David, Finally getting back to this thread. New webrev will be out shortly. Thanks for the careful review of the test. Buried in the refactoring of the test, I used the testlibrary code to find jstack. I looked at other parts of the testlibrary code, but I don't think they'll work for what I'm trying to do here. I've added comments to the test to explain my rationale. I'm sticking with using the 'test JDK' version of jstack since that program is often closely coupled with the VM itself. This means that the test is still flagged as 'jdk only'. I understand that in the embedded world, the 'test JRE' is a subset of 'compile JDK', but we don't know if that's true in all non-embedded cases. Dan On 5/23/14 6:04 PM, David Holmes wrote: > On 23/05/2014 11:56 PM, Daniel D. Daugherty wrote: >> Thanks for the quick review! >> >> >> On 5/23/14 1:39 AM, David Holmes wrote: >>> Hi Dan, >>> >>> Functional changes look good. >> >> Thanks. >> >> >>> For the test I thought the preferred approach these days was to use >>> the testlibrary utilities to launch the test in another VM and also >>> launch the tool. That way the test isn't restricted to needs_jdk. >> >> Don't really know the right answer here. I usually write tests >> like this as a Java program with a shell script wrapper, but I >> know we're trying not to do that anymore. >> >> When I grep'ed for jstack in the hotspot tests, I ran across one: >> >> $ rgrep jstack test >> test/runtime/7194254/Test7194254.java: * whether jstack reports >> correct priorities for them. >> test/runtime/7194254/Test7194254.java: String jstack = >> System.getProperty("java.home") + "/../bin/jstack"; >> test/runtime/7194254/Test7194254.java: Process process = new >> ProcessBuilder(jstack, pid) >> >> It turns out that your test for 7194254 was pretty much perfect >> for me to adapt for what I was trying to test. For now, I'm >> inclined to go with the current test if you're OK with that. > > That was then, this is now :) testlibrary gives you everything you > need without having to resort to java.home hacks. And the added > benefit is that the test is not restricted to being a JDK-only test. > > Your call - you are the one that has to do the work. > > Cheers, > David > > >> Dan >> >> >>> >>> Cheers, >>> David >>> >>> On 23/05/2014 2:18 PM, Daniel D. Daugherty wrote: >>>> Zhengyu is tied up with some other work so I've taken on this fix. >>>> >>>> Here's the webrev URL for the next round: >>>> >>>> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >>>> >>>> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >>>> The new test has been run with the fix via JPRT and passes on all >>>> JPRT platforms. The new test has also been run without the fix and >>>> fails on most platforms. Since the default sample size is just 30, >>>> it is possible to get 30 runs in a row without failing. >>>> >>>> Thanks, in advance, for any comments, questions or suggestions. >>>> >>>> Dan >>>> >>>> >>>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>>> This is a simple fix for incorrect lock state. >>>>> >>>>> The timing on setting thread's pending monitor can result stack trace >>>>> dump reporting incorrect lock state. The solution is to check the >>>>> monitor's owner, if the owner is other than the current thread, then >>>>> the monitor, is or is in process of being, set the pending monitor of >>>>> current thread. >>>>> >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>>> >>>> >> From daniel.daugherty at oracle.com Thu May 29 14:49:44 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 29 May 2014 08:49:44 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <537ECC1A.1030409@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> Message-ID: <53874908.5070805@oracle.com> One more round of review after refactoring the test based on comments from David H and Serguei. Here's the webrev for this round: http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ Had to change the default sample size from 30 -> 15 in order to get the test to pass reliably on Solaris SPARC JPRT machines. Thanks, in advance, for any comments, questions or suggestions. Dan On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: > Zhengyu is tied up with some other work so I've taken on this fix. > > Here's the webrev URL for the next round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ > > The fix has been tested with vm.quick on all Aurora Adhoc platforms. > The new test has been run with the fix via JPRT and passes on all > JPRT platforms. The new test has also been run without the fix and > fails on most platforms. Since the default sample size is just 30, > it is possible to get 30 runs in a row without failing. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/19/14 7:58 AM, Zhengyu Gu wrote: >> This is a simple fix for incorrect lock state. >> >> The timing on setting thread's pending monitor can result stack trace >> dump reporting incorrect lock state. The solution is to check the >> monitor's owner, if the owner is other than the current thread, then >> the monitor, is or is in process of being, set the pending monitor of >> current thread. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >> >> >> Thanks, >> >> -Zhengyu >> > > > From george.triantafillou at oracle.com Thu May 29 17:19:35 2014 From: george.triantafillou at oracle.com (George Triantafillou) Date: Thu, 29 May 2014 13:19:35 -0400 Subject: RFR (L) 8044140: [TESTBUG] Create NMT (Native Memory Tracking) tests for NMT2 In-Reply-To: <53861C20.60303@oracle.com> References: <53861884.1050407@oracle.com> <53861C20.60303@oracle.com> Message-ID: <53876C27.1070301@oracle.com> Hi Zhengyu, Thanks for your comments. On 5/28/2014 1:25 PM, Zhengyu Gu wrote: > Hi, > > JcmdScale.java: > There is no change, should not update copyright ... Fixed. > > AutoshutdownNMT.java line #42 > > 42 output.shouldContain("Unrecognized VM option > 'AutoshutdownNMT'"); > > Should be > > 42 output.shouldContain("Unrecognized VM option > 'AutoShutdownNMT'"); > > > Also -XX:-AutoshutdownNMT => -XX:-AutoShutdownNMT > Fixed. -George > > -Zhengyu > > > On 5/28/2014 1:10 PM, George Triantafillou wrote: >> Please review these new and modified tests for the redesigned Native >> Memory Tracking (NMT) functionality (8028541: Native Memory Tracking >> enhancement ). NMT >> was redesigned internally to address scalability issues in the first >> implementation. Existing functionality has not changed, with the >> exception of the removal of the "-AutoshutdownNMT" flag. >> >> Please note that the following new stress tests are excluded with the >> JTREG @ignore tag: >> >> test/runtime/NMT/MallocSiteHashOverflow.java >> test/runtime/NMT/MallocStressTest.java >> >> These tests can be run by using the jtreg override option >> "-ignore:run". The new key "@key stress" was also added to these tests. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8044140 >> Webrev: >> http://cr.openjdk.java.net/~gtriantafill/bug_8044140/webrev.00/ >> >> Testing: JTREG to exercise the added tests >> >> Thank you, >> >> George > From serguei.spitsyn at oracle.com Thu May 29 18:33:31 2014 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 29 May 2014 11:33:31 -0700 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <53874908.5070805@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> Message-ID: <53877D7B.8010506@oracle.com> Dan, It looks good. Thank you for the test refactoring! Thanks, Serguei On 5/29/14 7:49 AM, Daniel D. Daugherty wrote: > One more round of review after refactoring the test based on comments > from David H and Serguei. > > Here's the webrev for this round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ > > Had to change the default sample size from 30 -> 15 in order to > get the test to pass reliably on Solaris SPARC JPRT machines. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >> Zhengyu is tied up with some other work so I've taken on this fix. >> >> Here's the webrev URL for the next round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >> >> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >> The new test has been run with the fix via JPRT and passes on all >> JPRT platforms. The new test has also been run without the fix and >> fails on most platforms. Since the default sample size is just 30, >> it is possible to get 30 runs in a row without failing. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack >>> trace dump reporting incorrect lock state. The solution is to check >>> the monitor's owner, if the owner is other than the current thread, >>> then the monitor, is or is in process of being, set the pending >>> monitor of current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >> >> >> > From lev.priima at oracle.com Thu May 29 18:33:52 2014 From: lev.priima at oracle.com (Lev Priima) Date: Thu, 29 May 2014 22:33:52 +0400 Subject: [9] RFR (XS) 8044339: wrong comment for FilterSpuriousWakeups option Message-ID: <53877D90.7090105@oracle.com> Please review and help me with integration: diff -r 102506d9d873 src/share/vm/runtime/globals.hpp --- a/src/share/vm/runtime/globals.hpp Wed May 28 14:42:00 2014 +0400 +++ b/src/share/vm/runtime/globals.hpp Thu May 29 22:29:37 2014 +0400 @@ -1144,7 +1144,7 @@ \ product(bool, FilterSpuriousWakeups, true, \ "Prevent spurious or premature wakeups from object.wait " \ - "(Solaris only)") \ + "(Ignored for Windows)") \ \ product(intx, NativeMonitorTimeout, -1, "(Unstable)") \ \ bug: https://bugs.openjdk.java.net/browse/JDK-8044339 -- Lev From igor.ignatyev at oracle.com Thu May 29 18:44:12 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 29 May 2014 22:44:12 +0400 Subject: [9] RFR (XS) 8044339: wrong comment for FilterSpuriousWakeups option In-Reply-To: <53877D90.7090105@oracle.com> References: <53877D90.7090105@oracle.com> Message-ID: <53877FFC.2080001@oracle.com> Lev, I've uploaded your webrev -- http://cr.openjdk.java.net/~iignatyev/lpriima/8044339/webrev.00/ there's other flags which have 'Solaris only', have you looked at them? > $ cat ./src/share/vm/runtime/globals.hpp | grep -B2 "Solaris only" > \ > product_pd(bool, DontYieldALot, \ > "Throw away obvious excess yield calls (for Solaris only)") \ > -- > product(bool, ConvertYieldToSleep, false, \ > "Convert yield to a sleep of MinSleepInterval to simulate Win32 " \ > "behavior (Solaris only)") \ > \ > product(bool, UseBoundThreads, true, \ > "Bind user level threads to kernel threads (for Solaris only)") \ > -- > develop(bool, UseDetachedThreads, true, \ > "Use detached threads that are recycled upon termination " \ > "(for Solaris only)") \ > -- > product(bool, UseAltSigs, false, \ > "Use alternate signals instead of SIGUSR1 & SIGUSR2 for VM " \ > "internal signals (Solaris only)") \ > -- > \ > product(bool, CompilerThreadHintNoPreempt, true, \ > "(Solaris only) Give compiler threads an extra quanta") \ > \ > product(bool, VMThreadHintNoPreempt, false, \ > "(Solaris only) Give VM thread an extra quanta") \ Igor On 05/29/2014 10:33 PM, Lev Priima wrote: > Please review and help me with integration: > > diff -r 102506d9d873 src/share/vm/runtime/globals.hpp > --- a/src/share/vm/runtime/globals.hpp Wed May 28 14:42:00 2014 +0400 > +++ b/src/share/vm/runtime/globals.hpp Thu May 29 22:29:37 2014 +0400 > @@ -1144,7 +1144,7 @@ > \ > product(bool, FilterSpuriousWakeups, > true, \ > "Prevent spurious or premature wakeups from object.wait > " \ > - "(Solaris > only)") \ > + "(Ignored for > Windows)") \ > \ > product(intx, NativeMonitorTimeout, -1, > "(Unstable)") \ > \ > > bug: https://bugs.openjdk.java.net/browse/JDK-8044339 > From igor.veresov at oracle.com Thu May 29 19:25:56 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Thu, 29 May 2014 09:25:56 -1000 Subject: [8u20] RFR(XS): 8044090: C1: Old value instead of new one is passed to post-barrier in UnsafeGetAndSetObject Message-ID: <178E522A-6A50-4298-A2AB-012FAFE08654@oracle.com> I?d like to backport this small fix, since the problem may manifest itself with nasty and hard to diagnose symptoms. The patch applies cleanly, nightlies seem clean as well. Webrev: http://cr.openjdk.java.net/~iveresov/8044090/webrev.00 jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/dd46a4a1299c JBS: https://bugs.openjdk.java.net/browse/JDK-8044090 Thanks! igor From daniel.daugherty at oracle.com Thu May 29 20:40:33 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 29 May 2014 14:40:33 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <53877D7B.8010506@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> <53877D7B.8010506@oracle.com> Message-ID: <53879B41.3080700@oracle.com> Thanks for the re-review! Dan On 5/29/14 12:33 PM, serguei.spitsyn at oracle.com wrote: > Dan, > > It looks good. > Thank you for the test refactoring! > > Thanks, > Serguei > > On 5/29/14 7:49 AM, Daniel D. Daugherty wrote: >> One more round of review after refactoring the test based on comments >> from David H and Serguei. >> >> Here's the webrev for this round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ >> >> Had to change the default sample size from 30 -> 15 in order to >> get the test to pass reliably on Solaris SPARC JPRT machines. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >>> Zhengyu is tied up with some other work so I've taken on this fix. >>> >>> Here's the webrev URL for the next round: >>> >>> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >>> >>> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >>> The new test has been run with the fix via JPRT and passes on all >>> JPRT platforms. The new test has also been run without the fix and >>> fails on most platforms. Since the default sample size is just 30, >>> it is possible to get 30 runs in a row without failing. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >>> >>> >>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>> This is a simple fix for incorrect lock state. >>>> >>>> The timing on setting thread's pending monitor can result stack >>>> trace dump reporting incorrect lock state. The solution is to check >>>> the monitor's owner, if the owner is other than the current thread, >>>> then the monitor, is or is in process of being, set the pending >>>> monitor of current thread. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>> >>> >>> >> > From jon.masamitsu at oracle.com Thu May 29 21:51:40 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 29 May 2014 14:51:40 -0700 Subject: 8042933: assert(capacity_until_gc >= committed_bytes) failed In-Reply-To: <4471789.3Gp5iV3Z2a@ehelin-desktop> References: <4471789.3Gp5iV3Z2a@ehelin-desktop> Message-ID: <5387ABEC.90806@oracle.com> Erik, Is it correct that this fix allows us to allocate more space than MaxMetaspaceSize for metadata? And if, yes, why do that instead of fail initialization? I can see that it might be a mess to actually fail during start up but maybe fail in the post_initialize(). Jon On 5/28/14 4:10 AM, Erik Helin wrote: > Hi all, > > this is the second attemp to fix JDK-8034852 [0], but since I backed > out the previous attempt, I have to use a new bug [1] (you can't use > the same bug-id for two different committs). > > For a background to the original problem, please see the old email > thread [2]. This patch is just and updated version of the original > patch [3]. The issue with the original path is that > MetaspaceGC::allowed_expansion can, during special circumstances, > allow the Metaspace high-water mark (HWM aka > MetaspaceGC::_capacity_until_GC) to become larger than the amount of > committed memory. Keeping the HWM less than or equal to how much we > have committed is an invariant we would like to be true in Metaspace > at all times to make it easier to reason about the sizing heuristics. > > This patch changes MetaspaceGC::allowed_expansion to always enforce > the invariant that HWM <= committed_bytes. However, we still want to > allow the VM to allocate as much metadata it wants during > initialization. To achieve this, we set the HWM to MaxMetaspaceSize > when initializing Metaspace and then reset the HWM to > max(MetaspaceSize, HWM) once the initialization is done. Thanks > StefanK for coming up with this approach! > > MetaspaceGC::allowed_expansion previously also allowed the VM to > allocate as much metadata it wants when the GC lock is active and we > are awaiting a GC. The new behaviour will be to try to expand the HWM > and then allocate (the expand_and_allocate call in > CollectorPolicy::satisfy_failed_metadata_allocation). Under very > special circumstances this might cause a thread to now await the GC in > GC_locker::stall_until_clear instead of getting its metadata and > continue executing. > > Webrev: > http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/ > > Testing: > - JPRT > - Ad-hoc testing (on all platforms): > - Kitchensink > - runThese > - Parallel Class Loading testlist > - Metaspace testlist > - GC nightly testlist > - OOM testlist > - Quick testlist > - JTREG tests in both VM and JDK > > Thanks, > Erik > > [0]: https://bugs.openjdk.java.net/browse/JDK-8034852 > [1]: https://bugs.openjdk.java.net/browse/JDK-8042933 > [2]: http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013716.html > [3]: http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ > From vladimir.kozlov at oracle.com Thu May 29 22:13:54 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 29 May 2014 15:13:54 -0700 Subject: [8u20] RFR(XS): 8044090: C1: Old value instead of new one is passed to post-barrier in UnsafeGetAndSetObject In-Reply-To: <178E522A-6A50-4298-A2AB-012FAFE08654@oracle.com> References: <178E522A-6A50-4298-A2AB-012FAFE08654@oracle.com> Message-ID: <5387B122.6010302@oracle.com> Good. Vladimir On 5/29/14 12:25 PM, Igor Veresov wrote: > I?d like to backport this small fix, since the problem may manifest itself with nasty and hard to diagnose symptoms. > The patch applies cleanly, nightlies seem clean as well. > > Webrev: http://cr.openjdk.java.net/~iveresov/8044090/webrev.00 > jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/dd46a4a1299c > JBS: https://bugs.openjdk.java.net/browse/JDK-8044090 > > Thanks! > igor > From vladimir.kozlov at oracle.com Thu May 29 22:24:14 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 29 May 2014 15:24:14 -0700 Subject: [8u20] RFR (XS) 8043274: Test compiler/7184394/TestAESMain.java gets NPE on solaris Message-ID: <5387B38E.4020602@oracle.com> 8u20 backport request. The fix was pushed into jdk9 week ago and nightly testing shows no problems. Changes from jdk9 applied to 8u without conflicts. http://cr.openjdk.java.net/~kvn/8043274/webrev/ https://bugs.openjdk.java.net/browse/JDK-8043274 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/3ba6e0092f65 Thanks, Vladimir From igor.veresov at oracle.com Thu May 29 22:34:20 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Thu, 29 May 2014 12:34:20 -1000 Subject: [8u20] RFR (XS) 8043274: Test compiler/7184394/TestAESMain.java gets NPE on solaris In-Reply-To: <5387B38E.4020602@oracle.com> References: <5387B38E.4020602@oracle.com> Message-ID: <0342142C-DBA3-47BD-9F0B-7002FB3A1C0F@oracle.com> Looks fine. igor On May 29, 2014, at 12:24 PM, Vladimir Kozlov wrote: > 8u20 backport request. The fix was pushed into jdk9 week ago and nightly testing shows no problems. > > Changes from jdk9 applied to 8u without conflicts. > > http://cr.openjdk.java.net/~kvn/8043274/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8043274 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/3ba6e0092f65 > > Thanks, > Vladimir From igor.veresov at oracle.com Thu May 29 22:34:30 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Thu, 29 May 2014 12:34:30 -1000 Subject: [8u20] RFR(XS): 8044090: C1: Old value instead of new one is passed to post-barrier in UnsafeGetAndSetObject In-Reply-To: <5387B122.6010302@oracle.com> References: <178E522A-6A50-4298-A2AB-012FAFE08654@oracle.com> <5387B122.6010302@oracle.com> Message-ID: <449E0026-98EB-422F-B552-4627903B649B@oracle.com> Thanks! igor On May 29, 2014, at 12:13 PM, Vladimir Kozlov wrote: > Good. > > Vladimir > > On 5/29/14 12:25 PM, Igor Veresov wrote: >> I?d like to backport this small fix, since the problem may manifest itself with nasty and hard to diagnose symptoms. >> The patch applies cleanly, nightlies seem clean as well. >> >> Webrev: http://cr.openjdk.java.net/~iveresov/8044090/webrev.00 >> jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/dd46a4a1299c >> JBS: https://bugs.openjdk.java.net/browse/JDK-8044090 >> >> Thanks! >> igor >> From vladimir.kozlov at oracle.com Thu May 29 22:38:58 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 29 May 2014 15:38:58 -0700 Subject: [8u20] RFR (XS) 8043274: Test compiler/7184394/TestAESMain.java gets NPE on solaris In-Reply-To: <0342142C-DBA3-47BD-9F0B-7002FB3A1C0F@oracle.com> References: <5387B38E.4020602@oracle.com> <0342142C-DBA3-47BD-9F0B-7002FB3A1C0F@oracle.com> Message-ID: <5387B702.5000202@oracle.com> Thank you, Igor Vladimir On 5/29/14 3:34 PM, Igor Veresov wrote: > Looks fine. > > igor > > On May 29, 2014, at 12:24 PM, Vladimir Kozlov wrote: > >> 8u20 backport request. The fix was pushed into jdk9 week ago and nightly testing shows no problems. >> >> Changes from jdk9 applied to 8u without conflicts. >> >> http://cr.openjdk.java.net/~kvn/8043274/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8043274 >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/3ba6e0092f65 >> >> Thanks, >> Vladimir > From vitalyd at gmail.com Fri May 30 01:20:13 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 29 May 2014 21:20:13 -0400 Subject: CodeHeap::expand_by malloc failed In-Reply-To: <5387D545.9080404@oracle.com> References: <5387D545.9080404@oracle.com> Message-ID: Yes, swap and overcommit are turned off. But, there's substantial free physical memory available and looks like 65kb available to expand the code heap contiguoisly. So I'm a bit puzzled ... Taking Chris' suggestion and cc'ing hotspot dev ... Sent from my phone On May 29, 2014 8:47 PM, "Vladimir Kozlov" wrote: > Could be because there is no swap on this machine: > > Memory: 4k page, physical 49521668k(5596796k free), swap 0k(0k free) > > Vladimir > > On 5/29/14 5:35 PM, Christian Thalinger wrote: > >> Although it?s the code cache I assume runtime folk would know more about >> this. Maybe send to hotspot-dev. >> >> On May 29, 2014, at 9:01 AM, Vitaly Davidovich > > wrote: >> >> Hi guys, >>> >>> Need a bit of help explaining a hotspot malloc failure crash on 7u51. >>> I'm going to paste the relevant snippets from the hs_err log. >>> >>> # >>> # There is insufficient memory for the Java Runtime Environment to >>> continue. >>> # Native memory allocation (malloc) failed to allocate 65536 bytes for >>> committing reserved memory. >>> # Possible reasons: >>> # The system is out of physical RAM or swap space >>> # In 32 bit mode, the process size limit was hit >>> # Possible solutions: >>> # Reduce memory load on the system >>> # Increase physical memory or swap space >>> # Check if swap backing store is full >>> # Use 64 bit Java on a 64 bit OS >>> # Decrease Java heap size (-Xmx/-Xms) >>> # Decrease number of Java threads >>> # Decrease Java thread stack sizes (-Xss) >>> # Set larger code cache with -XX:ReservedCodeCacheSize= >>> # This output file may be truncated or incomplete. >>> # >>> # Out of Memory Error (os_linux.cpp:2726), pid=10643, tid=47319048501520 >>> # >>> # JRE version: Java(TM) SE Runtime Environment (7.0_51-b13) (build >>> 1.7.0_51-b13) >>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.51-b03 mixed mode >>> linux-amd64 compressed oops) >>> >>> >>> --------------- T H R E A D --------------- >>> >>> Current thread (0x0000000000716800): JavaThread "C2 CompilerThread0" >>> daemon [_thread_in_vm, id=10689, >>> stack(0x00002b095303b000,0x00002b095313c000)] >>> >>> Stack: [0x00002b095303b000,0x00002b095313c000], >>> sp=0x00002b0953138d20, free space=1015k >>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>> C=native code) >>> V [libjvm.so+0x992f4a] VMError::report_and_die()+0x2ea >>> V [libjvm.so+0x4931ab] report_vm_out_of_memory(char const*, int, >>> unsigned long, char const*)+0x9b >>> V [libjvm.so+0x81338e] os::Linux::commit_memory_impl(char*, unsigned >>> long, bool)+0xfe >>> V [libjvm.so+0x81383f] os::Linux::commit_memory_impl(char*, unsigned >>> long, unsigned long, bool)+0x4f >>> V [libjvm.so+0x813a2c] os::pd_commit_memory(char*, unsigned long, >>> unsigned long, bool)+0xc >>> V [libjvm.so+0x80daea] os::commit_memory(char*, unsigned long, >>> unsigned long, bool)+0x2a >>> V [libjvm.so+0x98e849] VirtualSpace::expand_by(unsigned long, >>> bool)+0x1c9 >>> V [libjvm.so+0x58a62c] CodeHeap::expand_by(unsigned long)+0x8c >>> V [libjvm.so+0x42111d] CodeCache::allocate(int)+0x4d >>> V [libjvm.so+0x7e1a39] nmethod::new_nmethod(methodHandle, int, int, >>> CodeOffsets*, int, DebugInformationRecorder*, Dependencies*, >>> CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*, >>> ImplicitExceptionTable*, AbstractCompiler*, int)+0x179 >>> V [libjvm.so+0x3cfc54] ciEnv::register_method(ciMethod*, int, >>> CodeOffsets*, int, CodeBuffer*, int, OopMapSet*, >>> ExceptionHandlerTable*, ImplicitExceptionTable*, AbstractCompiler*, >>> int, bool, bool)+0x364 >>> V [libjvm.so+0x4458fb] Compile::Compile(ciEnv*, C2Compiler*, >>> ciMethod*, int, bool, bool)+0x11cb >>> V [libjvm.so+0x3afa76] C2Compiler::compile_method(ciEnv*, ciMethod*, >>> int)+0x176 >>> V [libjvm.so+0x44ba9e] >>> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x33e >>> V [libjvm.so+0x44c87d] CompileBroker::compiler_thread_loop()+0x43d >>> V [libjvm.so+0x94d5ff] JavaThread::thread_main_inner()+0xdf >>> V [libjvm.so+0x94d705] JavaThread::run()+0xf5 >>> V [libjvm.so+0x815538] java_start(Thread*)+0x108 >>> >>> Code Cache [0x00002b091755c000, 0x00002b091869c000, 0x00002b091a55c000) >>> total_blobs=7033 nmethods=6349 adapters=634 free_code_cache=31715Kb >>> largest_free_block=32247296 >>> >>> Memory mappings around the code cache virtual space (bolded line is >>> the code cache segment, I believe, based on the Code Cache output above): >>> >>> *2b091755c000-2b091869c000 rwxp 00000000 00:00 0* >>> 2b09186ac000-2b091d66f000 rw-p 00000000 00:00 0 >>> 2b091d66f000-2b091d670000 ---p 00000000 00:00 0 >>> 2b091d670000-2b091d770000 rwxp 00000000 00:00 0 >>> 2b091d770000-2b091d771000 ---p 00000000 00:00 0 >>> 2b091d771000-2b091d871000 rwxp 00000000 00:00 0 >>> 2b091d871000-2b091d872000 ---p 00000000 00:00 0 >>> 2b091d872000-2b091d972000 rwxp 00000000 00:00 0 >>> >>> /proc/meminfo: >>> MemTotal: 49521668 kB >>> MemFree: 5596796 kB >>> Buffers: 294684 kB >>> Cached: 34205856 kB >>> SwapCached: 0 kB >>> Active: 12745128 kB >>> Inactive: 28516788 kB >>> Active(anon): 9280636 kB >>> Inactive(anon): 3090264 kB >>> Active(file): 3464492 kB >>> Inactive(file): 25426524 kB >>> Unevictable: 14420 kB >>> Mlocked: 14420 kB >>> SwapTotal: 0 kB >>> SwapFree: 0 kB >>> Dirty: 20392 kB >>> Writeback: 0 kB >>> AnonPages: 6776540 kB >>> Mapped: 6292204 kB >>> Shmem: 5604620 kB >>> Slab: 1828656 kB >>> SReclaimable: 1567928 kB >>> SUnreclaim: 260728 kB >>> KernelStack: 4648 kB >>> PageTables: 58800 kB >>> NFS_Unstable: 0 kB >>> Bounce: 0 kB >>> WritebackTmp: 0 kB >>> CommitLimit: 47045584 kB >>> Committed_AS: 46046480 kB >>> VmallocTotal: 34359738367 kB >>> VmallocUsed: 460188 kB >>> VmallocChunk: 34333703480 kB >>> HugePages_Total: 0 >>> HugePages_Free: 0 >>> HugePages_Rsvd: 0 >>> HugePages_Surp: 0 >>> Hugepagesize: 2048 kB >>> DirectMap4k: 8192 kB >>> DirectMap2M: 2080768 kB >>> DirectMap1G: 48234496 kB >>> >>> Memory: 4k page, physical 49521668k(5596796k free), swap 0k(0k free) >>> >>> * overcommit is turned off >>> >>> So the code heap was attempting to expand by 65kb at a fixed address. >>> There appears to be a 65kb mapping available between the end of the >>> current code heap mapping (2b091755c000-2b091869c000) and the next one >>> (2b09186ac000-2b091d66f000). There's about 30mb of free space left >>> (reserved, but uncommitted I take it) in the virtual space, so it's a >>> bit puzzling given there's ample physical mem available. Only thing I >>> can think of is expansion fails because it cannot get contiguous >>> space, but then I can't reconcile that with the mem mapping above. >>> >>> Does anyone have any ideas? >>> >>> Let me know if you need additional info from the hs_err file. >>> >>> Thanks >>> >>> >> From rednaxelafx at gmail.com Fri May 30 01:39:02 2014 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 29 May 2014 18:39:02 -0700 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <53874908.5070805@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> Message-ID: Hi Dan, The new fix looks good to me (not a Reviewer). Thumbs up! Thanks, Kris On Thu, May 29, 2014 at 7:49 AM, Daniel D. Daugherty < daniel.daugherty at oracle.com> wrote: > One more round of review after refactoring the test based on comments > from David H and Serguei. > > Here's the webrev for this round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ > > Had to change the default sample size from 30 -> 15 in order to > get the test to pass reliably on Solaris SPARC JPRT machines. > > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: > >> Zhengyu is tied up with some other work so I've taken on this fix. >> >> Here's the webrev URL for the next round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >> >> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >> The new test has been run with the fix via JPRT and passes on all >> JPRT platforms. The new test has also been run without the fix and >> fails on most platforms. Since the default sample size is just 30, >> it is possible to get 30 runs in a row without failing. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >> >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack trace >>> dump reporting incorrect lock state. The solution is to check the monitor's >>> owner, if the owner is other than the current thread, then the monitor, is >>> or is in process of being, set the pending monitor of current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >>> >> >> >> > From jon.masamitsu at oracle.com Fri May 30 02:30:16 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 29 May 2014 19:30:16 -0700 Subject: 8042933: assert(capacity_until_gc >= committed_bytes) failed In-Reply-To: <5387ABEC.90806@oracle.com> References: <4471789.3Gp5iV3Z2a@ehelin-desktop> <5387ABEC.90806@oracle.com> Message-ID: <5387ED38.7050305@oracle.com> Erik, I think the answer to my question is no. Change looks good. Reviewed. Jon On 5/29/2014 2:51 PM, Jon Masamitsu wrote: > Erik, > > Is it correct that this fix allows us to allocate more space > than MaxMetaspaceSize for metadata? And if, yes, why > do that instead of fail initialization? I can see that it might > be a mess to actually fail during start up but maybe fail > in the post_initialize(). > > Jon > > On 5/28/14 4:10 AM, Erik Helin wrote: >> Hi all, >> >> this is the second attemp to fix JDK-8034852 [0], but since I backed >> out the previous attempt, I have to use a new bug [1] (you can't use >> the same bug-id for two different committs). >> >> For a background to the original problem, please see the old email >> thread [2]. This patch is just and updated version of the original >> patch [3]. The issue with the original path is that >> MetaspaceGC::allowed_expansion can, during special circumstances, >> allow the Metaspace high-water mark (HWM aka >> MetaspaceGC::_capacity_until_GC) to become larger than the amount of >> committed memory. Keeping the HWM less than or equal to how much we >> have committed is an invariant we would like to be true in Metaspace >> at all times to make it easier to reason about the sizing heuristics. >> >> This patch changes MetaspaceGC::allowed_expansion to always enforce >> the invariant that HWM <= committed_bytes. However, we still want to >> allow the VM to allocate as much metadata it wants during >> initialization. To achieve this, we set the HWM to MaxMetaspaceSize >> when initializing Metaspace and then reset the HWM to >> max(MetaspaceSize, HWM) once the initialization is done. Thanks >> StefanK for coming up with this approach! >> >> MetaspaceGC::allowed_expansion previously also allowed the VM to >> allocate as much metadata it wants when the GC lock is active and we >> are awaiting a GC. The new behaviour will be to try to expand the HWM >> and then allocate (the expand_and_allocate call in >> CollectorPolicy::satisfy_failed_metadata_allocation). Under very >> special circumstances this might cause a thread to now await the GC in >> GC_locker::stall_until_clear instead of getting its metadata and >> continue executing. >> >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8042933/webrev.00/ >> >> Testing: >> - JPRT >> - Ad-hoc testing (on all platforms): >> - Kitchensink >> - runThese >> - Parallel Class Loading testlist >> - Metaspace testlist >> - GC nightly testlist >> - OOM testlist >> - Quick testlist >> - JTREG tests in both VM and JDK >> >> Thanks, >> Erik >> >> [0]: https://bugs.openjdk.java.net/browse/JDK-8034852 >> [1]: https://bugs.openjdk.java.net/browse/JDK-8042933 >> [2]: >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-April/013716.html >> [3]: http://cr.openjdk.java.net/~ehelin/8034852/webrev.00/ >> > From christian.thalinger at oracle.com Fri May 30 03:06:09 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 29 May 2014 20:06:09 -0700 Subject: RFR (XS): 8044071: Print format/argument warnings In-Reply-To: <53868762.9040400@oracle.com> References: <53851037.8040801@oracle.com> <434ACD4F-CAD7-426D-9D91-224D7556D470@oracle.com> <53866E1F.8050104@oracle.com> <53868762.9040400@oracle.com> Message-ID: <880A3E31-C4E0-46A4-86F4-B9123F093171@oracle.com> On May 28, 2014, at 6:03 PM, Mikael Vidstedt wrote: > > I think you're right. The only use appears to be in the unit test code itself. I just tried #if 0'ing out everything related to NativeMovRegMemPatching and it builds just fine. > > I filed an RFE to cover the removal of the class: https://bugs.openjdk.java.net/browse/JDK-8044242 Thanks. > > Cheers, > Mikael > > On 2014-05-28 16:15, Dean Long wrote: >> NativeMovRegMemPatching might be dead code. I can't find where it's used. >> >> dl >> >> On 5/28/2014 12:48 PM, Christian Thalinger wrote: >>> This made me curious because I was expecting a cast too: >>> void NativeMovRegMemPatching::print() { >>> if (is_immediate()) { >>> ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %x]", instruction_address(), offset()); >>> ! tty->print_cr(INTPTR_FORMAT ": mov reg, [reg + %d]", p2i(instruction_address()), offset()); >>> Turns out that NativeMovConstRegPatching::data() returns an int: >>> >>> int NativeMovConstRegPatching::data() const { >>> #ifdef _LP64 >>> return data64(addr_at(sethi_offset), long_at(add_offset)); >>> #else >>> return data32(long_at(sethi_offset), long_at(add_offset)); >>> #endif >>> } >>> >>> but on 64-bit we truncate an intptr_t: >>> >>> static intptr_t data64( address pc, int arith_insn ) { >>> >>> That?s odd because in NativeMovRegMem we use intptr_t for offset exactly because of the non-immediate case. >>> >>> On May 27, 2014, at 3:22 PM, Mikael Vidstedt wrote: >>> >>>> Please review the below change which addresses a number of print format/argument mismatches when compiling linux/sparc using gcc. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8044071 >>>> Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8044071/webrev.00/webrev/ >>>> >>>> Most of these are straightforward changes to INTPTR_FORMAT & adding p2i. A couple of changes worth bringing up: >>>> >>>> * nativeInst_sparc.cpp >>>> >>>> I changed the format of the offset in mov related prints (two occurrences) to %d, since it's really a signed decimal int. This will hopefully reduce confusion when the offset is negative, where previously it would have looked like a relatively large positive offset. >>>> >>>> * macroAssembler_sparc.cpp >>>> >>>> The new code is in line with other similar patterns in the same file, but the ugly cast should IMHO really be rewritten to cast through a union. RFE? >>>> >>>> Cheers, >>>> Mikael >>>> >> > From daniel.daugherty at oracle.com Fri May 30 03:52:29 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 29 May 2014 21:52:29 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> Message-ID: <5388007D.9070803@oracle.com> Thanks for the review! BTW, I'm listing you, Zhengyu and me on the 'Contributed-by' line since the three of us had a hand in various versions and parts of this fix. :-) Dan On 5/29/14 7:39 PM, Krystal Mok wrote: > Hi Dan, > > The new fix looks good to me (not a Reviewer). Thumbs up! > > Thanks, > Kris > > > On Thu, May 29, 2014 at 7:49 AM, Daniel D. Daugherty > > wrote: > > One more round of review after refactoring the test based on comments > from David H and Serguei. > > Here's the webrev for this round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ > > > Had to change the default sample size from 30 -> 15 in order to > get the test to pass reliably on Solaris SPARC JPRT machines. > > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: > > Zhengyu is tied up with some other work so I've taken on this fix. > > Here's the webrev URL for the next round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ > > The fix has been tested with vm.quick on all Aurora Adhoc > platforms. > The new test has been run with the fix via JPRT and passes on all > JPRT platforms. The new test has also been run without the fix and > fails on most platforms. Since the default sample size is just 30, > it is possible to get 30 runs in a row without failing. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/19/14 7:58 AM, Zhengyu Gu wrote: > > This is a simple fix for incorrect lock state. > > The timing on setting thread's pending monitor can result > stack trace dump reporting incorrect lock state. The > solution is to check the monitor's owner, if the owner is > other than the current thread, then the monitor, is or is > in process of being, set the pending monitor of current > thread. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 > Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ > > > > Thanks, > > -Zhengyu > > > > > > From david.holmes at oracle.com Fri May 30 05:57:20 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 30 May 2014 15:57:20 +1000 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <53874908.5070805@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> Message-ID: <53881DC0.8090009@oracle.com> Thanks Dan! Good to go. David On 30/05/2014 12:49 AM, Daniel D. Daugherty wrote: > One more round of review after refactoring the test based on comments > from David H and Serguei. > > Here's the webrev for this round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ > > Had to change the default sample size from 30 -> 15 in order to > get the test to pass reliably on Solaris SPARC JPRT machines. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >> Zhengyu is tied up with some other work so I've taken on this fix. >> >> Here's the webrev URL for the next round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >> >> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >> The new test has been run with the fix via JPRT and passes on all >> JPRT platforms. The new test has also been run without the fix and >> fails on most platforms. Since the default sample size is just 30, >> it is possible to get 30 runs in a row without failing. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack trace >>> dump reporting incorrect lock state. The solution is to check the >>> monitor's owner, if the owner is other than the current thread, then >>> the monitor, is or is in process of being, set the pending monitor of >>> current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >> >> >> > From david.holmes at oracle.com Fri May 30 06:03:07 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 30 May 2014 16:03:07 +1000 Subject: [9] RFR (XS) 8044339: wrong comment for FilterSpuriousWakeups option In-Reply-To: <53877D90.7090105@oracle.com> References: <53877D90.7090105@oracle.com> Message-ID: <53881F1B.4090805@oracle.com> Hi Lev, On 30/05/2014 4:33 AM, Lev Priima wrote: > Please review and help me with integration: > > diff -r 102506d9d873 src/share/vm/runtime/globals.hpp > --- a/src/share/vm/runtime/globals.hpp Wed May 28 14:42:00 2014 +0400 > +++ b/src/share/vm/runtime/globals.hpp Thu May 29 22:29:37 2014 +0400 > @@ -1144,7 +1144,7 @@ > \ > product(bool, FilterSpuriousWakeups, > true, \ > "Prevent spurious or premature wakeups from object.wait > " \ > - "(Solaris > only)") \ > + "(Ignored for > Windows)") \ > \ Sorry to make life complicated but as I put in the bug report: The "Solaris" part dates from the days when Solaris meant "not Windows". :) A bigger problem here is that the main part of the description: "Prevent spurious or premature wakeups from object.wait " may not match what people expect - it depends on whether "filter" means to remove the item being filtered, or to keep only the item being filtered. It would clearer if it read "When true prevents ..." but even then it is not accurate as this only excludes spurious wakeups from the OS pthread_cond[timed]wait implementation - the JVM can still induce its own "spurious wakeups". So the better description would be: "When true prevents OS-level spurious, or premature, wakeups from Object.wait" But it should also be noted that where this flag is checked there is an even more inaccurate comment: // The underlying Solaris implementation, cond_timedwait, admits // spurious/premature wakeups, but the JLS/JVM spec prevents the // JVM from making those visible to Java code. Ironically this comment appears in the linux and bsd versions but not the Solaris one! Further it is incorrect - the JLS/JVMS does not prevent spurious wakeups from being made visible to Java code. That has always been the intent but it took until Java 5 before we got the specification for Object.wait updated to make it clear. So really this is all legacy crud that should be removed along with the other planned cleanups (ie removing the safe_cond_* functions and WorkAroundNPTLTimedWaitHang) and the os_posix refactoring. --- Sorry to mess up your "starter" bug. David > product(intx, NativeMonitorTimeout, -1, > "(Unstable)") \ > \ > > bug: https://bugs.openjdk.java.net/browse/JDK-8044339 > From rednaxelafx at gmail.com Fri May 30 06:07:35 2014 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 29 May 2014 23:07:35 -0700 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <5388007D.9070803@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> <5388007D.9070803@oracle.com> Message-ID: Hi Dan, That's very nice of you. Thank you very much! Best regards, Kris On Thu, May 29, 2014 at 8:52 PM, Daniel D. Daugherty < daniel.daugherty at oracle.com> wrote: > Thanks for the review! > > BTW, I'm listing you, Zhengyu and me on the 'Contributed-by' line > since the three of us had a hand in various versions and parts of > this fix. :-) > > Dan > > > On 5/29/14 7:39 PM, Krystal Mok wrote: > > Hi Dan, > > The new fix looks good to me (not a Reviewer). Thumbs up! > > Thanks, > Kris > > > On Thu, May 29, 2014 at 7:49 AM, Daniel D. Daugherty < > daniel.daugherty at oracle.com> wrote: > >> One more round of review after refactoring the test based on comments >> from David H and Serguei. >> >> Here's the webrev for this round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ >> >> Had to change the default sample size from 30 -> 15 in order to >> get the test to pass reliably on Solaris SPARC JPRT machines. >> >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >> >>> Zhengyu is tied up with some other work so I've taken on this fix. >>> >>> Here's the webrev URL for the next round: >>> >>> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >>> >>> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >>> The new test has been run with the fix via JPRT and passes on all >>> JPRT platforms. The new test has also been run without the fix and >>> fails on most platforms. Since the default sample size is just 30, >>> it is possible to get 30 runs in a row without failing. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >>> >>> >>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>> >>>> This is a simple fix for incorrect lock state. >>>> >>>> The timing on setting thread's pending monitor can result stack trace >>>> dump reporting incorrect lock state. The solution is to check the monitor's >>>> owner, if the owner is other than the current thread, then the monitor, is >>>> or is in process of being, set the pending monitor of current thread. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>> >>> >>> >>> >> > > From staffan.larsen at oracle.com Fri May 30 06:36:11 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 30 May 2014 08:36:11 +0200 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: <53864680.106@univ-mlv.fr> References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> <7390BCB6-5572-481F-B984-A2AABB76280B@oracle.com> <53863E5B.6050205@oracle.com> <53864680.106@univ-mlv.fr> Message-ID: Leaving the naming behind for a moment. Any thoughts on the rest of the code? /Staffan On 28 maj 2014, at 22:26, Remi Forax wrote: > > On 05/28/2014 09:51 PM, Coleen Phillimore wrote: >> >> On 5/28/14, 3:49 PM, Christian Thalinger wrote: >>> On May 28, 2014, at 12:46 PM, Staffan Larsen wrote: >>> >>>> On 28 maj 2014, at 21:39, Christian Thalinger wrote: >>>> >>>>> On May 28, 2014, at 7:50 AM, Staffan Larsen wrote: >>>>> >>>>>> This change adds a new Diagnostic Command to list all ClassLoaders and some statistics for each classloader. The command is called ?GC.classloader_stats? and the information listed is: >>>>>> >>>>>> * An id for the ClassLoader. This is the pointer to the Klass of the ClassLoader. The reason for using the Klass* (instead of the oop) is that it is stable across invocations. >>>>>> * The id of the ClassLoader?s parent ClassLoader. >>>>>> * The pointer to the ClassLoaderData structure in the VM. This can be useful for debugging. >>>>>> * The number of classes loaded by the ClassLoader. >>>>>> * The total size of all allocated metaspace chunks for the ClassLoader. >>>>>> * The total size of all allocated metaspace blocks for the ClassLoader. >>>>>> >>>>>> If there are anonymous classes (invokedynamic classes) attributed to the ClassLoader, the following additional information is listed: >>>>>> * The number of anonymous classes loaded by the ClassLoader. >>>>>> * The total size of all allocated metaspace chunks for anonymous classes in the ClassLoader. >>>>>> * The total size of all allocated metaspace blocks for anonymous classes in the ClassLoader. >>>>>> >>>>>> The information is gathered during a safe point to guarantee that the data structures are consistent. >>>>>> >>>>>> I have added a small test and have run this through jprt. A CCC request has been filed. >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8044107 >>>>>> >>>>>> Example output: >>>>>> >>>>>> ClassLoader Parent CLData* Classes ChunkSz BlockSz Type >>>>>> 0x00000007c002d908 0x0000000000000000 0x0000000000000000 0 0 0 sun.misc.Launcher$ExtClassLoader >>>>>> 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 4694016 4241312 >>>>>> 37 75776 50928 + invokedynamic classes >>>>> You shouldn?t name these ?invokedynamic classes?; they can be used for different purposes. ?anonymous classes? is the correct name. >>>> Hmm, yeah. Not a good name, I know. The problem with ?anonymous classes? is that for a Java developer, that term is already taken and means something completely different. This is very unfortunate and creates a lot of confusion. My first version called them ?lambdas?, but that was even more wrong. Better names are welcome! >>> These classes are defined via: >>> >>> public native Class defineAnonymousClass(Class hostClass, byte[] data, Object[] cpPatches); >>> >>> so this is what they are called. Sorry :-) >> >> It's still a bad name. I call them jsr292-anonymous classes. maybe "unsafe anonymous classes" ? >> >> Coleen > > John Rose has coined the name 'anonymous klasses' which is not that bad. > > R?mi > >>> >>>> /Staffan >>>> >>>>>> 0x00000007c0061028 0x00000007c0036878 0x00007fb239c2de60 1 6144 1976 ClassLoaderStatsTest$DummyClassLoader >>>>>> 1 2048 1288 + invokedynamic classes >>>>>> 0x00000007c0036878 0x00000007c002d908 0x00007fb239e10fc0 8 88064 31544 sun.misc.Launcher$AppClassLoader >>>>>> Total = 4 808 4866048 4327048 >>>>>> ChunkSz: Total size of all allocated metaspace chunks >>>>>> BlockSz: Total size of all allocated metaspace blocks (each chunk has several blocks) >>>>>> >>>>>> >>>>>> Thanks, >>>>>> /Staffan From mikael.gerdin at oracle.com Fri May 30 08:49:08 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 30 May 2014 10:49:08 +0200 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> Message-ID: <1563822.vJaocFZGNS@mgerdin03> Hi Staffan, On Wednesday 28 May 2014 16.50.28 Staffan Larsen wrote: > This change adds a new Diagnostic Command to list all ClassLoaders and some > statistics for each classloader. The command is called > ?GC.classloader_stats? and the information listed is: > > * An id for the ClassLoader. This is the pointer to the Klass of the > ClassLoader. The reason for using the Klass* (instead of the oop) is that > it is stable across invocations. * The id of the ClassLoader?s parent > ClassLoader. > * The pointer to the ClassLoaderData structure in the VM. This can be useful > for debugging. * The number of classes loaded by the ClassLoader. > * The total size of all allocated metaspace chunks for the ClassLoader. > * The total size of all allocated metaspace blocks for the ClassLoader. > > If there are anonymous classes (invokedynamic classes) attributed to the > ClassLoader, the following additional information is listed: * The number > of anonymous classes loaded by the ClassLoader. > * The total size of all allocated metaspace chunks for anonymous classes in > the ClassLoader. * The total size of all allocated metaspace blocks for > anonymous classes in the ClassLoader. > > The information is gathered during a safe point to guarantee that the data > structures are consistent. > > I have added a small test and have run this through jprt. A CCC request has > been filed. > > webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8044107 I'm going to leave the naming of the anonymous classes and dig into the code a bit: in classLoaderStats.hpp: Use the (at runtime) known object alignment? 105 return hash ^ (hash >> 3); // just in case we're dealing with aligned ptrs The &-operator is not needed here: 108 typedef ResourceHashtable StatsTable; do_entry does not need to be virtual since the template instantiation binds to the symbol during compilation 129 virtual bool do_entry(oop const& key, ClassLoaderStats* const& cls); in classLoaderStats.cpp: Every time you encounter a CLD which reports a ClassLoader oop you will overwrite the _cld field of the ClassLoaderStats. 59 cls->_cld = cld; I'm not sure if this was your intention or not, but you are relying on the subtle fact that since CLD:s are prepended to the linked list in CLDG the _cld field will end up pointing to the ClassLoader's CLD instead of any of the anonymous classes which also hash to the same ClassLoder oop but have different CLDs. Can you convert this to an if()-block or add parenthesis for clarity? 60 cls->_classloader = cl == NULL ? NULL : cl->klass(); The ResourceMark here is unnecessary, all the resource allocations done by the ClassLoaderStats-classes are done by the VMThread and are in the scope of a ResourceMark covering the call to VM_Operation::doit() 165 ResourceMark rm; in ClassLoaderStatsTest.java: There is at least one other variant of "DummyClassLoader" in the testlibrary, can you see if any of them are reusable? /Mikael > > Example output: > > ClassLoader Parent CLData* Classes ChunkSz > BlockSz Type 0x00000007c002d908 0x0000000000000000 0x0000000000000000 > 0 0 0 sun.misc.Launcher$ExtClassLoader > 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 > 4694016 4241312 37 75776 50928 + > invokedynamic classes 0x00000007c0061028 0x00000007c0036878 > 0x00007fb239c2de60 1 6144 1976 > ClassLoaderStatsTest$DummyClassLoader 1 2048 1288 + > invokedynamic classes 0x00000007c0036878 0x00000007c002d908 > 0x00007fb239e10fc0 8 88064 31544 > sun.misc.Launcher$AppClassLoader Total = 4 > 808 4866048 4327048 ChunkSz: Total size of all > allocated metaspace chunks > BlockSz: Total size of all allocated metaspace blocks (each chunk has > several blocks) > > > Thanks, > /Staffan From igor.ignatyev at oracle.com Fri May 30 09:01:23 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 30 May 2014 13:01:23 +0400 Subject: RFR(S) : 8038756 : new WB API :: get/setVMFlag In-Reply-To: <4F499D2F-C30C-421F-89A3-6AC3BF1F99D5@oracle.com> References: <53388795.9070005@oracle.com> <533A892C.2090208@oracle.com> <330EABDB-3376-4559-A077-2FD853D3C0FA@oracle.com> <533ACC52.4000000@oracle.com> <5EB6C74E-71DC-47DF-A961-053C39C1B3B4@oracle.com> <533BF65C.8080509@oracle.com> <4F499D2F-C30C-421F-89A3-6AC3BF1F99D5@oracle.com> Message-ID: <538848E3.7080106@oracle.com> On 04/02/2014 03:46 PM, Staffan Larsen wrote: > > On 2 apr 2014, at 13:37, Igor Ignatyev wrote: > >> Staffan, >> >> I'd like not to have verifying of flags' value in these WB methods. Since it's a "Unsupported internal testing API", we don't have to have them. > > That could work. Until someone uses the WB Api to change some flag that really should cause other flags to be updated as well. But I?ll guess we can take that problem when it comes. > >> Also I'd like to have a possibility to change even "readonly" flags, which is against verifying logic I guess, e.g. UseCodeCacheFlushing for stress testing of code cache (JDK-8028595). > > Changing a flag that is not meant to be changed at runtime (not marked as ?manageable? or ?read_write') could cause problems. For example if having the flag set requires some init code to have been run. If the flag is safe for changing at runtime, can?t you just mark it as read_write? Again, since WB API is a "Unsupported internal testing API", I think it's ok to get such problems. Moreover, it can be a valid test scenario, e.g. change flags which ain't supposed to be changed and get a crash. > /Staffan > >> >> Thanks >> Igor >> >> On 04/02/2014 02:27 PM, Staffan Larsen wrote: >>> >>> On 1 apr 2014, at 16:25, Lev Priima wrote: >>> >>>> Staffan, >>>> >>>> sun.tools.* API(as well as all tools.jar) does not present in compact1 profile. >>> >>> I see, that?s a reason to add this. However, do we really need to test this specifically on compact1? I?m not sure what tests you are planning to use this feature for, but if it is for testing code that we don?t believe is special for compact1, then I?m not sure we need to run the tests on compact1. Maybe a stupid optimization. >>> >>> One larger problem here is that this introduced yet another way to set the flags. So far we have three other ways: attachListener.cpp, management.cpp, and arguments.cpp. Currently these three all have duplicated code for verifying the consistency of some of the flags. See the calls to Arguments::verify_MinHeapFreeRatio() for example. If we add yet another way, then that code needs to be copied again. I didn?t like the copying in the first place, but allowed it because we were too lazy at that point to refactor the code? At this point, however, I would like to see that duplication taken care of before we add another way to set the flags. I?m sorry that this is going to make this change harder for you, but we really should only have that code in one place. >>> >>> /Staffan >>> >>> >>>> >>>> Lev >>>> >>>> On 04/01/2014 05:02 PM, Staffan Larsen wrote: >>>>> There is already an API for getting / setting flags in sun.tools.attach.HotSpotVirtualMachine. If at all possible, we should reuse that API instead of adding a new one. >>>>> >>>>> Typical use: >>>>> >>>>> HotSpotVirtualMachine vm = (HotSpotVirtualMachine) VirtualMachine.attach(pid); >>>>> vm.setFlag(?MyFlag?, ?MyValue?); >>>>> >>>>> There is also an API in sun.management.HotSpotDiagnostic for setting / getting as well as querying for options in JVM. >>>>> >>>>> Thanks, >>>>> /Staffan >>>>> >>>>> >>>>> On 1 apr 2014, at 11:38, Igor Ignatyev wrote: >>>>> >>>>>> adding hotspot-dev alias. >>>>>> >>>>>> On 03/31/2014 01:07 AM, Igor Ignatyev wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review the patch which introduces new WhiteBox methods to get and >>>>>>> set VM flags. >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8038756/webrev.00/ >>>>>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8038756 >>>>>>> testing: jprt >>>> >>> > From staffan.larsen at oracle.com Fri May 30 09:40:54 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 30 May 2014 11:40:54 +0200 Subject: RFR: 8044107 Add Diagnostic Command to list all ClassLoaders In-Reply-To: <1563822.vJaocFZGNS@mgerdin03> References: <3921F7BD-DB5B-4610-B126-9CA89D5C381C@oracle.com> <1563822.vJaocFZGNS@mgerdin03> Message-ID: On 30 maj 2014, at 10:49, Mikael Gerdin wrote: > Hi Staffan, > > On Wednesday 28 May 2014 16.50.28 Staffan Larsen wrote: >> This change adds a new Diagnostic Command to list all ClassLoaders and some >> statistics for each classloader. The command is called >> ?GC.classloader_stats? and the information listed is: >> >> * An id for the ClassLoader. This is the pointer to the Klass of the >> ClassLoader. The reason for using the Klass* (instead of the oop) is that >> it is stable across invocations. * The id of the ClassLoader?s parent >> ClassLoader. >> * The pointer to the ClassLoaderData structure in the VM. This can be useful >> for debugging. * The number of classes loaded by the ClassLoader. >> * The total size of all allocated metaspace chunks for the ClassLoader. >> * The total size of all allocated metaspace blocks for the ClassLoader. >> >> If there are anonymous classes (invokedynamic classes) attributed to the >> ClassLoader, the following additional information is listed: * The number >> of anonymous classes loaded by the ClassLoader. >> * The total size of all allocated metaspace chunks for anonymous classes in >> the ClassLoader. * The total size of all allocated metaspace blocks for >> anonymous classes in the ClassLoader. >> >> The information is gathered during a safe point to guarantee that the data >> structures are consistent. >> >> I have added a small test and have run this through jprt. A CCC request has >> been filed. >> >> webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8044107 > > I'm going to leave the naming of the anonymous classes and dig into the code a > bit: > > in classLoaderStats.hpp: > > Use the (at runtime) known object alignment? > 105 return hash ^ (hash >> 3); // just in case we're dealing with aligned > ptrs I just copied code from resourceHash.hpp? What would you suggest? > > The &-operator is not needed here: > 108 typedef ResourceHashtable 109 &ClassLoaderStatsClosure::oop_hash, > &ClassLoaderStatsClosure::oop_equals> StatsTable; Removed. > do_entry does not need to be virtual since the template instantiation binds to > the symbol during compilation > 129 virtual bool do_entry(oop const& key, ClassLoaderStats* const& cls); I want to override it in a closed source class, so I?ll keep it as virtual. > in classLoaderStats.cpp: > > Every time you encounter a CLD which reports a ClassLoader oop you will > overwrite the _cld field of the ClassLoaderStats. > 59 cls->_cld = cld; > > I'm not sure if this was your intention or not, but you are relying on the > subtle fact that since CLD:s are prepended to the linked list in CLDG the _cld > field will end up pointing to the ClassLoader's CLD instead of any of the > anonymous classes which also hash to the same ClassLoder oop but have > different CLDs. Good catch. The code should be: if (!cld->is_anonymous()) { cls->_cld = cld; } > Can you convert this to an if()-block or add parenthesis for clarity? > 60 cls->_classloader = cl == NULL ? NULL : cl->klass(); Added parenthesis. > > The ResourceMark here is unnecessary, all the resource allocations done by the > ClassLoaderStats-classes are done by the VMThread and are in the scope of a > ResourceMark covering the call to VM_Operation::doit() > 165 ResourceMark rm; I wasn?t aware that doit() was already covered. Removed. > in ClassLoaderStatsTest.java: > There is at least one other variant of "DummyClassLoader" in the testlibrary, > can you see if any of them are reusable? I could not find one that did what I wanted. Which one are you referring to? I also changed the name of the diagnostic command from ?GC.classloader_stats? to ?VM.classloader_stats? and the output of anonymous classes to "+ unsafe anonymous classes?. new webrev: http://cr.openjdk.java.net/~sla/8044107/webrev.01/ Thanks, /Staffan > > /Mikael > >> >> Example output: >> >> ClassLoader Parent CLData* Classes ChunkSz >> BlockSz Type 0x00000007c002d908 0x0000000000000000 0x0000000000000000 >> 0 0 0 sun.misc.Launcher$ExtClassLoader >> 0x0000000000000000 0x0000000000000000 0x00007fb239c08de0 761 >> 4694016 4241312 37 75776 50928 + >> invokedynamic classes 0x00000007c0061028 0x00000007c0036878 >> 0x00007fb239c2de60 1 6144 1976 >> ClassLoaderStatsTest$DummyClassLoader 1 2048 1288 + >> invokedynamic classes 0x00000007c0036878 0x00000007c002d908 >> 0x00007fb239e10fc0 8 88064 31544 >> sun.misc.Launcher$AppClassLoader Total = 4 >> 808 4866048 4327048 ChunkSz: Total size of all >> allocated metaspace chunks >> BlockSz: Total size of all allocated metaspace blocks (each chunk has >> several blocks) >> >> >> Thanks, >> /Staffan From staffan.larsen at oracle.com Fri May 30 09:42:55 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 30 May 2014 11:42:55 +0200 Subject: RFR(S) : 8038756 : new WB API :: get/setVMFlag In-Reply-To: <538848E3.7080106@oracle.com> References: <53388795.9070005@oracle.com> <533A892C.2090208@oracle.com> <330EABDB-3376-4559-A077-2FD853D3C0FA@oracle.com> <533ACC52.4000000@oracle.com> <5EB6C74E-71DC-47DF-A961-053C39C1B3B4@oracle.com> <533BF65C.8080509@oracle.com> <4F499D2F-C30C-421F-89A3-6AC3BF1F99D5@oracle.com> <538848E3.7080106@oracle.com> Message-ID: <2A4E2CEF-4C19-45F2-BEAD-1E52A937C681@oracle.com> On 30 maj 2014, at 11:01, Igor Ignatyev wrote: > On 04/02/2014 03:46 PM, Staffan Larsen wrote: >> >> On 2 apr 2014, at 13:37, Igor Ignatyev wrote: >> >>> Staffan, >>> >>> I'd like not to have verifying of flags' value in these WB methods. Since it's a "Unsupported internal testing API", we don't have to have them. >> >> That could work. Until someone uses the WB Api to change some flag that really should cause other flags to be updated as well. But I?ll guess we can take that problem when it comes. >> >>> Also I'd like to have a possibility to change even "readonly" flags, which is against verifying logic I guess, e.g. UseCodeCacheFlushing for stress testing of code cache (JDK-8028595). >> >> Changing a flag that is not meant to be changed at runtime (not marked as ?manageable? or ?read_write') could cause problems. For example if having the flag set requires some init code to have been run. If the flag is safe for changing at runtime, can?t you just mark it as read_write? > Again, since WB API is a "Unsupported internal testing API", I think it's ok to get such problems. Moreover, it can be a valid test scenario, e.g. change flags which ain't supposed to be changed and get a crash. Ok. >> /Staffan >> >>> >>> Thanks >>> Igor >>> >>> On 04/02/2014 02:27 PM, Staffan Larsen wrote: >>>> >>>> On 1 apr 2014, at 16:25, Lev Priima wrote: >>>> >>>>> Staffan, >>>>> >>>>> sun.tools.* API(as well as all tools.jar) does not present in compact1 profile. >>>> >>>> I see, that?s a reason to add this. However, do we really need to test this specifically on compact1? I?m not sure what tests you are planning to use this feature for, but if it is for testing code that we don?t believe is special for compact1, then I?m not sure we need to run the tests on compact1. Maybe a stupid optimization. >>>> >>>> One larger problem here is that this introduced yet another way to set the flags. So far we have three other ways: attachListener.cpp, management.cpp, and arguments.cpp. Currently these three all have duplicated code for verifying the consistency of some of the flags. See the calls to Arguments::verify_MinHeapFreeRatio() for example. If we add yet another way, then that code needs to be copied again. I didn?t like the copying in the first place, but allowed it because we were too lazy at that point to refactor the code? At this point, however, I would like to see that duplication taken care of before we add another way to set the flags. I?m sorry that this is going to make this change harder for you, but we really should only have that code in one place. >>>> >>>> /Staffan >>>> >>>> >>>>> >>>>> Lev >>>>> >>>>> On 04/01/2014 05:02 PM, Staffan Larsen wrote: >>>>>> There is already an API for getting / setting flags in sun.tools.attach.HotSpotVirtualMachine. If at all possible, we should reuse that API instead of adding a new one. >>>>>> >>>>>> Typical use: >>>>>> >>>>>> HotSpotVirtualMachine vm = (HotSpotVirtualMachine) VirtualMachine.attach(pid); >>>>>> vm.setFlag(?MyFlag?, ?MyValue?); >>>>>> >>>>>> There is also an API in sun.management.HotSpotDiagnostic for setting / getting as well as querying for options in JVM. >>>>>> >>>>>> Thanks, >>>>>> /Staffan >>>>>> >>>>>> >>>>>> On 1 apr 2014, at 11:38, Igor Ignatyev wrote: >>>>>> >>>>>>> adding hotspot-dev alias. >>>>>>> >>>>>>> On 03/31/2014 01:07 AM, Igor Ignatyev wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review the patch which introduces new WhiteBox methods to get and >>>>>>>> set VM flags. >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8038756/webrev.00/ >>>>>>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8038756 >>>>>>>> testing: jprt From zhengyu.gu at oracle.com Fri May 30 12:15:22 2014 From: zhengyu.gu at oracle.com (Zhengyu Gu) Date: Fri, 30 May 2014 08:15:22 -0400 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <53874908.5070805@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> Message-ID: <5388765A.2040103@oracle.com> Good to me. Thanks Dan, -Zhengyu On 5/29/2014 10:49 AM, Daniel D. Daugherty wrote: > One more round of review after refactoring the test based on comments > from David H and Serguei. > > Here's the webrev for this round: > > http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ > > Had to change the default sample size from 30 -> 15 in order to > get the test to pass reliably on Solaris SPARC JPRT machines. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >> Zhengyu is tied up with some other work so I've taken on this fix. >> >> Here's the webrev URL for the next round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >> >> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >> The new test has been run with the fix via JPRT and passes on all >> JPRT platforms. The new test has also been run without the fix and >> fails on most platforms. Since the default sample size is just 30, >> it is possible to get 30 runs in a row without failing. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>> This is a simple fix for incorrect lock state. >>> >>> The timing on setting thread's pending monitor can result stack >>> trace dump reporting incorrect lock state. The solution is to check >>> the monitor's owner, if the owner is other than the current thread, >>> then the monitor, is or is in process of being, set the pending >>> monitor of current thread. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>> >>> >>> Thanks, >>> >>> -Zhengyu >>> >> >> >> > From vladimir.x.ivanov at oracle.com Fri May 30 12:22:06 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 30 May 2014 16:22:06 +0400 Subject: RFR(S) : 8038756 : new WB API :: get/setVMFlag In-Reply-To: <533A892C.2090208@oracle.com> References: <53388795.9070005@oracle.com> <533A892C.2090208@oracle.com> Message-ID: <538877EE.5080001@oracle.com> Looks good. Best regards, Vladimir Ivanov On 4/1/14 1:38 PM, Igor Ignatyev wrote: > adding hotspot-dev alias. > > On 03/31/2014 01:07 AM, Igor Ignatyev wrote: >> Hi all, >> >> Please review the patch which introduces new WhiteBox methods to get and >> set VM flags. >> >> webrev: http://cr.openjdk.java.net/~iignatyev/8038756/webrev.00/ >> jbs: https://bugs.openjdk.java.net/browse/JDK-8038756 >> testing: jprt From daniel.daugherty at oracle.com Fri May 30 13:13:06 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 30 May 2014 07:13:06 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <53881DC0.8090009@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> <53881DC0.8090009@oracle.com> Message-ID: <538883E2.9000109@oracle.com> Thanks! Dan On 5/29/14 11:57 PM, David Holmes wrote: > Thanks Dan! Good to go. > > David > > On 30/05/2014 12:49 AM, Daniel D. Daugherty wrote: >> One more round of review after refactoring the test based on comments >> from David H and Serguei. >> >> Here's the webrev for this round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ >> >> Had to change the default sample size from 30 -> 15 in order to >> get the test to pass reliably on Solaris SPARC JPRT machines. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >>> Zhengyu is tied up with some other work so I've taken on this fix. >>> >>> Here's the webrev URL for the next round: >>> >>> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >>> >>> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >>> The new test has been run with the fix via JPRT and passes on all >>> JPRT platforms. The new test has also been run without the fix and >>> fails on most platforms. Since the default sample size is just 30, >>> it is possible to get 30 runs in a row without failing. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >>> >>> >>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>> This is a simple fix for incorrect lock state. >>>> >>>> The timing on setting thread's pending monitor can result stack trace >>>> dump reporting incorrect lock state. The solution is to check the >>>> monitor's owner, if the owner is other than the current thread, then >>>> the monitor, is or is in process of being, set the pending monitor of >>>> current thread. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>> >>> >>> >> From daniel.daugherty at oracle.com Fri May 30 13:13:27 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 30 May 2014 07:13:27 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: <5388765A.2040103@oracle.com> References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> <5388765A.2040103@oracle.com> Message-ID: <538883F7.70106@oracle.com> Thanks! Dan On 5/30/14 6:15 AM, Zhengyu Gu wrote: > Good to me. > > Thanks Dan, > > -Zhengyu > > On 5/29/2014 10:49 AM, Daniel D. Daugherty wrote: >> One more round of review after refactoring the test based on comments >> from David H and Serguei. >> >> Here's the webrev for this round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ >> >> Had to change the default sample size from 30 -> 15 in order to >> get the test to pass reliably on Solaris SPARC JPRT machines. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >>> Zhengyu is tied up with some other work so I've taken on this fix. >>> >>> Here's the webrev URL for the next round: >>> >>> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >>> >>> The fix has been tested with vm.quick on all Aurora Adhoc platforms. >>> The new test has been run with the fix via JPRT and passes on all >>> JPRT platforms. The new test has also been run without the fix and >>> fails on most platforms. Since the default sample size is just 30, >>> it is possible to get 30 runs in a row without failing. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >>> >>> >>> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >>>> This is a simple fix for incorrect lock state. >>>> >>>> The timing on setting thread's pending monitor can result stack >>>> trace dump reporting incorrect lock state. The solution is to check >>>> the monitor's owner, if the owner is other than the current thread, >>>> then the monitor, is or is in process of being, set the pending >>>> monitor of current thread. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >>>> >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>> >>> >>> >> > From igor.ignatyev at oracle.com Fri May 30 13:14:45 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 30 May 2014 17:14:45 +0400 Subject: RFR(S) : 8038756 : new WB API :: get/setVMFlag In-Reply-To: <538877EE.5080001@oracle.com> References: <53388795.9070005@oracle.com> <533A892C.2090208@oracle.com> <538877EE.5080001@oracle.com> Message-ID: <53888445.5000801@oracle.com> Vladimir/Staffan, Thanks for reviews. Igor On 05/30/2014 04:22 PM, Vladimir Ivanov wrote: > Looks good. > > Best regards, > Vladimir Ivanov > > On 4/1/14 1:38 PM, Igor Ignatyev wrote: >> adding hotspot-dev alias. >> >> On 03/31/2014 01:07 AM, Igor Ignatyev wrote: >>> Hi all, >>> >>> Please review the patch which introduces new WhiteBox methods to get and >>> set VM flags. >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev/8038756/webrev.00/ >>> jbs: https://bugs.openjdk.java.net/browse/JDK-8038756 >>> testing: jprt From daniel.daugherty at oracle.com Fri May 30 14:14:19 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 30 May 2014 08:14:19 -0600 Subject: CodeHeap::expand_by malloc failed In-Reply-To: References: <5387D545.9080404@oracle.com> Message-ID: <5388923B.1000608@oracle.com> When 'os::commit_memory()' fails, there should be a message like the following in stderr for the Java process: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0xfffffd7fe8c00000, 2097152, 1) failed; error='Resource temporarily unavailable' (errno=11) The address and size should match what you see in the stack trace which provides confirmation that we have the right mesg, but the important piece is the errno value... Dan On 5/29/14 7:20 PM, Vitaly Davidovich wrote: > Yes, swap and overcommit are turned off. But, there's substantial free > physical memory available and looks like 65kb available to expand the code > heap contiguoisly. So I'm a bit puzzled ... > > Taking Chris' suggestion and cc'ing hotspot dev ... > > Sent from my phone > On May 29, 2014 8:47 PM, "Vladimir Kozlov" > wrote: > >> Could be because there is no swap on this machine: >> >> Memory: 4k page, physical 49521668k(5596796k free), swap 0k(0k free) >> >> Vladimir >> >> On 5/29/14 5:35 PM, Christian Thalinger wrote: >> >>> Although it?s the code cache I assume runtime folk would know more about >>> this. Maybe send to hotspot-dev. >>> >>> On May 29, 2014, at 9:01 AM, Vitaly Davidovich >> > wrote: >>> >>> Hi guys, >>>> Need a bit of help explaining a hotspot malloc failure crash on 7u51. >>>> I'm going to paste the relevant snippets from the hs_err log. >>>> >>>> # >>>> # There is insufficient memory for the Java Runtime Environment to >>>> continue. >>>> # Native memory allocation (malloc) failed to allocate 65536 bytes for >>>> committing reserved memory. >>>> # Possible reasons: >>>> # The system is out of physical RAM or swap space >>>> # In 32 bit mode, the process size limit was hit >>>> # Possible solutions: >>>> # Reduce memory load on the system >>>> # Increase physical memory or swap space >>>> # Check if swap backing store is full >>>> # Use 64 bit Java on a 64 bit OS >>>> # Decrease Java heap size (-Xmx/-Xms) >>>> # Decrease number of Java threads >>>> # Decrease Java thread stack sizes (-Xss) >>>> # Set larger code cache with -XX:ReservedCodeCacheSize= >>>> # This output file may be truncated or incomplete. >>>> # >>>> # Out of Memory Error (os_linux.cpp:2726), pid=10643, tid=47319048501520 >>>> # >>>> # JRE version: Java(TM) SE Runtime Environment (7.0_51-b13) (build >>>> 1.7.0_51-b13) >>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.51-b03 mixed mode >>>> linux-amd64 compressed oops) >>>> >>>> >>>> --------------- T H R E A D --------------- >>>> >>>> Current thread (0x0000000000716800): JavaThread "C2 CompilerThread0" >>>> daemon [_thread_in_vm, id=10689, >>>> stack(0x00002b095303b000,0x00002b095313c000)] >>>> >>>> Stack: [0x00002b095303b000,0x00002b095313c000], >>>> sp=0x00002b0953138d20, free space=1015k >>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>>> C=native code) >>>> V [libjvm.so+0x992f4a] VMError::report_and_die()+0x2ea >>>> V [libjvm.so+0x4931ab] report_vm_out_of_memory(char const*, int, >>>> unsigned long, char const*)+0x9b >>>> V [libjvm.so+0x81338e] os::Linux::commit_memory_impl(char*, unsigned >>>> long, bool)+0xfe >>>> V [libjvm.so+0x81383f] os::Linux::commit_memory_impl(char*, unsigned >>>> long, unsigned long, bool)+0x4f >>>> V [libjvm.so+0x813a2c] os::pd_commit_memory(char*, unsigned long, >>>> unsigned long, bool)+0xc >>>> V [libjvm.so+0x80daea] os::commit_memory(char*, unsigned long, >>>> unsigned long, bool)+0x2a >>>> V [libjvm.so+0x98e849] VirtualSpace::expand_by(unsigned long, >>>> bool)+0x1c9 >>>> V [libjvm.so+0x58a62c] CodeHeap::expand_by(unsigned long)+0x8c >>>> V [libjvm.so+0x42111d] CodeCache::allocate(int)+0x4d >>>> V [libjvm.so+0x7e1a39] nmethod::new_nmethod(methodHandle, int, int, >>>> CodeOffsets*, int, DebugInformationRecorder*, Dependencies*, >>>> CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*, >>>> ImplicitExceptionTable*, AbstractCompiler*, int)+0x179 >>>> V [libjvm.so+0x3cfc54] ciEnv::register_method(ciMethod*, int, >>>> CodeOffsets*, int, CodeBuffer*, int, OopMapSet*, >>>> ExceptionHandlerTable*, ImplicitExceptionTable*, AbstractCompiler*, >>>> int, bool, bool)+0x364 >>>> V [libjvm.so+0x4458fb] Compile::Compile(ciEnv*, C2Compiler*, >>>> ciMethod*, int, bool, bool)+0x11cb >>>> V [libjvm.so+0x3afa76] C2Compiler::compile_method(ciEnv*, ciMethod*, >>>> int)+0x176 >>>> V [libjvm.so+0x44ba9e] >>>> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x33e >>>> V [libjvm.so+0x44c87d] CompileBroker::compiler_thread_loop()+0x43d >>>> V [libjvm.so+0x94d5ff] JavaThread::thread_main_inner()+0xdf >>>> V [libjvm.so+0x94d705] JavaThread::run()+0xf5 >>>> V [libjvm.so+0x815538] java_start(Thread*)+0x108 >>>> >>>> Code Cache [0x00002b091755c000, 0x00002b091869c000, 0x00002b091a55c000) >>>> total_blobs=7033 nmethods=6349 adapters=634 free_code_cache=31715Kb >>>> largest_free_block=32247296 >>>> >>>> Memory mappings around the code cache virtual space (bolded line is >>>> the code cache segment, I believe, based on the Code Cache output above): >>>> >>>> *2b091755c000-2b091869c000 rwxp 00000000 00:00 0* >>>> 2b09186ac000-2b091d66f000 rw-p 00000000 00:00 0 >>>> 2b091d66f000-2b091d670000 ---p 00000000 00:00 0 >>>> 2b091d670000-2b091d770000 rwxp 00000000 00:00 0 >>>> 2b091d770000-2b091d771000 ---p 00000000 00:00 0 >>>> 2b091d771000-2b091d871000 rwxp 00000000 00:00 0 >>>> 2b091d871000-2b091d872000 ---p 00000000 00:00 0 >>>> 2b091d872000-2b091d972000 rwxp 00000000 00:00 0 >>>> >>>> /proc/meminfo: >>>> MemTotal: 49521668 kB >>>> MemFree: 5596796 kB >>>> Buffers: 294684 kB >>>> Cached: 34205856 kB >>>> SwapCached: 0 kB >>>> Active: 12745128 kB >>>> Inactive: 28516788 kB >>>> Active(anon): 9280636 kB >>>> Inactive(anon): 3090264 kB >>>> Active(file): 3464492 kB >>>> Inactive(file): 25426524 kB >>>> Unevictable: 14420 kB >>>> Mlocked: 14420 kB >>>> SwapTotal: 0 kB >>>> SwapFree: 0 kB >>>> Dirty: 20392 kB >>>> Writeback: 0 kB >>>> AnonPages: 6776540 kB >>>> Mapped: 6292204 kB >>>> Shmem: 5604620 kB >>>> Slab: 1828656 kB >>>> SReclaimable: 1567928 kB >>>> SUnreclaim: 260728 kB >>>> KernelStack: 4648 kB >>>> PageTables: 58800 kB >>>> NFS_Unstable: 0 kB >>>> Bounce: 0 kB >>>> WritebackTmp: 0 kB >>>> CommitLimit: 47045584 kB >>>> Committed_AS: 46046480 kB >>>> VmallocTotal: 34359738367 kB >>>> VmallocUsed: 460188 kB >>>> VmallocChunk: 34333703480 kB >>>> HugePages_Total: 0 >>>> HugePages_Free: 0 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> DirectMap4k: 8192 kB >>>> DirectMap2M: 2080768 kB >>>> DirectMap1G: 48234496 kB >>>> >>>> Memory: 4k page, physical 49521668k(5596796k free), swap 0k(0k free) >>>> >>>> * overcommit is turned off >>>> >>>> So the code heap was attempting to expand by 65kb at a fixed address. >>>> There appears to be a 65kb mapping available between the end of the >>>> current code heap mapping (2b091755c000-2b091869c000) and the next one >>>> (2b09186ac000-2b091d66f000). There's about 30mb of free space left >>>> (reserved, but uncommitted I take it) in the virtual space, so it's a >>>> bit puzzling given there's ample physical mem available. Only thing I >>>> can think of is expansion fails because it cannot get contiguous >>>> space, but then I can't reconcile that with the mem mapping above. >>>> >>>> Does anyone have any ideas? >>>> >>>> Let me know if you need additional info from the hs_err file. >>>> >>>> Thanks >>>> >>>> From daniel.daugherty at oracle.com Fri May 30 14:26:39 2014 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 30 May 2014 08:26:39 -0600 Subject: RFR(XS) 8036823: Stack trace sometimes shows 'locked' instead of 'waiting to lock' In-Reply-To: References: <537A0E1C.4070605@oracle.com> <537ECC1A.1030409@oracle.com> <53874908.5070805@oracle.com> <5388007D.9070803@oracle.com> Message-ID: <5388951F.30709@oracle.com> On 5/30/14 12:07 AM, Krystal Mok wrote: > Hi Dan, > > That's very nice of you. Thank you very much! No problem. Happy to share the blame if we get this wrong again! :-O Just for the record, any problems with the test are mine! Normally I would write a test like this using shell (and the first version attached to the bug is implemented that way), but we're trying very hard to not to add any more shell tests to the repository. Dan > > Best regards, > Kris > > > On Thu, May 29, 2014 at 8:52 PM, Daniel D. Daugherty > > wrote: > > Thanks for the review! > > BTW, I'm listing you, Zhengyu and me on the 'Contributed-by' line > since the three of us had a hand in various versions and parts of > this fix. :-) > > Dan > > > On 5/29/14 7:39 PM, Krystal Mok wrote: >> Hi Dan, >> >> The new fix looks good to me (not a Reviewer). Thumbs up! >> >> Thanks, >> Kris >> >> >> On Thu, May 29, 2014 at 7:49 AM, Daniel D. Daugherty >> > > wrote: >> >> One more round of review after refactoring the test based on >> comments >> from David H and Serguei. >> >> Here's the webrev for this round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/2-jdk9-hs-rt/ >> >> >> Had to change the default sample size from 30 -> 15 in order to >> get the test to pass reliably on Solaris SPARC JPRT machines. >> >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> On 5/22/14 10:18 PM, Daniel D. Daugherty wrote: >> >> Zhengyu is tied up with some other work so I've taken on >> this fix. >> >> Here's the webrev URL for the next round: >> >> http://cr.openjdk.java.net/~dcubed/8036823-webrev/1-jdk9-hs-rt/ >> >> >> The fix has been tested with vm.quick on all Aurora Adhoc >> platforms. >> The new test has been run with the fix via JPRT and >> passes on all >> JPRT platforms. The new test has also been run without >> the fix and >> fails on most platforms. Since the default sample size is >> just 30, >> it is possible to get 30 runs in a row without failing. >> >> Thanks, in advance, for any comments, questions or >> suggestions. >> >> Dan >> >> >> On 5/19/14 7:58 AM, Zhengyu Gu wrote: >> >> This is a simple fix for incorrect lock state. >> >> The timing on setting thread's pending monitor can >> result stack trace dump reporting incorrect lock >> state. The solution is to check the monitor's owner, >> if the owner is other than the current thread, then >> the monitor, is or is in process of being, set the >> pending monitor of current thread. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8036823 >> Webrev: >> http://cr.openjdk.java.net/~zgu/8036823/webrev.00/ >> >> >> >> Thanks, >> >> -Zhengyu >> >> >> >> >> >> > > From vitalyd at gmail.com Fri May 30 15:04:28 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 30 May 2014 11:04:28 -0400 Subject: CodeHeap::expand_by malloc failed In-Reply-To: <5388923B.1000608@oracle.com> References: <5387D545.9080404@oracle.com> <5388923B.1000608@oracle.com> Message-ID: Hi Dan, Thanks for the pointer. Here's the relevant line: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00002b091869c000, 65536, 1) failed; error='Cannot allocate memory' (errno=12) So it confirms the fixed address at which it's trying to commit and the 65kb size. The address matches the end of the currently committed code heap region, but still somewhat puzzling as to why ENOMEM is returned given the rest of the output in the error report. Perhaps there was a transient spike in mem consumption when mmap failed but the condition disappeared (ie. mem was free'd back to os by another process) by the time the error reporter pulled mem stats from the kernel? Thanks On Fri, May 30, 2014 at 10:14 AM, Daniel D. Daugherty < daniel.daugherty at oracle.com> wrote: > When 'os::commit_memory()' fails, there should be a message like the > following in stderr for the Java process: > > Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0xfffffd7fe8c00000, > 2097152, 1) failed; error='Resource temporarily unavailable' (errno=11) > > The address and size should match what you see in the stack trace > which provides confirmation that we have the right mesg, but the > important piece is the errno value... > > Dan > > > > On 5/29/14 7:20 PM, Vitaly Davidovich wrote: > >> Yes, swap and overcommit are turned off. But, there's substantial free >> physical memory available and looks like 65kb available to expand the code >> heap contiguoisly. So I'm a bit puzzled ... >> >> Taking Chris' suggestion and cc'ing hotspot dev ... >> >> Sent from my phone >> On May 29, 2014 8:47 PM, "Vladimir Kozlov" >> wrote: >> >> Could be because there is no swap on this machine: >>> >>> Memory: 4k page, physical 49521668k(5596796k free), swap 0k(0k free) >>> >>> Vladimir >>> >>> On 5/29/14 5:35 PM, Christian Thalinger wrote: >>> >>> Although it?s the code cache I assume runtime folk would know more about >>>> this. Maybe send to hotspot-dev. >>>> >>>> On May 29, 2014, at 9:01 AM, Vitaly Davidovich >>> > wrote: >>>> >>>> Hi guys, >>>> >>>>> Need a bit of help explaining a hotspot malloc failure crash on 7u51. >>>>> I'm going to paste the relevant snippets from the hs_err log. >>>>> >>>>> # >>>>> # There is insufficient memory for the Java Runtime Environment to >>>>> continue. >>>>> # Native memory allocation (malloc) failed to allocate 65536 bytes for >>>>> committing reserved memory. >>>>> # Possible reasons: >>>>> # The system is out of physical RAM or swap space >>>>> # In 32 bit mode, the process size limit was hit >>>>> # Possible solutions: >>>>> # Reduce memory load on the system >>>>> # Increase physical memory or swap space >>>>> # Check if swap backing store is full >>>>> # Use 64 bit Java on a 64 bit OS >>>>> # Decrease Java heap size (-Xmx/-Xms) >>>>> # Decrease number of Java threads >>>>> # Decrease Java thread stack sizes (-Xss) >>>>> # Set larger code cache with -XX:ReservedCodeCacheSize= >>>>> # This output file may be truncated or incomplete. >>>>> # >>>>> # Out of Memory Error (os_linux.cpp:2726), pid=10643, >>>>> tid=47319048501520 >>>>> # >>>>> # JRE version: Java(TM) SE Runtime Environment (7.0_51-b13) (build >>>>> 1.7.0_51-b13) >>>>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.51-b03 mixed mode >>>>> linux-amd64 compressed oops) >>>>> >>>>> >>>>> --------------- T H R E A D --------------- >>>>> >>>>> Current thread (0x0000000000716800): JavaThread "C2 CompilerThread0" >>>>> daemon [_thread_in_vm, id=10689, >>>>> stack(0x00002b095303b000,0x00002b095313c000)] >>>>> >>>>> Stack: [0x00002b095303b000,0x00002b095313c000], >>>>> sp=0x00002b0953138d20, free space=1015k >>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, >>>>> C=native code) >>>>> V [libjvm.so+0x992f4a] VMError::report_and_die()+0x2ea >>>>> V [libjvm.so+0x4931ab] report_vm_out_of_memory(char const*, int, >>>>> unsigned long, char const*)+0x9b >>>>> V [libjvm.so+0x81338e] os::Linux::commit_memory_impl(char*, unsigned >>>>> long, bool)+0xfe >>>>> V [libjvm.so+0x81383f] os::Linux::commit_memory_impl(char*, unsigned >>>>> long, unsigned long, bool)+0x4f >>>>> V [libjvm.so+0x813a2c] os::pd_commit_memory(char*, unsigned long, >>>>> unsigned long, bool)+0xc >>>>> V [libjvm.so+0x80daea] os::commit_memory(char*, unsigned long, >>>>> unsigned long, bool)+0x2a >>>>> V [libjvm.so+0x98e849] VirtualSpace::expand_by(unsigned long, >>>>> bool)+0x1c9 >>>>> V [libjvm.so+0x58a62c] CodeHeap::expand_by(unsigned long)+0x8c >>>>> V [libjvm.so+0x42111d] CodeCache::allocate(int)+0x4d >>>>> V [libjvm.so+0x7e1a39] nmethod::new_nmethod(methodHandle, int, int, >>>>> CodeOffsets*, int, DebugInformationRecorder*, Dependencies*, >>>>> CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*, >>>>> ImplicitExceptionTable*, AbstractCompiler*, int)+0x179 >>>>> V [libjvm.so+0x3cfc54] ciEnv::register_method(ciMethod*, int, >>>>> CodeOffsets*, int, CodeBuffer*, int, OopMapSet*, >>>>> ExceptionHandlerTable*, ImplicitExceptionTable*, AbstractCompiler*, >>>>> int, bool, bool)+0x364 >>>>> V [libjvm.so+0x4458fb] Compile::Compile(ciEnv*, C2Compiler*, >>>>> ciMethod*, int, bool, bool)+0x11cb >>>>> V [libjvm.so+0x3afa76] C2Compiler::compile_method(ciEnv*, ciMethod*, >>>>> int)+0x176 >>>>> V [libjvm.so+0x44ba9e] >>>>> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x33e >>>>> V [libjvm.so+0x44c87d] CompileBroker::compiler_thread_loop()+0x43d >>>>> V [libjvm.so+0x94d5ff] JavaThread::thread_main_inner()+0xdf >>>>> V [libjvm.so+0x94d705] JavaThread::run()+0xf5 >>>>> V [libjvm.so+0x815538] java_start(Thread*)+0x108 >>>>> >>>>> Code Cache [0x00002b091755c000, 0x00002b091869c000, >>>>> 0x00002b091a55c000) >>>>> total_blobs=7033 nmethods=6349 adapters=634 free_code_cache=31715Kb >>>>> largest_free_block=32247296 >>>>> >>>>> Memory mappings around the code cache virtual space (bolded line is >>>>> the code cache segment, I believe, based on the Code Cache output >>>>> above): >>>>> >>>>> *2b091755c000-2b091869c000 rwxp 00000000 00:00 0* >>>>> 2b09186ac000-2b091d66f000 rw-p 00000000 00:00 0 >>>>> 2b091d66f000-2b091d670000 ---p 00000000 00:00 0 >>>>> 2b091d670000-2b091d770000 rwxp 00000000 00:00 0 >>>>> 2b091d770000-2b091d771000 ---p 00000000 00:00 0 >>>>> 2b091d771000-2b091d871000 rwxp 00000000 00:00 0 >>>>> 2b091d871000-2b091d872000 ---p 00000000 00:00 0 >>>>> 2b091d872000-2b091d972000 rwxp 00000000 00:00 0 >>>>> >>>>> /proc/meminfo: >>>>> MemTotal: 49521668 kB >>>>> MemFree: 5596796 kB >>>>> Buffers: 294684 kB >>>>> Cached: 34205856 kB >>>>> SwapCached: 0 kB >>>>> Active: 12745128 kB >>>>> Inactive: 28516788 kB >>>>> Active(anon): 9280636 kB >>>>> Inactive(anon): 3090264 kB >>>>> Active(file): 3464492 kB >>>>> Inactive(file): 25426524 kB >>>>> Unevictable: 14420 kB >>>>> Mlocked: 14420 kB >>>>> SwapTotal: 0 kB >>>>> SwapFree: 0 kB >>>>> Dirty: 20392 kB >>>>> Writeback: 0 kB >>>>> AnonPages: 6776540 kB >>>>> Mapped: 6292204 kB >>>>> Shmem: 5604620 kB >>>>> Slab: 1828656 kB >>>>> SReclaimable: 1567928 kB >>>>> SUnreclaim: 260728 kB >>>>> KernelStack: 4648 kB >>>>> PageTables: 58800 kB >>>>> NFS_Unstable: 0 kB >>>>> Bounce: 0 kB >>>>> WritebackTmp: 0 kB >>>>> CommitLimit: 47045584 kB >>>>> Committed_AS: 46046480 kB >>>>> VmallocTotal: 34359738367 kB >>>>> VmallocUsed: 460188 kB >>>>> VmallocChunk: 34333703480 kB >>>>> HugePages_Total: 0 >>>>> HugePages_Free: 0 >>>>> HugePages_Rsvd: 0 >>>>> HugePages_Surp: 0 >>>>> Hugepagesize: 2048 kB >>>>> DirectMap4k: 8192 kB >>>>> DirectMap2M: 2080768 kB >>>>> DirectMap1G: 48234496 kB >>>>> >>>>> Memory: 4k page, physical 49521668k(5596796k free), swap 0k(0k free) >>>>> >>>>> * overcommit is turned off >>>>> >>>>> So the code heap was attempting to expand by 65kb at a fixed address. >>>>> There appears to be a 65kb mapping available between the end of the >>>>> current code heap mapping (2b091755c000-2b091869c000) and the next one >>>>> (2b09186ac000-2b091d66f000). There's about 30mb of free space left >>>>> (reserved, but uncommitted I take it) in the virtual space, so it's a >>>>> bit puzzling given there's ample physical mem available. Only thing I >>>>> can think of is expansion fails because it cannot get contiguous >>>>> space, but then I can't reconcile that with the mem mapping above. >>>>> >>>>> Does anyone have any ideas? >>>>> >>>>> Let me know if you need additional info from the hs_err file. >>>>> >>>>> Thanks >>>>> >>>>> >>>>> > From lev.priima at oracle.com Fri May 30 15:57:16 2014 From: lev.priima at oracle.com (Lev Priima) Date: Fri, 30 May 2014 19:57:16 +0400 Subject: [9] RFR (XS) 8044339: wrong comment for FilterSpuriousWakeups option In-Reply-To: <53881F1B.4090805@oracle.com> References: <53877D90.7090105@oracle.com> <53881F1B.4090805@oracle.com> Message-ID: <5388AA5C.2060207@oracle.com> Thanks for comments, please look at update: http://cr.openjdk.java.net/~iignatyev/lpriima/8044339/webrev.01/ Lev On 05/30/2014 10:03 AM, David Holmes wrote: > Hi Lev, > > On 30/05/2014 4:33 AM, Lev Priima wrote: >> Please review and help me with integration: >> >> diff -r 102506d9d873 src/share/vm/runtime/globals.hpp >> --- a/src/share/vm/runtime/globals.hpp Wed May 28 14:42:00 2014 +0400 >> +++ b/src/share/vm/runtime/globals.hpp Thu May 29 22:29:37 2014 +0400 >> @@ -1144,7 +1144,7 @@ >> \ >> product(bool, FilterSpuriousWakeups, >> true, \ >> "Prevent spurious or premature wakeups from object.wait >> " \ >> - "(Solaris >> only)") \ >> + "(Ignored for >> Windows)") \ >> \ > > Sorry to make life complicated but as I put in the bug report: > > The "Solaris" part dates from the days when Solaris meant "not > Windows". :) > > A bigger problem here is that the main part of the description: > > "Prevent spurious or premature wakeups from object.wait " > > may not match what people expect - it depends on whether "filter" > means to remove the item being filtered, or to keep only the item > being filtered. It would clearer if it read "When true prevents ..." > but even then it is not accurate as this only excludes spurious > wakeups from the OS pthread_cond[timed]wait implementation - the JVM > can still induce its own "spurious wakeups". So the better description > would be: > > "When true prevents OS-level spurious, or premature, wakeups from > Object.wait" > > But it should also be noted that where this flag is checked there is > an even more inaccurate comment: > > // The underlying Solaris implementation, cond_timedwait, admits > // spurious/premature wakeups, but the JLS/JVM spec prevents the > // JVM from making those visible to Java code. > > Ironically this comment appears in the linux and bsd versions but not > the Solaris one! Further it is incorrect - the JLS/JVMS does not > prevent spurious wakeups from being made visible to Java code. That > has always been the intent but it took until Java 5 before we got the > specification for Object.wait updated to make it clear. > > So really this is all legacy crud that should be removed along with > the other planned cleanups (ie removing the safe_cond_* functions and > WorkAroundNPTLTimedWaitHang) and the os_posix refactoring. > > --- > > Sorry to mess up your "starter" bug. > > David > > >> product(intx, NativeMonitorTimeout, -1, >> "(Unstable)") \ >> \ >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8044339 >> From vladimir.kozlov at oracle.com Fri May 30 19:35:15 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 30 May 2014 12:35:15 -0700 Subject: [8u20] RFR (S) 8043354: OptimizePtrCompare too aggressive when allocations are present Message-ID: <5388DD73.20806@oracle.com> 8u20 backport request. The fix was pushed into jdk9 week ago and nightly testing shows no problems. Changes from jdk9 applied to 8u without conflicts. http://cr.openjdk.java.net/~rasbold/8043354/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8043354 http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/729d5480f7db Thanks, Vladimir